If you are playing the Steam version with an Internet connection, you can just let Steam do all that work for you by enabling cloud saving for your character.

Otherwise, your characters are stored in:
- Windows:
%USERPROFILE%\Documents\My Games\Terraria\Players
- Mac:
~/Library/Application Support/Terraria/Players
- Linux:
~/.local/share/Terraria/Players
However, if you're using Steam cloud saves, the files will be in a different location.
<Steam directory>/userdata/<Steam3 ID>/105600/remote/players
To copy a character, you will need both the .plr file and the folder that are named like your character. Technically, only the file is necessary, but the folder contains your minimap data.
.plr. I think it's the previous version of the save file; if the save becomes corrupted for some reason, Terraria tries to load the backup. In any case, it's not needed. – a cat Jun 13 '11 at 15:56