101

I want to transfer files (a music folder) between two Linux computers. After searching for the best way to do this, I've seen that there are lots of ways of doing this. I know this has been asked a lot, everywhere and all the time. The main problem with this is that there is no clear, recent consensus on one best way to do this task in 2011 for Linux beginners (even depending on some parameters).

So in the spirit of the Stack Exchange websites, I want this not to be related to my particular situation, but more of a guide to others as well on how to transfer files between two Linux computers over a local network. I think a wiki would be useful for many.

Here's what I found so far:

  • ssh
  • sshfs
  • scp
  • sftp
  • nfs
  • samba
  • giver

What is the easiest? Most flexible? Simplest? Best solution? What are the pros and cons of each? Are there other (better) options? What are the parameters in choosing the best method (solution might depend on number of files, filesize, easiness vs. flexibility, ...)?

slhck
  • 228,104
jonallard
  • 1,356
  • 2
    Could anyone explain where rsync comes to play in all of this? – Konerak Aug 22 '11 at 08:39
  • jonallard, please don't add the answers to the question (doesn't really make sense to do this, does it?) -- if you feel some answers need additional info, you can suggest edits on them, or create your own answer which summarizes everything! – slhck Aug 22 '11 at 10:19

12 Answers12

84

In a Linux environment, for both security and ease of use, ssh is the best way to go. SSH, SSHFS, SCP, and SFTP as you list are all just different services built on top of the SSH protocol. SCP is very easy to use, it works just like CP but you can provide user and machine names in the path. So, we might do a CP like cp ~/music/ ~/newmusic/, but we could just as easily do scp ~/music/ user@host:~/newmusic to send it to the computer named host. That's it - we don't need to set anything up. You'll be prompted for the account password on the other machine if you don't have certificate or some other authentication set up (scp shares those settings with ssh, of course).

SFTP is a tool that makes it easy to do a lot of operations on a remote file system - it works just like FTP, but it runs through SSH so it's secure and requires only an SSH server. man sftp will tell you all about how to use it. I don't use SFTP just to move a folder between two machines, it's more useful when you have a lot of operations to do, like if you're rearranging files on another computer.

SSHFS just extends SFTP in to a file system: it allows you to mount a virtual host in to your file system, so the network stuff happens totally transparently. SSHFS is for semi-permanent setups, not just a one-time file transfer. It takes some more effort to get set up, which you can read about on the project website.

If you need to work in a mixed-OS environment, Samba becomes your next best bet. Windows and OS X support Samba completely automatically, and Linux does as well although it's sometimes rough to use.

jcrawfordor
  • 16,329
  • 5
    Exactly the kind of answer I was wishing for: complete, exhaustive, detailed, to the point. – jonallard Aug 22 '11 at 05:12
  • 3
    One thing though, for scp to work, do we need to set up some kind of ssh server, listener or unblock something on the other side? I'm getting "Connection refused" errors. – jonallard Aug 22 '11 at 05:28
  • 3
    scp uses ssh, so it will work if SSH works. This means, of course, that you need to have an SSH server running (default in every linux distro I'm aware of) and a connection must be possible (firewalls, NAT, etc must have the appropriate exceptions). – jcrawfordor Aug 22 '11 at 05:42
  • 9
    Apparently openssh-server has to be installed in Ubuntu Natty. – jonallard Aug 22 '11 at 05:51
  • 4
    Note that ssh uses encryption, which will cause some additional overhead. If the computers involved have fairly slow CPUs, this might make a difference. In that case netcat or similar (see Caspar's answer) might be preferrable. Of course only if you don't actually need encryption (in a protected LAN). – sleske Aug 22 '11 at 07:25
  • 2
    Protip: if you set up SSH connection sharing, tab-completion will work for files on the remote host when using SCP. True computer magic! – jcrawfordor Aug 22 '11 at 07:29
  • most linux distros have a package called openssh-server (like jonallard said, but its not only for ubuntu) and filezilla will work fine if you want a gui tool just connect on port 22 or use sftp:// – HTDutchy Aug 22 '11 at 14:36
82

My personal favorite for cases where security doesn't matter is netcat + tar:

To send a directory, cd to inside the directory whose contents you want to send on the computer doing the sending and do:

tar -cz . | nc -q 10 -l -p 45454

On the computer receiving the contents, cd to where you want the contents to appear and do:

nc -w 10 $REMOTE_HOST 45454 | tar -xz

Replace $REMOTE_HOST with ip / hostname of computer doing the sending. You can also use a different port instead of 45454.

What's actually happening here is that the 'receiving' computer is connecting to the sending computer on port 45454 and receiving the tar'd and gzip'd contents of the directory, and is passing that directly to tar (and gzip) to extract it into the current directory.

Quick example (using localhost as a remote host)

Computer 1

caspar@jumpy:~/nctest/a/mydir$ ls
file_a.txt  file_b.log
caspar@jumpy:~/nctest/a/mydir$ tar -cz . | nc -q 10 -l -p 45454

Computer 2

caspar@jumpy:~/nctest/b$ ls
caspar@jumpy:~/nctest/b$ nc -w 10 localhost 45454 | tar -xz
caspar@jumpy:~/nctest/b$ ls
file_a.txt  file_b.log
Caspar
  • 2,173
  • 1
  • 16
  • 17
  • kinda reminds me of sendnet and recnet from Novel's ipx services of the olden days... – sum1stolemyname Aug 22 '11 at 12:33
  • 6
    netcat + bzip2 may be a little faster on slow connections ## Sending server # cat file.txt | bzip2 -c | nc -l 1234 ## Receiving server # nc $sending_ip 1234 | bzip2 -cd > file.txt – shantanuo Aug 24 '11 at 03:20
  • @Caspar: How about editing this answer to suggest bzip2 or lzma compression? – einpoklum Nov 23 '13 at 18:59
  • 5
    the -q option indicates that you are using openbsd-netcat, while gnu-netcat is also quite common (default in Arch Linux). Could you extend your answer to include the syntax of gnu-netcat? – Sebastian Jan 06 '14 at 20:48
  • 1
    The current nc man says about the -l option: "It is an error to use this option in conjunction with the -p, -s, or -z options", but strangely it doesn't throw an error when using it. I think using 'nc -l 45454' should work just as well. – Claudiu Sep 25 '14 at 07:58
  • 1
    I like this solution because it doesn't require you to install and configure any form of file server. Just issue a command and ready to go. Also it's the fastest, only limited by the network connection. – Calmarius Dec 23 '16 at 09:31
  • since ssh is almost universally available, I don't know why anyone would prefer this netcat method. Time saved for extra compression of a big file is probably lost in researching the syntax or the extra keystrokes! – Jack Wasey Jul 19 '18 at 13:51
  • 1
    @JackWasey - one can have two systems in a LAN without ssh set up. – adamczi Sep 05 '18 at 16:23
  • @adamczi true. Is netcat more likely to be available than ssh? I guess ssh does need a server process, but usually this is available with zero or minimal config on default linux installations I'm familiar with. – Jack Wasey Sep 05 '18 at 19:56
  • @JackWasey true, I meant netcat is preinstalled on Ubuntu and to run ssh server you need to install eg. openssh-server package (what is of course not demanding, but is leaves you with a thing you must remember about security-wise) – adamczi Sep 05 '18 at 21:10
  • 2
    @JackWasey For a one-off transfer this is easier than installing the SSH server, editing the SSH config file to allow connections (it may be disabled by default), setting up authentication and then still having to research the syntax because I don't know how to use scp anyway (it doesn't look complicated at all so I should probably learn it though) – Heath Mitchell Oct 10 '22 at 17:22
  • Note to Debian12 users you may need to do sudo apt-get install -y netcat-traditional or apt-get install -y netcat-openbsdto get thenc` command. (I believe for Debian the first is the right one; I should know soon.) – John Smith Nov 13 '23 at 02:52
21

For one time moves, scp is recommended.

But if you find that this dir may work and you need to move it many times to keep the other position updated then you can use rsync (with ssh).

Since rsync has a lot of arguments I usually put it in a small shell so I get it right (every time). The idea is to only send things that has changed since the last time it ran.

#!/bin/bash

user="nisse"
host="192.168.0.33"

echo "Sync: /home/media/music/"
rsync --archive --delete -v --progress -e "ssh -l $user " /home/media/music/ $host:/home/media/music/

This will move a dir called "/home/media/music/" from the local computer to the the pc called 192.168.0.33, using user "nisse". And delete anything on the target that doesn't exist on the local pc.

Glorfindel
  • 4,099
Johan
  • 5,435
  • This seems very promising (and easily reuseable) but I've got dirs and files with spaces and I'm getting rsync error: syntax or usage error (code 1) at main.c(1348) [sender=3.1.1] -- any suggestions? – Torben Gundtofte-Bruun Aug 22 '15 at 21:54