Rsync and take name/path changes into account

Question

I maintain a large folder, and this folder is regularly rsync'ed to another folder far away.

Sometimes I change a name on my local folder, and then the next time I rsync to the remote folder the whole file needs to be sent, even though the only thing that needs to be sent really is the information that the filename has changed.

Is there any way to get rsync to figure out name/path changes?

I looked through here: http://linux.die.net/man/1/rsync, and couldn't find anything like that.

it would be really brilliant if there were a program like that though. — john-jones, Aug 08 '13 at 11:32
There is. It's called 'git' or 'svn' or 'mercurial' or even 'cvs' — Shadur-don't-feed-the-AI, Aug 08 '13 at 13:52

score 1 · Answer 1 · answered Aug 08 '13 at 11:49

1

Unison is more likely to do what you want than raw rsync. It uses the rsync protocol to synchronize files.

answered Aug 08 '13 at 11:49

BillThor

8,965

1

thanks for pointing that out, unison does seem to get questionable reviews though: http://superuser.com/questions/65548/how-reliable-is-unison-did-it-ever-ruin-your-data – john-jones Aug 08 '13 at 12:21
1

@HermannIngjaldsson Unison isn't maintained, and that is a concern. However, for this use case, it is better than rsync, because it detects moves with no fuss. – Gilles 'SO- stop being evil' Aug 08 '13 at 22:45
I have been using Unison now but it seems very slow and am gonna go back to Rsync. Most reviews seem to complain about how slow it is and it seems many stop using it eventually. http://hans.fugal.net/blog/2007/02/06/on-rsync-and-unison/ – john-jones Aug 13 '13 at 13:45

score 1 · Accepted Answer · edited Aug 08 '13 at 18:01

If both the local and remote server support the ability to do hard links you can use this trick to get what you want. The method is discussed in this blog post, titled: Detecting File Moves & Renames with Rsync.

General steps

normal sync

$ rsync -avHP --delete-after ~/family/Photos remotebox:backups

now make some changes
```
$ cd ~/family
$ cp -rlp Photos Photos-work
```
The cp is done very quickly when its switches are: copy directories *R*ecursively + *L*ink files instead of copying + *P*reserve mode, ownership and timestamps (for non-hardlinked content such as directories)

Do the reorganization in the Photos-work directory: you can rename, move, add and delete any files. But DON’T TOUCH the tree in Photos, this directory (with the same sets of paths on both machines), will allow rsync to quickly find the data to clone under Photos-work on the remote machine.
When you’re done reorganizing, you run this:
```
$ rsync -avHP --delete-after --no-inc-recursive ~/family/Photos ~/family/Photos-work remotebox:backups
```
By transferring both trees at once and by turning off incremental recursion, rsync collects all hard-links before it transfers anything. It is now able to reconstruct Photos-work on the remote machine IN SECONDS. Next you finalize by:
```
$ mv Photos Photos-OLD
$ mv Photos-work Photos
```
And you do this on both local and remote machines. You can keep the OLD directory around for as long as you want, the space it uses is usually negligible.

this should so be added as an option to Rsync's behavior. Would make mirroring large folders far apart, far easier. — john-jones, Aug 08 '13 at 13:26

Rsync and take name/path changes into account

2 Answers2

General steps