Cygwin rsync slow when backing up 2 external hard drives

Question

I wrote a shell script to backup external hard drives through my laptop. I basically connected the external hard drives to the laptop's USB ports and ran the following command in a script to backup 1 HDD to another hard drive.

rsync -arvv "$flg" --log-file="$LOGFILE" "$srcdir" $BAKUP_DIR 2> "$ERRFILE" $ rsync --version rsync version 3.0.9 protocol version 30 

flg is either --delete, or spaces, depending on how I want to back up the directory. I believe that this is slow for some reason I can not understand. I think that when backing up files are unnecessarily transferred. I see that parts of the files are sent (and grow) when viewed through Explorer and wonder why this is so.

and the log file ...

  grep -C 15 delta 2011-12-23-37_bk.log 2011/12/23 14:28:34 [8712] total: matches=0 hash_hits=0 false_alarms=0 data=2133674141 2011/12/23 14:28:35 [8712] sent 2133939896 bytes received 772 bytes 1854794.15 bytes/sec 2011/12/23 14:28:35 [8712] total size is 3901493999 speedup is 1.83 2011/12/23 14:28:35 [5072] building file list 2011/12/23 14:28:39 [5072] delta-transmission disabled for local transfer or --whole-file 

Is this normal or is there something I am missing?

Answer 1

I encountered the same problem on the system when I recently installed Cygwin: rsync was too slow. Using the --progress option, I could see that rsync only transferred files about 2/3 MB / s between an external USB 3.0 and an internal drive.

In a similar system, the same work worked as expected (60/70 MB / s).

After many attempts, I realized that “rsync -version” was 3.0.9 on the system, I just installed Cygwin and 3.0.8 on the system because it worked fine. So 3.0.9 seems to have broken something.

I copied the rsync.exe (v3.0.8) file from one system through rsync.exe (v3.0.9) to another system and now the files are transferred at the expected speed (60/70 MB / s).

Answer 2

I had exactly the same problem and I also decided to install rsync version 3.0.8. But instead, if you copy the .exe file, you can simply restart the cygwin installation. In the package manager, the search for “rsync” in the configuration still contains the installation option 3.0.8, which means that you can easily downgrade from 3.0.9 to 3.0.8. From there, everything returns to normal and max. speed.

Answer 3

I also had this problem with rsync 3.0.9 on Cygwin64 (transfer ~ 3 MB / s).

Going back to 3.0.8 is not included in Cygwin64 because it is not in the repo.

However, I downloaded rsync 3.1.0 and created it myself (I just need gcc to make it installed), and this has improved significantly.

3.1.0 built itself, runs ~ 35 MB / s. Windows Explorer was faster (around 50 MB / s), but I'll take it!

Answer 4

I have the same problem with rsync ~ 4 MB / s transfer rate. This speed is reported using both the resource monitor and -P to rsync. He was instructed to copy ~ 200 GB backup files from one drive to another, and for several years he worked for about 1.5 hours to copy them. Not very fast, but good enough for me. Something was supposed to break recently and, no doubt, due to some updating of the Windows system, since I did not update Cygwin after a few years. Work began to take ~ 14 hours, and this made me look at it closer. I upgraded the x32 Cygwin that I used to the latest version (rsync version 3.1.0 version 31) and installed the x64 version of Cygwin to find out it doesn't matter. No.

Thinking that it could be one or the other hard drive, which could deteriorate, I made sure that both of them are completely defragmented and cleaned of any garbage. Then I replaced the “rsync -a” command that I used with “cp -au” and measured the throughput. Using cp, it reads and writes at a speed of ~ 100 MB / s, which supports hardware, and copies ~ 200 GB in 30 minutes. So something in rsync is not quite right.

My workaround is to give up using rsync in favor of cp. Most likely, just delete the entire directory tree on the target computer first and try cp copy completely new than for rsync to perform its task in order to synchronize these two directories. There is probably an even better option, but it works for me.