Comparison of two large directories

Question

I have a large directory containing only material in CS and Math. It has a size of over 16 GB. Types are text, png, pdf and chm. I currently have two branches: a branch of my brother and mine. The initial files were the same. I need to compare them. I tried using Git, but there is a long download time.

What is the best way to compare two large directories?

[Mixed solution]

• Make "ls -R> different_files" in both directories [1]
• "sdiff <(echo file1 | md5deep) <(echo file2 | md5deep)" [2]

What do you think? Any flaws?

[1] thanks to Paul Tomblin [2] Many thanks to all the guys!

Answer 1

2 /:

, . .

.

2 , , . 2 , .

. MD5, SHA, CRC,... , .

2 /:

, , , diff -r windiff .

Answer 2

fslint: . - "". : FSlint . lint - , . , , . , , , . , . , FSlint, .

Answer 3

md5deep, md5sum .

.

Answer 4

, , , ? :

• "ls -R" , .

• "rsync -n" , , rsync , . (-n , rsync, , , -n)

Answer 5

diff ing, md5sum * | sort

, /

Answer 6

I know that this question has already been answered, however, if you yourself do not write such a tool, there is a very well working open source project called tardiff , available on sourceforge, which basically does exactly what you want, and even supports the automatic creation of patches (in tar format explicitly) to account for differences.

Hope this helps