Copying Data

Copy data with verification.

Install Utilities.
apt install md5deep rsync
Archive Copying Data.
cd /X/
rsync -avxhHAX {DIRECTORY} /Y


  • -a archive (-rlptgoD).

  • -v verbose.

  • -x don’t cross FS boundaries.

  • -h human readable.

  • -H preserve hard links.

  • -A preserve ACL’s.

  • -X preserve extended attributes.

Create Hashfile of All Original Files and Sort.
cd /X/
md5deep -l -r -e {DIRECTORY} | sort > /tmp/{DIRECTORY}.md5


  • -l use relative file paths.

  • -r recursive.

  • -e display progress indicator (not written to hashfile).

Verify copied files with rsync.
cd /Y/
md5deep -l -r -e {DIRECTORY} | sort > /tmp/{DIRECTORY}2.md5
md5sum /tmp/{DIRECTORY}.md5 /tmp/{DIRECTORY}2.md5
diff /tmp/{DIRECTORY}.md5 /tmp/{DIRECTORY}2.md5


  • This basically verifies that both hashfiles are the same (and therefore all files are the same).

  • md5sum will tell you if there is a difference in the hashfiles.

  • diff will list the files that are actually changed.

Verifying Copied Files Across OS’s are Accurate.
cut -f 1 -d ‘ ‘ {SOURCE}.md5 > {SOURCE}-hash-only.md5
grep -v -f {SOURCE}-hash-only.md5 {TARGET}.md5


This removes path differences, and only compares source hashes to destination hashes. Only non-matching lines (e.g. those hashes that don’t match) should be printed.


  1. md5deep usage tricks

  2. md5deep validating moved files between directories

  3. Using rsync to backup hard drives

  4. rsync common commands