Problem
I’d want to run a Linux command that compares two folders recursively and outputs just the file names that vary. This includes discrepancies in text and anything that is present in one directory but not the other.
Asked by barfoon
Solution #1
The following is taken from the diff man page:
Example command:
diff -qr dir1 dir2
The following is an example of output (which varies depending on the locale):
$ ls dir1 dir2
dir1:
same-file different only-1
dir2:
same-file different only-2
$ diff -qr dir1 dir2
Files dir1/different and dir2/different differ
Only in dir1: only-1
Only in dir2: only-2
Answered by John Kugelman
Solution #2
You can also use rsync
rsync -rv --size-only --dry-run /my/source/ /my/dest/ > diff.out
Answered by boksiora
Solution #3
If you only want a list of files in one directory, not their subdirectories, and only their file names, use the following command:
diff -q /dir1 /dir2 | grep /dir1 | grep -E "^Only in*" | sed -n 's/[^:]*: //p'
If you want to list all the files and directories that are different with their full paths in a recursive manner, type:
diff -rq /dir1 /dir2 | grep -E "^Only in /dir1*" | sed -n 's/://p' | awk '{print $3"/"$4}'
This allows you to run different commands on all of the files.
I could, for example, delete all the files and directories in dir1 but not in dir2:
diff -rq /dir1 /dir2 | grep -E "^Only in /dir1*" | sed -n 's/://p' | awk '{print $3"/"$4}' xargs -I {} rm -r {}
Answered by N D
Solution #4
One big disadvantage of using diff -qr old/ new/ is that it may overlook files in freshly formed directories. E.g. in the example below the file data/pages/playground/playground.txt is not in the output of diff -qr old/ new/ whereas the directory data/pages/playground/ is (search for playground.txt in your browser to quickly compare). I also posted the following solution on Unix & Linux Stack Exchange, but I’ll copy it here as well:
The best approach I could come up with for creating a list of new or modified files programmatically is to use rsync, sort, and uniq:
(rsync -rcn --out-format="%n" old/ new/ && rsync -rcn --out-format="%n" new/ old/) | sort | uniq
Let me give you an example: we want to compare two Dokuwiki versions to determine which files have been altered and which have been added.
We use wget to download the tars and extract them into the old/ and new/ directories:
wget http://download.dokuwiki.org/src/dokuwiki/dokuwiki-2014-09-29d.tgz
wget http://download.dokuwiki.org/src/dokuwiki/dokuwiki-2014-09-29.tgz
mkdir old && tar xzf dokuwiki-2014-09-29.tgz -C old --strip-components=1
mkdir new && tar xzf dokuwiki-2014-09-29d.tgz -C new --strip-components=1
As the following comparison of rsync and diff reveals, running rsync one method may overlook freshly produced files:
rsync -rcn --out-format="%n" old/ new/
produces the following result:
VERSION
doku.php
conf/mime.conf
inc/auth.php
inc/lang/no/lang.php
lib/plugins/acl/remote.php
lib/plugins/authplain/auth.php
lib/plugins/usermanager/admin.php
Running rsync only in one direction misses the newly created files and the other way round would miss deleted files, compare the output of diff:
diff -qr old/ new/
produces the following result:
Files old/VERSION and new/VERSION differ
Files old/conf/mime.conf and new/conf/mime.conf differ
Only in new/data/pages: playground
Files old/doku.php and new/doku.php differ
Files old/inc/auth.php and new/inc/auth.php differ
Files old/inc/lang/no/lang.php and new/inc/lang/no/lang.php differ
Files old/lib/plugins/acl/remote.php and new/lib/plugins/acl/remote.php differ
Files old/lib/plugins/authplain/auth.php and new/lib/plugins/authplain/auth.php differ
Files old/lib/plugins/usermanager/admin.php and new/lib/plugins/usermanager/admin.php differ
Running rsync in both directions and sifting the output to eliminate duplicates reveals that the directory data/pages/playground/ and the file data/pages/playground/playground.txt were initially overlooked:
(rsync -rcn --out-format="%n" old/ new/ && rsync -rcn --out-format="%n" new/ old/) | sort | uniq
produces the following result:
VERSION
conf/mime.conf
data/pages/playground/
data/pages/playground/playground.txt
doku.php
inc/auth.php
inc/lang/no/lang.php
lib/plugins/acl/remote.php
lib/plugins/authplain/auth.php
lib/plugins/usermanager/admin.php
The following arguments are passed to rsync:
The rsync output (list of files) in both directions is merged and sorted with sort, and then this sorted list is condensed by deleting all duplicates with uniq.
Answered by iolsmit
Solution #5
To get just the filenames on my Linux system
diff -q /dir1 /dir2|cut -f2 -d' '
Answered by gerardw
Post is based on https://stackoverflow.com/questions/6217628/diff-to-output-only-the-file-names