Comparing Two Folders for Duplicates

commissarmo

Distinguished
Jan 5, 2010
36
1
18,590
This simple task has been extremely frustrating. I have a unique case where I require a folder sync-ed to another folder, but on one side there is a folder structure with subfolders, on the other I have the folder "Flat" (it strips all the files out of the folders).

I have done this with software called Syncovery, and it works (mostly).

Unfortunately, it does NOT seem to copy every file, as there is a discrepancy (of about 100 files) between the source and destination, with the total being 2900 files or so in the source, and about 100 less in the destination.

The program provides a log, but it only threw about 10 conflicts, clearly not enough to explain the discrepancy (Notes: when one uses flat files, inevitably some files share the same name - so I renamed all such files, additionally I have told the program to ignore certain temp files (e.g. thumbs.db)).

---
1. Nevertheless I now want to compare the two folders and see which files are present in the source but not the destination. This has proven immensely difficult. I have tried 3 different file comparison tools and 2 explorer replacements, and they all exhibit the exact same behavior:

They list all the files in one folder as different from all the files in the other - I can only imagine this is because one folder is the flattened version of the other one, and the programs are somehow NOT comparing the exact file names (despite that being the option I chose), but are being confused by the file pathnames - which are different of course.

2. Does anyone have any prior experience with this sort of thing?

3. Additionally; I have realized that perhaps the best way to deal with such cases is to just copy the folder initially with something like TeraCopy where I can verify the exact files, and THEN set up a sync software program which will handle new files.

Thanks in advance...
 

commissarmo

Distinguished
Jan 5, 2010
36
1
18,590
Thanks - I tried Beyond Compare... and ran the file-only compare binary comparison between my two folders (one is the folder-divided one, the other is the 'flat file version' of that same data which my sync software created).

NB: One of the folders is local, the other is mounted on a WebDav share...

Unfortunately, it still reports that ALL the files are different (with a single exception I'll mention). I've tested it several times - opening different random files, and have found them to be exactly the same (Windows reported size for instance). The dates modified are different (since the WebDav files have the date of transfer, and the originals have date of creation of last modified).

1. But I don't think the dates should matter if its doing a binary content comparison?

2. *** The software DID mark one file as identical on both the WebDav Flat-folder and the Local Tree-Folder. It was a test document I created but saved in the parent folder root, OUTSIDE THE FOLDER TREE. I created another to test this, and it ALSO identified it in both places as THE SAME.

---

***Clearly, something about the process of "flattening" the files changes them. The files in the root directory of the folder being sync-ed report being identical on both sides. But all the files INSIDE a sub-folder, which the sync software then strips out to 'flatten it' are NOT reported as identical by the binary comparison.

Is there any explanation for this? And... perhaps, a solution?

 
Solution