Strange data corruption with large files

freelunch

Distinguished
Jan 25, 2009
5
0
18,510
I have 4 machines.
1 windows 2000 server with sata-ii drives
1 xp laptop sp2
1 xp laptop sp3
1 vista laptop sp1
All avg pro v8

When I copy a 10gb file from any machine to any other machine(copy/xcopy) and follow this with a "fc /b" all copies are identical.

However.... if I use ftp, rsync, or any other delta copying method I always get errors from "fc /b", usually 2 bits (02 -> 00) sometimes a few more.

When I overwrite the same file with copy /b or xcopy or even xxcopy, no problems, 100% ok.

Since this is happening to 4 different OS's and hardware systems I am excluding bad HD's and ram.

I have tried this with a smaller file like 2gb, no problems here.

I've been running tests for days now, recompiling rsync and other delta tools, added extra layers that run a md5 and sha1 checksum (who always report that the data written to disk matches both checksums) and still fc detects at least 2 wrong bits. xxcopy again and all is well.. slightly going insane here...

Any ideas?
 

freelunch

Distinguished
Jan 25, 2009
5
0
18,510
Nobody any ideas? could someone do a test?
Ie. take a DVD and copy this as a ISO to PC-1
take another DVD and copy this as a ISO to PC-2
Install something like deltacopy and (r)sync one ISO with the other between pc's.
Then run a fc /b between pc's ISO's.
 

Flakes

Distinguished
Dec 30, 2005
46
0
18,590
if the file still works then the extra two bits are probably just header information that the ftp program has put there.... don't worry about it.
 

freelunch

Distinguished
Jan 25, 2009
5
0
18,510
I wish... the bits are located at offsets like ecd09981 ie. anywhere beyond the 2.5g range. and no its not a 16/24bit limit problem.
 

freelunch

Distinguished
Jan 25, 2009
5
0
18,510
Yep, confirmed, its windows disk compression doing this on all platforms, so not just a bug in home server, compress files larger then 1gb and you risk corruption... it took a few days to test this with 400gb, decompressing and repeat.