Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

Hi,

Take these 3 concepts and then look at the implementation/performance:

( 3 Concepts for reading/writing with harddisks )

Concept 1:

Seek, Read, Read, Read, Read, Read, Etc

Results: FAST

Concept 2:

Seek, Write, Write, Write, Write, Write, Etc

Results: FAST

Concept 3:

Seek, Read, Write, Read, Write, Read, Write,

Results: SLOW ???

The pseudo code is like:

Seek( 0 ); // offset 0

For I:=0 to FileBlocks-1 do // number of 4KB blocks in file.
begin
if Random(2) = 0 then
Read( 4 KB )
Else
Write( 4 KB );
end;

The original concept is:

Do a seek once.

Then read or write a block of data. The head is automatically forwarded to
the next block. So no extra seek is needed.

For concept 1 and concept 2 this works just fine and gives good performance.

However concept 3 has very bad performance.

Is this a software issue ? ( Windows XP )

Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
can't work together like this and an extra seek is needed ? )

Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )

Bye,
Skybuck.

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

On Wed, 14 Jul 2004 17:53:52 +0200, "Skybuck Flying"
<nospam@hotmail.com> wrote:

>Hi,
>
>Take these 3 concepts and then look at the implementation/performance:
>
>( 3 Concepts for reading/writing with harddisks )
>
>Concept 1:
>
>Seek, Read, Read, Read, Read, Read, Etc
>
>Results: FAST
>
>Concept 2:
>
>Seek, Write, Write, Write, Write, Write, Etc
>
>Results: FAST
>
>Concept 3:
>
>Seek, Read, Write, Read, Write, Read, Write,
>
>Results: SLOW ???
>
>The pseudo code is like:
>
>Seek( 0 ); // offset 0
>
>For I:=0 to FileBlocks-1 do // number of 4KB blocks in file.
>begin
> if Random(2) = 0 then
> Read( 4 KB )
> Else
> Write( 4 KB );
>end;
>
>The original concept is:
>
>Do a seek once.
>
>Then read or write a block of data. The head is automatically forwarded to
>the next block. So no extra seek is needed.
>
>For concept 1 and concept 2 this works just fine and gives good performance.
>
>However concept 3 has very bad performance.
>
>Is this a software issue ? ( Windows XP )
>
>Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
>can't work together like this and an extra seek is needed ? )
>
>Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )
>
>Bye,
> Skybuck.
>


I may not know the answer but feel an important question might be
"What is very bad performance?", compared to good performance...
numbers are our friends.

Could it simply be that you're switching back and forth with data
flow so the caching (on the drive) isn't effective?

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

Actually, what is happening is that the heads on the platter need to
reconfigure themselves each time you read or write. It is a hardware
limitation. By reading, the head can stay in the same location, but
once it is told to write, the head has to position itself to change the
magnetism of the sector of the platter it is on. Then once you request
to read again, the head has to move again. It is the same as seek time
when you have two files on different partitions it is going to take
longer to find those files than if they were right next to each other.

Nathan McNulty

kony wrote:

> On Wed, 14 Jul 2004 17:53:52 +0200, "Skybuck Flying"
> <nospam@hotmail.com> wrote:
>
>
>>Hi,
>>
>>Take these 3 concepts and then look at the implementation/performance:
>>
>>( 3 Concepts for reading/writing with harddisks )
>>
>>Concept 1:
>>
>>Seek, Read, Read, Read, Read, Read, Etc
>>
>>Results: FAST
>>
>>Concept 2:
>>
>>Seek, Write, Write, Write, Write, Write, Etc
>>
>>Results: FAST
>>
>>Concept 3:
>>
>>Seek, Read, Write, Read, Write, Read, Write,
>>
>>Results: SLOW ???
>>
>>The pseudo code is like:
>>
>>Seek( 0 ); // offset 0
>>
>>For I:=0 to FileBlocks-1 do // number of 4KB blocks in file.
>>begin
>> if Random(2) = 0 then
>> Read( 4 KB )
>> Else
>> Write( 4 KB );
>>end;
>>
>>The original concept is:
>>
>>Do a seek once.
>>
>>Then read or write a block of data. The head is automatically forwarded to
>>the next block. So no extra seek is needed.
>>
>>For concept 1 and concept 2 this works just fine and gives good performance.
>>
>>However concept 3 has very bad performance.
>>
>>Is this a software issue ? ( Windows XP )
>>
>>Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
>>can't work together like this and an extra seek is needed ? )
>>
>>Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )
>>
>>Bye,
>> Skybuck.
>>
>
>
>
> I may not know the answer but feel an important question might be
> "What is very bad performance?", compared to good performance...
> numbers are our friends.
>
> Could it simply be that you're switching back and forth with data
> flow so the caching (on the drive) isn't effective?

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

"Skybuck Flying" <nospam@hotmail.com> wrote in message
news:cd3kli$4k5$1@news5.tilbu1.nb.home.nl...

> Do a seek once.

Seeks to a particular track and NOT a particular block of data.

> Then read or write a block of data.

When the head settles and certain switching times have passed and the
appropriate data block rotates under the head.

> The head is automatically forwarded to
> the next block.

There no kind of automatic "forwarding". The disk however does tend to keep
spinning at a constant angular velocity. Often the next data the head
encounters is the next data in the file if it's a sequential file.

> So no extra seek is needed.

Huh?

> For concept 1 and concept 2 this works just fine and gives good
performance.
>
> However concept 3 has very bad performance.

Huh, it's just how disks work.

> Is this a software issue ? ( Windows XP )

WHAT?

> Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
> can't work together like this and an extra seek is needed ? )

There are NOT two heads but just a single head per disk surface. That head
both reads an writes but can do only one or the other at a given instant.
There is a finite switch time between read and write mode. During that
switch time the disk continues to spin.

> Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )

What issue?

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

"Nathan McNulty" <525676@betaweb.com> wrote in message
newsauKdFeaEHA.1048@tk2msftngp13.phx.gbl...
> Actually, what is happening is that the heads on the platter need to
> reconfigure themselves each time you read or write.

Not really.

> It is a hardware
> limitation.

What limitation is that exactly?

> By reading, the head can stay in the same location, but
> once it is told to write, the head has to position itself to change the
> magnetism of the sector of the platter it is on.

That's false.

> Then once you request
> to read again, the head has to move again.

Not necessarily.

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

1. Yes really. You cannot read and write at the same time. The command
has to be either read or write.

2. Hardware limitation that it cannot read and write at the same time.

3. From what I have seen on my old harddrives I have taken apart and
what I have read, there is a tip on the head that prevents the magnetic
part from affecting nearby sectors. I was under the impression these do
not always stay in the same position but are required to move each time
a write process is initiated.

4. Again, the tips would have to move back to their resting position.
Maybe this is just old hardware (especially since my books are all very
old) or maybe I've read it wrong, but that was my understanding of how
that part worked.

Nathan McNulty

Ron Reaugh wrote:

> "Nathan McNulty" <525676@betaweb.com> wrote in message
> newsauKdFeaEHA.1048@tk2msftngp13.phx.gbl...
>
>>Actually, what is happening is that the heads on the platter need to
>>reconfigure themselves each time you read or write.
>
>
> Not really.
>
>
>>It is a hardware
>>limitation.
>
>
> What limitation is that exactly?
>
>
>> By reading, the head can stay in the same location, but
>>once it is told to write, the head has to position itself to change the
>>magnetism of the sector of the platter it is on.
>
>
> That's false.
>
>
>> Then once you request
>>to read again, the head has to move again.
>
>
> Not necessarily.
>
>

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

"Nathan McNulty" <525676@betaweb.com> wrote in message
news:%23G%23AUyeaEHA.3524@TK2MSFTNGP12.phx.gbl...
> 1. Yes really. You cannot read and write at the same time. The command
> has to be either read or write.
>
> 2. Hardware limitation that it cannot read and write at the same time.

Not really, a logical/design fact.

> 3. From what I have seen on my old harddrives I have taken apart and
> what I have read, there is a tip on the head that prevents the magnetic
> part from affecting nearby sectors. I was under the impression these do
> not always stay in the same position but are required to move each time
> a write process is initiated.

Not true for anything recent.

> 4. Again, the tips would have to move back to their resting position.

No tips.

> Ron Reaugh wrote:
>
> > "Nathan McNulty" <525676@betaweb.com> wrote in message
> > newsauKdFeaEHA.1048@tk2msftngp13.phx.gbl...
> >
> >>Actually, what is happening is that the heads on the platter need to
> >>reconfigure themselves each time you read or write.
> >
> >
> > Not really.
> >
> >
> >>It is a hardware
> >>limitation.
> >
> >
> > What limitation is that exactly?
> >
> >
> >> By reading, the head can stay in the same location, but
> >>once it is told to write, the head has to position itself to change the
> >>magnetism of the sector of the platter it is on.
> >
> >
> > That's false.
> >
> >
> >> Then once you request
> >>to read again, the head has to move again.
> >
> >
> > Not necessarily.
> >
> >

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

"Nathan McNulty" <525676@betaweb.com> wrote in message
newsauKdFeaEHA.1048@tk2msftngp13.phx.gbl...
> Actually, what is happening is that the heads on the platter need to
> reconfigure themselves each time you read or write. It is a hardware
> limitation. By reading, the head can stay in the same location, but
> once it is told to write, the head has to position itself to change the
> magnetism of the sector of the platter it is on. Then once you request
> to read again, the head has to move again. It is the same as seek time
> when you have two files on different partitions it is going to take
> longer to find those files than if they were right next to each other.

Yes, a hardware limitation seems to make more sense.

I changed the code to this pseudo code, to illiminate the randomness and
make it regular switching:

if BooleanRead then
begin
Read 4 KB
BooleanRead := false;
end else
begin
Write 4 KB
BooleanRead := true;
end;

This will cause regular switching like:

Seek, Read, Write, Read, Write, Read, Write, etc

Instead of randomness like:

Seek, Read, Read, Write, Read, Write, Write, etc

My harddisk performs as follows for these 3 concepts (with the updated
concept 3 )

Concept 1 ( Reading )

It achieves 8 MByte/Sec to 12 MByte/Sec for reading a 36 MB file. ( with 4
KB block buffer )

After a few seconds the harddisk led stops burning. The speed increases to
100 or 180 MByte/Sec.

This is ofcourse only possible because it's fetched from some RAM I think. I
am not sure if it is coming from the Harddisk Read Cache or Windows XP file
cache... or whatever it's called. My guess would be Windows XP's file cache.

Concept 2 (Writing )

This test is very fluctating... sometimes 600 Kbyte/sec at worst...
sometimes a peak to 36 MByte/Sec.

Mostly it's 2 MByte/Sec with some peaks to 4 MByte/Sec.

My file system is pretty fragmented though.

Concept 3 ( Reading / Writing )

Well this is also pretty weird.

Sometimes it's 80 KByte/sec to 400 KByte/Sec.

Now it's 1 to 2 MByte/Sec

I suspect the small file size of only 36 MByte has to do with that.

So I will now test again on 100 MB file, to prevent any caching or to detect
it... possibly memory jump in task manager.

Concept 1 (Reading)

This time the speed is pretty constant. 8 MByte/Sec... with some lower peaks
to 6MByte/Sec and a few high peaks to 9 MByte/Sec.

But no more caching apperently in windows xp...

8 MByte/Sec or any multiple of MByte/Sec is what I would call good
performance.

This is in the light of my file transfer tool and 100 Megabit network cards
which are common nowadays.

100 Megabit is roughly 8 MByte/Sec or so... give or take a few...

Not that my file transfer tool reaches these speeds just... it has problems
with harddisk =D

One more artificact of my benchmark programs after stop the test it takes a
while stop... probably because of the for loop... no big deal.

Concept 2 ( Writing )

Well this time writing was a lot thougher... 1 MByte/Sec to max 2
MByte/Sec... the program is less responsive than the read program... but
code is almost the same... and multi threaded. But not using async stuff...
just sync stuff At least I think so However async stuff aint that
great either
Since I wrote a program to do async stuff as well. So never mind that

Now comes the big test, since it's getting late

Concept 3 (Reading and Writing, with regular switches )

Oh my... now that's what I call bad performance !

The first 10 seconds it was 80 Kbyte/Sec... the next 20 seconds 160
Kbyte/sec.

Max was 200 KByte/sec.

So it seems switching from read to write to read to write requires extra
seeking etc...

Maximum seek time for my HD and most HD's is 20 milliseconds.

Average Seek time is around 10 to 15 milliseconds.

So that means 50 seeks per sec for worst case.

The buffer is 4 KB...

50x4 KB = 200 KB.

That's pretty amazing... the harddisk performs even worse than the worst
case scenerio.

I did not expect that.

Well I found this document about my harddisk...

It contains some information about seek times and such.

However the term 'Head Switch' means something else in this case.

It means moving from track to next track or something I think...

As far as I could tell there is no mentioning of latency from read to write
to read to write etc...

Or maybe I missed it... Anyway time for me to call it a day.

http://www.hitachigst.com/tech/tec [...] tta_sp.pdf

Bye,
Skybuck.




















































This clearly indicates the file



With the new code my harddisk now performs in the range of:

80 KByte / Sec to 400 KByte / Sec which I call poor/bad performance.

Concept 1 ( The Reading ) can achieve easily 8 MByte/Sec to 12 MByte/Sec.


After a few seconds on a 36 MB file it achieves 180 MByte/Sec this is
probably from Windows XP cache or maybe Harddisk Cache ?

The harddisk is completely silent. I can't imagina th






can achieve 1 MByte / Sec up to 12 MByte/Sec













>
> Nathan McNulty
>
> kony wrote:
>
> > On Wed, 14 Jul 2004 17:53:52 +0200, "Skybuck Flying"
> > <nospam@hotmail.com> wrote:
> >
> >
> >>Hi,
> >>
> >>Take these 3 concepts and then look at the implementation/performance:
> >>
> >>( 3 Concepts for reading/writing with harddisks )
> >>
> >>Concept 1:
> >>
> >>Seek, Read, Read, Read, Read, Read, Etc
> >>
> >>Results: FAST
> >>
> >>Concept 2:
> >>
> >>Seek, Write, Write, Write, Write, Write, Etc
> >>
> >>Results: FAST
> >>
> >>Concept 3:
> >>
> >>Seek, Read, Write, Read, Write, Read, Write,
> >>
> >>Results: SLOW ???
> >>
> >>The pseudo code is like:
> >>
> >>Seek( 0 ); // offset 0
> >>
> >>For I:=0 to FileBlocks-1 do // number of 4KB blocks in file.
> >>begin
> >> if Random(2) = 0 then
> >> Read( 4 KB )
> >> Else
> >> Write( 4 KB );
> >>end;
> >>
> >>The original concept is:
> >>
> >>Do a seek once.
> >>
> >>Then read or write a block of data. The head is automatically forwarded
to
> >>the next block. So no extra seek is needed.
> >>
> >>For concept 1 and concept 2 this works just fine and gives good
performance.
> >>
> >>However concept 3 has very bad performance.
> >>
> >>Is this a software issue ? ( Windows XP )
> >>
> >>Is this a hardware issue ? ( Harddisk Read Head and Harddisk Write Head
> >>can't work together like this and an extra seek is needed ? )
> >>
> >>Or some sort of driver issue ? ( Harddisk driver / firmware issue ? )
> >>
> >>Bye,
> >> Skybuck.
> >>
> >
> >
> >
> > I may not know the answer but feel an important question might be
> > "What is very bad performance?", compared to good performance...
> > numbers are our friends.
> >
> > Could it simply be that you're switching back and forth with data
> > flow so the caching (on the drive) isn't effective?

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

In windows xp I always have harddisk write cache off to prevent data loss in
case of power failure etc.

But just for the kick of it I enabled it, and I did no reboot. It had no
effect on performance, though I only tested shortly

Tomorrow I might test it once more... but after enabling it and before
testing it I will do a reboot.

I doubt I will see any performance difference.

If I don't see any performance difference in writing speed etc... that would
be a little bit suprising...

Since that would mean write cache doesn't help and can only cause loss of
data anyway.

Besides from that the document doesn't make any differences in Read and
Write speed.

My test programs clearly show there is a difference in Read and Write speed.

The document just mentions 'Data Transfer' speed... which is probably only
the Read speed.

The funny thing is my programs achieve those speeds roughly... so that's
correct... and my programs are correct

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

"Nathan McNulty" <525676@betaweb.com> wrote in message
news:%23G%23AUyeaEHA.3524@TK2MSFTNGP12.phx.gbl...
> 1. Yes really. You cannot read and write at the same time. The command
> has to be either read or write.
>
> 2. Hardware limitation that it cannot read and write at the same time.

That's kinda interesting.

That could mean when downloading and uploading a file for example... or
doing any other work.

The number of seeks that can be done per second has to be divided by 2.

So that would mean in worst case scenerio.. there are only 25 seeks per
second available !

So that's 25 seeks for reading and 25 seeks for writing

But... that could be stupid to divide it like that.

Since writing at least on my harddisk is much slower...

So maybe spending 25 seeks on writing would be stupid... or maybe it would
be necessary to still achieve 1 mbyte/sec.

So to achieve say 10 MByte/Sec... it's 10 MByte / 25 = roughly a 400 KB
buffer which can be split into 100 reads of 4 KB.

Which will all (hopefully) be read from the harddisk's read cache... so
those 100 reads don't require an extra seek.

For writing I don't dare to do the calculations since writing is weird...
much slower !

>
> 3. From what I have seen on my old harddrives I have taken apart and
> what I have read, there is a tip on the head that prevents the magnetic
> part from affecting nearby sectors. I was under the impression these do
> not always stay in the same position but are required to move each time
> a write process is initiated.
>
> 4. Again, the tips would have to move back to their resting position.
> Maybe this is just old hardware (especially since my books are all very
> old) or maybe I've read it wrong, but that was my understanding of how
> that part worked.
>
> Nathan McNulty
>
> Ron Reaugh wrote:
>
> > "Nathan McNulty" <525676@betaweb.com> wrote in message
> > newsauKdFeaEHA.1048@tk2msftngp13.phx.gbl...
> >
> >>Actually, what is happening is that the heads on the platter need to
> >>reconfigure themselves each time you read or write.
> >
> >
> > Not really.
> >
> >
> >>It is a hardware
> >>limitation.
> >
> >
> > What limitation is that exactly?
> >
> >
> >> By reading, the head can stay in the same location, but
> >>once it is told to write, the head has to position itself to change the
> >>magnetism of the sector of the platter it is on.
> >
> >
> > That's false.
> >
> >
> >> Then once you request
> >>to read again, the head has to move again.
> >
> >
> > Not necessarily.
> >
> >

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

Hmmm...

I examined the source code of my read and write benchmark programs

The read program reads the totaly file.

So it does

1 Seek

X reads until file is read.

The write program however uses a cache of 1 MB.

So it does

1 Seek

X writes until cache is written and then again

1 Seek

X writes until cache is written etc etc.

So that can explain why writing is slower.

So I will have to write some new benchmarks... which behave exactly the same
for reading and writing

Bye,
Skybuck.

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

"Skybuck Flying" <nospam@hotmail.com> wrote in message
news:cd4bur$b8r$1@news3.tilbu1.nb.home.nl...

> Concept 3 (Reading and Writing, with regular switches )
>
> Oh my... now that's what I call bad performance !
>
> The first 10 seconds it was 80 Kbyte/Sec... the next 20 seconds 160
> Kbyte/sec.
>
> Max was 200 KByte/sec.
>
> So it seems switching from read to write to read to write requires extra
> seeking etc...

Nope, calculate the theoretical rate for the data group to be transferred
at the rate of one per revolution. Each time you switch from read to write
you likely lose the rest of the current disk rotation.

> Maximum seek time for my HD and most HD's is 20 milliseconds.

What is the relation of the location of each 4KB block of data? Are they
sequential in a file? Where is a write located compared to the preceeding
read?.

> Average Seek time is around 10 to 15 milliseconds.

Seeking has to do with moving the head to a different track. Are you doing
random I/O of these blocks in a big file or to multiple files?

It sounds like you haven't well thought out or at least described what you
are doing.

Archived from groups: alt.comp.hardware,alt.comp.lang.borland-delphi,comp.sys.ibm.pc.hardware.storage,microsoft.public.win32.programmer,microsoft.public.windowsxp.hardware (More info?)

"Skybuck Flying" <nospam@hotmail.com> wrote in message

> My test programs clearly show there is a difference in Read and Write
speed.

Well what exactly is your test program doing to what kind of a file?