Validity of HDD Charts Results

[CYPER]

Can anyone explain to me how it is possible that WD VelociRaptor 600GB (WD6000HLHX) which has better linear speeds and seek times has lower score than for example Samsung F3 1TB (HD103SJ)

The supplementary text states that "These overall scores are calculated from three different tests measuring the read speed, write speed and seek time of hard disk drives."

If that is the case then why a disk with better specs in all of these 3 tests has a lower score than a disk with lower specs?

Just simply how on earth a standard 7200rpm disk can be considered better than a 10000rpm Velociraptor, which is like the King of HDDs?

[David (PassMark)]

Looking at the drive benchmark charts, the average results in the charts for these drives are very close in fact.

Looking at the individual results for the WD6000HLHX shows that many people don't get the max speed out of these WD drives. The best results are around the drive's rated max speed (~145MB/sec) but there are large numbers of people who are only seeing 90MB/sec to 120MB/sec in real life. This might be due to MB issues (e.g. older SATA interfaces) or higher drive fragmentation as the drive is smaller than the 1TB drives, or the fact that more people are running these drives at near full capacity. When near full these drives only run at 90MB/sec to 100MB/sec. See,
http://www.tomshardware.com/reviews/wd6000hlhx-velociraptor-600gb,2600-4.html

If you have 500GB of data to put on a drive, there is thus a reasonable argument to be made to buy the bigger Samsung drive and have it 50% empty. Look up Short Stroking as well, e.g.
http://www.tomshardware.com/reviews/short-stroking-hdd,2157-11.html
As running a drive near empty is similar in concept.

[CYPER]

First thank you for the useful info and allow me to ask some additional questions:

1 - Is there a publicly available list of all drives of the same make and model performance?
2 - What is the actual algorithm of awarding the score points? Can you say that linear speeds or seek times are more important and thus are awarded a bigger percentage of the points. Or are all factors equally treated?

I'm asking this, because I am a member of another hardware forum and we have a very heated discussion about WD 1TB Blue - WD10EALX vs Samsung F3 1TB - HD103SJ and I used your charts to prove that the Samsung is better, but the other party responded that your charts favour the linear speeds and pay less attention to seek times in calculating the final score. As a argument they pointed out that your chart lists the Samsung F3 1TB as superior to the Velociraptor, which you would agree is not true or is it?

Thank you.

[David (PassMark)]

You can download all the baseline results from within PerformanceTest. The problem is that at the moment there is no simple way to just search for results for the certain drive model. We can do this internally by direct query on our SQL database.

There are details about the tests and the calculation of the disk mark value here,
http://www.harddrivebenchmark.net/hdd_test_info.html

Ignoring price, power usage, future capacity requirement & noise. If I had a small amount of data (e.g. < 300MB) AND and motherboard that supported SATA 6Gb/s, I would get the Velociraptor. If I had 400GB or more of data I would get the Samsung F3.

The WD Blue WD10EALX would not be my first choice if I had to pick between these 3 drives based solely on performance.

[CYPER]

Ok, then let's take for example Samsung F3 1TB - HD103SJ which has a score of 879. Divide that by 10.85 and we get a basic score of 81.01.

Can you please clarify how exactly is this number calculated?

For example if the:
Disk Sequential Read is 110MB/s
Disk Sequential Write is 110MB/s
Disk Random Seek RW - 15ms

Then how do you get a score of 81.01 out of these numbers?

And one very important question - the Disk Random Seek RW test states that a file is created and them data is read randomly from that file. But I would guess that file is very small, compared to the overall size of the disk, therefore the head only moves small distances and does not travel the entire platter of the disk? Is this the best way to test seek times?

Thank you.

[David (PassMark)]

The Disk Random Seek RW result is not in ms. It is in MB/sec.

Also the numbers in the charts are averages of the disk marks values from around 400,000 different machines. And the Disk mark itself is an average of the 3 tests detailed in the link above.

As of today there are 3798 benchmark results for the HD103SJ drive that make up the average.

Taking a real example from baseline number 414863 (which you can search for in PerformanceTest if you want)

Read
114.4 MB/sec
Write
124.9 MB/sec
RW Seek
3.83 MB/sec

Average of the 3 results.
81.04 MB/sec

Multiply by 10.85 to get DiskMark of
879.5

This number is averaged with results from 3797 other machines for the HD103SJ drive to get the number in the charts. Every night the charts get completely recalculated, so the numbers change from day to day a bit. We get about 500 new results per day that get feed into the charts.

[David (PassMark)]

Oh and in regards to the question of seek time method. There is no point asking a modern hard to seek without doing a read or a write as well. Most drives won't bother doing the seek operation by itself.

It is pretty much impossible to accurately determine the layout of a drive. As soon as you have multiple platters, RAID, JBOD, multiple partitions, and solid state drives knowing where the "outside" of the disk is becomes hard to define. If you can't define the outside of the drive, then seeking to it is also hard.

Also we wanted the test to be non destructive and fast to run. So it will never touch existing data. Meaning that writing to the start and end of the disk is out of the question. More could be done if the test was destructive and ignored partitions and files, and if it was OK that the test ran for hours.

In our experience drives that have good seek times over short distances have good seek times over long distances we well. It is rare to seek the full span of the disk in real world applications.

[CYPER]

The Disk Random Seek RW result is not in ms. It is in MB/sec.

Also the numbers in the charts are averages of the disk marks values from around 400,000 different machines. And the Disk mark itself is an average of the 3 tests detailed in the link above.

As of today there are 3798 benchmark results for the HD103SJ drive that make up the average.

Taking a real example from baseline number 414863 (which you can search for in PerformanceTest if you want)

Read
114.4 MB/sec
Write
124.9 MB/sec
RW Seek
3.83 MB/sec

Average of the 3 results.
81.04 MB/sec

Multiply by 10.85 to get DiskMark of
879.5

This number is averaged with results from 3797 other machines for the HD103SJ drive to get the number in the charts. Every night the charts get completely recalculated, so the numbers change from day to day a bit. We get about 500 new results per day that get feed into the charts.

Useful info, but I personally find that the Random Seek RW part of the test is given less credit in comparison to real life scenarios. And I will give you an example:

Disk 1:
Read - 115MB/s
Write - 125MB/s
Seek - 5MB/s
============
Base score is 81.6

Disk 2:
Read - 100MB/s
Write - 110MB/s
Seek - 30MB/s
============
Base score is 80

So would you personally conclude that disk 1 is a better choice?
You probably wouldn't, because sequential speeds are less important than seeks times when the drive is populated with many small files, which is the case with an installation of Windows.
You would not see a noticeable performance difference between both drives when copying large files, but you would definitely see a big improvement in many other areas when using disk 2 such as loading Windows or searching for files or defragmentation to mention a few. And these are much more common in real life.

Your formula dilutes the seek time result and therefore skews the results as to present hard drives with faster sequential speeds as better.

I hope you accept my comment as constructive, as I have no intention to belittle your software.

Thank you.

[CYPER]

passmark, would you comment on my last post, please.

Thank you.

[David (PassMark)]

...such as loading Windows or searching for files or defragmentation to mention a few

With SuperFetch on (the default) Windows tried to do large sequentiual I/O during boot. See,
http://technet.microsoft.com/en-us/m...stakernel.aspx
Also people with Vista and Win7 probably aren't re-booting all that often, as the Sleep function works better than in XP.

Searching for files is now mostly done in using a pre-built index, without reading the files themselves. So it isn't all that disk bound.

De-fragmentation is done in the background on Vista and Win7. So the user will never notice if it runs fast or slow.

But of course a good seek time is important. A better example of when it is critical would be a large SQL database, or a web server on a large web site under heavy load.

So there is no single number that will be the ideal benchmark for everyone, people have different uses for PCs and they need to take this into account.

If the numbers are close however, then the performance will be close for typical usage.

I actually have a very early SSD drive that is like your example, slow sequential (~50MB/sec) but relatively fast seek (~40MB/sec), when compared to traditional drives. It doesn't feel that fast in real life.