Single Thread Score rating

[BenchmarkManiac]

Some more concerns. Take a look on i9-9900KS vs Ryzen 7 3800X https://www.cpubenchmark.net/compare...00X/3593vs3499
Both processors have the same number of cores. 9900KS is always boosted to higher frequency than 3800X.
9900KS is 20% faster than 3800X in single thread benchmark but still lose in multithreaded benchmark!?
How can it be? Math just don't sums up here. It should ether be slower in single thread test or should be much much faster running test on all cores.

[BenchmarkManiac]

But if it turns out the i9-9900KS is the new single threaded performance leader

9900 was the leader in the previous version of benchmark also. The question is why the new algorithm is so inconsistent on AMD CPUs. 3700 PRO is 20% faster than 3700X, the previous leader among AMD chips 3950X is now placed after the mobile Ryzen 3 4300U and lose to the most Intel chips as well. It looks like a weird bug, AVX is not used on some AMD chips or the single thread test is evaluated with some heavy second task running on the background.

[David (PassMark)]

Have you used the intel c++ compiler

No. As per the PT10 release notes, Visual Studio 2019 is now used.
We did try out the Intel compiler internally many years ago. Surprisingly it didn't actually favor Intel CPUs very much (much less than people who have never actually done the experiment expect). And it also didn't do greatly better than Visual Studio. We've been using Microsoft's compilers for every public release for the last 20 years. They are basically the standard for Windows development.

How can you triple the algorithm performance using the new compiler?

There were many changes. See the release notes.
Variable alignment was a surprisingly big issue, as was rewriting bits of the code to allow more out of order instructions and allowing compiler optimization of code that was not previously optimized.

Have you triple checked the algorithm performance on different CPU and sure that there is no bug in it?

Yep, we are pretty sure it is OK. We;ve only got around 20 test machines in house. So testing top 100 CPUs isn't possible.
Results for the top CPUs in single threaded is really really close. So it does't take much at all to move the ranking around. Just a few percent up or down. Expect the ranking to move around a bit more over the next week as new results come it before new average levels are found.

But if it turns out the i9-9900KS is the new single threaded performance leader, we've got zero problems with that.

Quotes:
Hothardware.com
"The Core i9-9900KS is the fastest processor we’ve tested for single and lightly-threaded workloads. It put up top marks in all of the single-threaded focused benchmarks"

techradar.com
"the best gaming chip"

anandtech.com
"In our benchmarks, we see it steaming ahead of any other consumer grade processor when it comes to single core performance"

[BenchmarkManiac]

To be fair it really looks like you've been paid to tweak benchmark and boost Intel processors in the rating. Standings changed dramatically and now the Ryzen results are inconsistent and not very plausible. The PRO version of 3700 ryzen is 500 points ahead. How can you triple the algorithm performance using the new compiler? Have you used the intel c++ compiler known to use sub-optimal code for processors from vendors other than Intel? Have you triple checked the algorithm performance on different CPU and sure that there is no bug in it?

[frag_lez]

Hi David,

Thank you for the explanation. Good call with the results scaling. It makes so much easier to understand the numbers.
Do you think that in the future it will be possible to for example filter the results to PT10 only?

Regards,
Les

[David (PassMark)]

Hi,

We released a new version of PerformanceTest a few days ago, version 10.
Improvements in the benchmark test algorithms & using a more modern compiler resulted the single threaded test performing a much higher number of operations per second. These changes should push the CPU harder and use modern CPU features (out of order execution and multiple pipelines) better. The result was roughly 3x times more operations per second being performed, compared to PerformanceTest V9.

Yesterday we started to switch over the graphs on the web site to start to use results from PerformanceTest V10. This accounts for the change in the results in the graphs.

However in hindsight we think have done the wrong thing. We should had scaled down the PT10 single threaded result to match the PT9 results for the single threaded test. This single threaded test was already an average of values from several different single threaded algorithms. So additional scaling wouldn’t have changed the significance of the value.

On Monday (9th March 2020) we plan to patch the version 10 release to scale the single threaded value back to the PT9 results. Things should then be back to normal. In the meantime we have reverted the single threaded graph on the web site to use only PT9 results.

As we collect more PT10 results we expect PT10 to perform better on modern CPUs compared to older ones (relative to PT9). So overtime there might be a spreading out of the single threaded results, with the newer hardware pulling away from the older hardware a bit more.

Sorry for any confusion all this has caused.

More info:
See also this post for some additional details
https://www.passmark.com/forum/pc-ha...s-huge-changes

[frag_lez]

Hi,
What happen to the CPU Single Thread score rating? Why was it changed?
CPU benchmarks are so useful for me when I spec systems for customers.. as I deal with all types of CPUs from Atom to Xeon.. having a quick look at the numbers (especially at single thread performance) gave me a good indication of the performance... I knew what to expect from which CPU without comparing it to anything else.. .and now.. :/