Scaling issue in PassMark CPU benchmark?

[pbrakemeier]

Hi,
I was closely monitoring the High End CPUs benchmark list to learn about Threadripper 3960X/3970X performance. Now, that some results are coming in I'm wondering if the test reflects the real performance. Given the 36015 score that I see for the Ryzen 9 3950X i would expect something above 50K for TR 3960X, but it just gets to 46891. Even worse, the TR 3970X, with 8 more cores, just scores 1500 points higher. So either the CPUs are scaling badly and the TR 3970X is not worth the extra money or the PassMark CPU benchmark is not yet prepared well for high core count CPUs. Can you acknowledge that behaviour? Are there plans for an update of the benchmark?

-p

[David (PassMark)]

We are working on PerformanceTest V10.

But I don't think it is going to change much in this regard (nor should it).
If you double the number of cores in a CPU, then you don't double the performance. For nearly all hardware / software there is a point of diminishing returns. Not all code can be highly threaded and other factors start to become a bottleneck as you increase the core count (RAM bandwidth in this case). Many applications struggle to use even 4 cores.

[pbrakemeier]

I am aware that there are diminishing returns for most applications. But I would think that a relevant benchmark reflects the best case potential the CPU has to offer. For applications that can't use even 4 cores you already have the single thread benchmark which is very helpful for that purpose.

People bying 24+ core CPUs are likely interested in running massively parallelizable applications such as rendering/number crunching, and I'm pretty sure for those the uplift from 24 to 32 cores will be higher than indicated in the performance test. Of course there is no right or wrong in what the test entails, my suspicion is just that it is running into a hard coded thread limit north of 16 cores.

[David (PassMark)]

There are issues with systems that have multiple process groups (e.g. 128 cores) and issues around NUMA nodes. But up to 64 cores our code is OK.
That doesn't mean the hardware has enough memory bandwidth & cache for 64 cores however.

People bying 24+ core CPUs are likely interested in running massively parallelizable applications

In our experience most people don't know what they are buying or how the code in their applications are written (threaded or not, or if the real bottle neck is disk or networking or RAM or cores or GPU).


See also this post for more discussion on scaling:
https://www.passmark.com/forum/pc-ha...-amd-epyc-7742