Single Thread Score rating

[BenchmarkManiac]

Well. It is still odd that 3800X outperforms 3950X, but margin is very tiny and may be they'll swap when more samples are collected.
Seeing the 8th generation Coffee Lake processors among top 5 is also odd but may be may be.
I don't believe that Kaby Lake's 7700K outperform the latest AMD, that is wrong for sure.

[David (PassMark)]

People underestimate the amount of performance variation between machines in the wild. Even when using the same CPU. There are dozens of small and large effects that impact the measured CPU performance. It is doubly bad for laptops as vendors often ship them with single channel RAM and bad cooling.

Here are some examples for PerformanceTest V9.

Ryzen 5 Desktop CPU has a pretty nice looking bell curve (as would be expected for real life sampling of any variable). But the max and min extremes are still pretty far apart.



Here's a distribution graph for a laptop CPU. I am guessing that there were just a few popular vendors using this 7700HQ CPU (e.g. Dell, HP, etc..) and the two major laptop models perform differently from each other due to design decisions like RAM and cooling. Thus giving two peaks on the graph. Making the spread of results even wider than for desktop CPUs.

[claurie]

Don't get how Zen2 on Ryzen 3000 (3900X and others) with far superior IPC to the points it equals i9-9900K in single thread in most of the single threads tests (including the one in Passmark 9, Cinebench R15, Cinebench R20, etc....) is suddenly (or magically) dropping on new passmark 10 build 1004 single thread test. Ryzen 9 3900X even beats in gaming the new i9-10900K ( https://wccftech.com/intel-core-i9-1...nchmarks-leak/ ).

[David (PassMark)]

I don't there there is any evidence that the Ryzen 3000 series has far superior IPC than the 9900K. It might have in some particular scenarios, but not across the board.

In V9 we ranked Ryzen 9 3900X and 9900K as basically the same (under 2% difference).
In V10 we (at the moment) are saying there is a 9% difference. The result will likely move around a bit more over the coming weeks.

The 9900K is clocked at 5Ghz, which is 400Mhz faster than 3900X. (9% difference).

In short: The algorithms changed (see previous posts). And as pointed out above, even tiny code changes can move results around a lot.

[CerianK]

Looking at recent benchmarks, many AMD builds cap the single-thread performance by locking the CPU to a speed well under its maximum rating.
This produces a lower single thread performance and increases the deviation in all results.
I just ran my son's new 3800X build and the single core result was about 2790, well above average since it was able to stretch up to 4.5GHz.
The new V10 results parallel my research into random number generation across multiple platforms. Kudos.
I had not looked at Rand() to notice any change and/or bias.
The V9 results were obviously skewed for comparison in that scenario (asm primitives), and it is unfortunate that the skew flipped in the other direction when V10 was first released.

[BenchmarkManiac]

The result will likely move around a bit more over the coming weeks.

So far it only getting worse. Ryzen 3950X is already losing even to 3600X. This is nonsense. It really looks like the test is made with no boost, like some heavy background task is run on the background or the single thread test is run just after the heavy multithreaded load and was very short. CPU are sorted by their base frequency.

Kaby Lake still outperform all the recent AMDs, this is also very strange.

[dylandog]

I don't there there is any evidence that the Ryzen 3000 series has far superior IPC than the 9900K. It might have in some particular scenarios, but not across the board.

In V9 we ranked Ryzen 9 3900X and 9900K as basically the same (under 2% difference).
In V10 we (at the moment) are saying there is a 9% difference. The result will likely move around a bit more over the coming weeks.

The 9900K is clocked at 5Ghz, which is 400Mhz faster than 3900X. (9% difference).

In short: The algorithms changed (see previous posts). And as pointed out above, even tiny code changed can move results around a lot.

David there are many test and benchmark that show ryzen 3000 with a higher ipc and multicore then coffee lake (like here https://www.youtube.com/watch?v=DjBC_SzEKh4) and it seems that you don't want to accept it.....ryzen 3950x beat 9900ks in most single thread applications but this new update totally broken ryzen 3000.......while in gaming coffee lake is faster due to lower latency

[CerianK]

I do still see some indications of as much as 5% single-thread bias against against AMD in Passmark if I compare to some custom workloads under Linux.
However, Linux is not really directly comparable to Windows, as the same code run under both Windows and Linux will often show a bias towards Linux, which could be nearly 10% depending on exactly what is being tested. That indicates that the delta is time wasted doing something else other than the intended workload due to many possible factors (e.g. compiler, security mitigations, thread scheduler thrashing, etc.).

Considering the inconsistencies in Windows benchmarks, it might be a good idea for Passmark to list single-thread sub-scores for individual tests in the interest of full-disclosure.

David there are many test and benchmark that show ryzen 3000 with a higher ipc and multicore then coffee lake (like here https://www.youtube.com/watch?v=DjBC_SzEKh4) and it seems that you don't want to accept it.....ryzen 3950x beat 9900ks in most single thread applications but this new update totally broken ryzen 3000.......while in gaming coffee lake is faster due to lower latency

That Youtube link seems incorrect... Ark Genesis in Italian.

[BenchmarkManiac]

I am thinking about your random data. Do you really make random data for the test? This makes test nonreproducible and nondeterministic. The data for the test should be identical from one run to another from one machine to the other machine. Otherwise you will see this crazy random effects in results. What did you expect if you are running the test in random data input?

I mean.. you can use a self made congruential pseudorandom generator to generate identical sequences of data based on the same seed but it seems that you aren't using this method and do not really bother about determinism of the test.

[CerianK]

Do you really make random data for the test?

If you are asking me, then no, no connection. In any case, Passmark is now using a (presumably constant seeded?) LCG, minstd, instead of Rand(), which David discussed previously in this thread.

If it matters, I have written PRNG benchmarks in Windows and Linux for my own research and took notice of speed discrepancies in Passmark only after a Reddit poster made a comment to me that prompted me to review my own prior data. It is a coincidence that the original issue (corrected by Version 10.0.1004) was actually related to random numbers, but that did prompt me to post here.

I am not convinced one way or the other if there are any remaining issues that need to be addressed in Passmark, but I can authoritatively state that an AMD 3800X has the potential to be up to 5% faster in some workloads than the Passmark results would predict. Word has it that Intel has a better AVX implementation, that I have not yet tested in my work, but will soon. It wouldn't suprise me to find Intel dominating that test by at least 5%.

[BenchmarkManiac]

I am asking David of course. If they used rand from the library I doubt they really focused on determinism. At least it can change from one compiler to another and from one library to another. When people care about reproducibility of the pseudorandom sequence they write their own generator.

[CerianK]

Consider the new code changes:

+    std::minstd_rand rng(RAND_SEED);
-        pbDataBuffer[i] = (rand() % 27) + 96;
+        pbDataBuffer[i] = (rng() % 27) + 96;

Yes, 'RAND_SEED' indicates a constant declaration, so the sequence should be deterministic if upper bound 'i' is a constant also. However, the issue that was addressed is spending too much time generating random numbers, even though (from David):

Generating random numbers was always part of the test, but it should have been a small part.

Based on that, I see some issues:
1. The random numbers generated are 32-bit.
2. The random numbers generated are not considered random by modern standards, even for non-cryptographic use (i.e. minstd_rand, and most others in the library, should be recommended for deprecation).

You might accept #2 as a non-issue for benchmark purposes, but there may be hidden caveats.
#1, however, will cause twice as much time (i.e. not a small part) to be spent on generating random numbers as should be necessary on a 64-bit processor, and could potentially end up testing how well a processor performs in 32-bit scenarios.

[David (PassMark)]

Looking at recent benchmarks, many AMD builds cap the single-thread performance by locking the CPU to a speed well under its maximum rating.
This produces a lower single thread performance and increases the deviation in all results.
I just ran my son's new 3800X build and the single core result was about 2790, well above average since it was able to stretch up to 4.5GHz.

Yes, there is a spread of results. But I don't think it is due to capped turbo clock speed. I think the main cause is differences in RAM setups (channels active, latency and RAM clocks).

Compare these two graphs below from PT10. The Physics test is more sensitive to RAM setups than Encryption.

Baseline 1207902 obviously has some minor (non RAM related) setup problem. But the other results are pretty consistent for Encryption.

There are really big differences in the Physics test however, for the same set of machines. You can see the same for Prime numbers which is also sensitive to RAM (graph not shown).

The single threaded is slightly effected by the RAM setup. No were near as much as the physics test however.

I just ran my son's new 3800X build and the single core result was about 2790, well above average since it was able to stretch up to 4.5GHz.

Single threaded average for 3800X today is 2750. So in fact you with 2% of average now.

[David (PassMark)]

So far it only getting worse. Ryzen 3950X is already losing even to 3600X. This is nonsense.

Comparison of AMD Ryzen 9 3950X and AMD Ryzen 5 3600X is here,
https://www.cpubenchmark.net/compare...00X/3598vs3494

Situation reversed itself over the week-end. AMD Ryzen 9 3950X is now very slightly in front.

But I think this is a reflection of their actual single threaded performance. Not an anomaly. Even the non-x version, then Ryzen 5 3600 can beat the 3950X, depending on the application.

Quote from TechRadar.
"The story continues in our Middle Earth: Shadow of War benchmark. There, the Ryzen 9 3950X scored an average of 116fps at Full HD and 49fps at 4K. The Ryzen 5 3600 hit 118fps and 51fps for the same tests, shockingly beating the Ryzen 9 3950X. The Intel Core i9-9900K came out slightly ahead with 125fps at Full HD and 52fps at 4K."

Just by coincidence, same quote also said Intel higher end chip was faster.

I am guessing you just assume that the more expensive CPU should perform significantly better in all tasks, but it doesn't.

[David (PassMark)]

David there are many test and benchmark that show ryzen 3000 with a higher ipc and multicore then coffee lake (like here [noscript]https://www.youtube.com/watch?v=DjBC_SzEKh4[/noscript]) and it seems that you don't want to accept it.....ryzen 3950x beat 9900ks in most single thread applications but this new update totally broken ryzen 3000.......while in gaming coffee lake is faster due to lower latency

That video was 20min of streaming game play from the game ARK doing a review to the Genesis DLC. Nothing at all to do with benchmarking Ryzen.
Plus it was all in Italian. I'll give you the benefit of the doubt I assume it was a typo and not blatant self promotion of a channel you are associated with.
We've got a super low tolerance for trolling, spammers, self promotional & bots building back links. Is just a huge waste of time for everyone. I sometimes ban up to a dozen people a day (and after 10 years of doing it it gets tiresome). You've been warned.