Significant difference in Memory test results between GUI and CLI versions

[phr]

Seems GUI and CLI versions of Performance test for MAC report significantly different results:

GUI: https://www.passmark.com/baselines/V...d=503823224274

CLI: https://www.passmark.com/baselines/V...d=503823351931

Looks like the animated picture, displayed during the tests is affecting the results and preventing memory caching - as cached and uncached memory speeds are very close in GUI test.

[David (PassMark)]

It would be strange that if the cache performance was cut in half that it didn't impact the CPU test.at all.

It might be that results on your machine are somewhat random and this was just co-incidence.
(random fluctuations can be due to background software, thermal throttling and other factors)

Can you repeat the tests maybe 5 times and see how consistent it is?

[phr]

Here are 3 GUI and 3 CLI tests performed on idle machine in round-robin fashion.
They are pretty consistent and always showing half the cache performace on GUI tests:

GUI1: https://www.passmark.com/baselines/V...d=503823857967
GUI2: https://www.passmark.com/baselines/V...d=503824049743
GUI3: https://www.passmark.com/baselines/V...d=503824258361

CLI1: https://www.passmark.com/baselines/V...d=503823741416
CLI2: https://www.passmark.com/baselines/V...d=503823909054
CLI3: https://www.passmark.com/baselines/V...d=503824156252

Encryption is ~3350 MB/s
Compression ~159500 KB/s
They don't appear on the website correctly.

[David (PassMark)]

We'll take a look and see if it can be reproduced.

[phr]

Reproduced on another MAC:

GUI: https://www.passmark.com/baselines/V...d=503835232276
CLI: https://www.passmark.com/baselines/V...d=503835136542

Encryption is ~6300 MB/s
Compression ~253000 KB/s

[Richard (PassMark)]

I was able to reproduce the discrepancy locally between the CLI/GUI for the Memory Read Cached. We are currently looking into it.

[Richard (PassMark)]

UPDATE: The Apple Store (GUI) and the CLI programs uses the same base code except for the user interface. The GUI calls the same test functions so there should be no difference. At this point, we hope it is a phantom issue. When rebuilding PerformanceTest for Mac, the test results match up with the CLI edition. We submitted the newly built version to the App Store and it should become available once it has been reviewed by Apple.

[phr]

I believe the difference might be caused by the fact, that CLI version is not doing anything else during the test - it prints "Running" before the start of a test and prints the test result afterwards. But the GUI version displays animated indicator rotating during the whole test, which certainly uses some resources.

Would it be possible to remove the rotating indicator and just print "Running" the same way as in CLI test? After all, it's a test program, which should focus on results accuracy and not on fancy graphics stuff.

Could you please also fix incorrect website upload for Encryption and Compression on both GUI and CLI versions for MAC? They are probably uploaded in wrong units.

[David (PassMark)]

We do not think the difference is caused by the spinning icon.
We think it is more likely due to compiler settings. We recompiled the code with current Apple compiler and dev environment (with no code change) and the problem is gone. Which of course is pretty strange.

Apple approved the new release a couple of hours ago. So you might be able to try new version now.

[phr]

Thanks, I can confirm the bogus GUI result for Memory Read Cached is gone:

GUI: https://www.passmark.com/baselines/V...d=503845204058
CLI: https://www.passmark.com/baselines/V...d=503845419517

Nevertheless, GUI results are still consistently a bit lower than CLI ones, which is most probably due to the spinning icon.

For the website:
Encryption is now displayed OK
Compression result 158287 KB/s is displayed as 154 MB/s on the website, which is mathematically correct but it might be better to display it the same way as in apps, i.e. in KB/s.
Sorting 17445 Thousand Strings/s is displayed as 17 Thousand String/s on the website.

[Richard (PassMark)]

The sig figs and units were more to match with what is displayed in the PerformanceTest Windows, but in any case, I've updated the display page for Mac baselines to match Mac's displayed values.

[phr]

Thanks, webpage baselines for MAC & Linux look OK now.

There's still problem with windows baseline webpages, since the windows app. also shows Compression in KB/s and Sorting as 15000 Thousand Strings/s, while webpages show Compression in MB/s and sorting values 1000-times lower (15 instead of 15000). Both problems are present also on CPU webpages (https://www.cpubenchmark.net/cpu.php)

As for the MAC GUI, it seems to consistently reduce results by approx 2 % when compared to MAC CLI.