Formula CPU mark, memory mark and disk mark

[David (PassMark)]

There are a bunch of other posts in the forum comparing V9, V10 and V11. But the short version is that.

1. The benchmarks changed only very slightly between V11 and V10 (more significantly on ARM however, compared to x86). There were much bigger changes between versions V9 and V10. Bigger changes are coming in V12 however.
2. We tried to make the CPUMark value comparable (and it roughly is) between all versions.
3. Use the same release if you can when making comparisons as it removes one more variable.

[Doug Barrett]

David - your mention of compatibility across OS's above raises a question that I was wondering about - If I have Performance Test v10 results and Performance Test V11 results for a given system, how well should they correlate? For example, running Windows 10 and Performance Test10, then install Windows 11 and run PerformanceTest11 on the exact same board, should my results be close? Especially on the CPU Mark and Single Thread Mark results? And other results? (And how about going back to earlier versions of Performance Test?)

Thanks.

(My apologies in advance if this question is handled elsewhere and I have missed it.)

[David (PassMark)]

Correct, these are derived values.

We'll be removing the Cross-platform Mark in the future as it isn't required anymore, this is because we all the CPU benchmarks are now cross platform (Windows, Linux, Mac & Mobile, x86 & ARM). So they are directly comparable now. In older releases just a sub-set of the CPU benchmarks were comparable.

[Doug Barrett]

The PerformanceTest 11.0 results include a couple values:

****
..
..
Cross-platform Mark (Composite average) : XXXXX (XX%)
Gaming Score (Composite average) : XXXX (XX%)
..
..
****

Am I correct to conclude that these values are not used in either the "CPUMark" or the "Passmark Rating" formula calculations in any way? (I do not see them in the formula calculations shown above, but maybe I missed something somewhere.)

[Simon (PassMark)]

For version 11, the updated numbers are:


Passmark Rating = 1 / (((1 / (CPU Mark * 0.33)) + (1 / (2D Mark * 50)) + (1 / (3D Mark * 0.5)) + (1 / (Memory Mark * 1.92)) + (1 / (Disk Mark * 0.37)))/5)
Rebalanced to make 2D have less of an impact on overall score


CPU Mark = 1 / AVERAGE ( test values ) * 1.65

1 / (0.149001983 * CPU Integer Math)
1 / (0.271311984 * CPU - Floating Point Math)
1 / (296.1871085 * CPU - Prime Numbers)
1 / (0.758484696 * CPU - Extended Instructions (SSE))
1 / (0.034970332 * CPU - Compression)
1 / (0.8766779 * CPU - Encryption)
1 / (17.81818182 * CPU - Physics)
1 / (0.392310713 * CPU - Sorting)
1 / (4.908772082 * CPU - Single Threaded)


2D Graphics Mark = 1 / AVERAGE ( test values )

1 / (60.25393099 * Graphics 2D - Simple Vectors)
1 / (3.649649659 * Graphics 2D - Fonts and Text)
1 / (15.78231555 * Graphics 2D - Windows Interface)
1 / (0.463690633 * Graphics 2D - Image Filters)
1 / (4.196513 * Graphics 2D - Image Rendering)
1 / (15.80278134 * Graphics 2D - Direct 2D)
1 / (16.39344251 * Graphics 2D - PDF Rendering)
1 / (13 * Graphics 2D - Direct 2D - SVG)


3D Graphics Mark = 1 / AVERAGE (test values) * 2.35

If no DX10 then 40% penalty is applied, if no DX11/DX12 a 20% penalty is applied
1 / (28.77991307 * Graphics 3D - DirectX 9 Complex)
1 / (88 * Graphics 3D - DirectX 10)
1 / (60 * Graphics 3D - DirectX 11)
1 / (95.96088498 * Graphics 3D - DirectX 12)
1 / (1.093968291 * Graphics 3D - DirectCompute)


Memory Mark = 1 / AVERAGE (test values)

1 / (2.5 * Memory - Database Operations)
1 / (0.077817619 * Memory - Read Cached)
1 / (0.171576582 * Memory - Read Uncached)
1 / (0.282054375 * Memory - Write)
1 / (0.44326614 * Memory - Available RAM)
1 / (48454.32714 * (1 / Memory - Latency))
1 / (0.126149451 * Memory - Threaded)


Disk Mark = test value * 12

Disk mark is a straight average unlike the inverse averages used for the other tests
(Sequential Read + Sequential Write + IOPS 32KQD20 + IOPS 4KQD1) / 4

[Leevis]

Hi, Tim,
Thanks a lot for your help. Appreciate your effort in showing me the calculation in such a clear way.

[Tim (PassMark)]

Data compression is displayed slightly differently in the mobile version (MBytes , while PT windows displays Kbytes).

Changing your data compression value to =8.5 * 1024 should result in the correct calculation (any discrepancy to what is displayed should then just be a rounding issue from not using the exact values and only be out slightly).

[Leevis]

Hi, Tim,
I couldn't get the CPU Mark value by substituting the values with the following formula:

CPU Mark = 1 / AVERAGE ( test values ) * 1.65
Overall CPU mark is scaled up to be closer to PT9 result

1 / ( 0.149001983 * CPU Integer Math)
1 / ( 0.271311984 * CPU - Floating Point Math)
1 / ( 296.1871085 * CPU - Prime Numbers)
1 / (0.758484696 * CPU - Extended Instructions (SSE))
1 / (0.034970332 * CPU - Compression)
1 / (0.8766779 * CPU - Encryption)
1 / (17.81818182 * CPU - Physics)
1 / (0.392310713 * CPU - Sorting)
1 / (4.908772082 * CPU - Single Threaded)

Below are my results getting from running the test on chromebook with Android app PerformanceTest Mobile V10:

	w	x	1 / (W * X)
CPU Test (Multi-thread)		2149
Integer Math	0.149001983	7173	0.000935636
Floating Point Math	0.271311984	3973	0.00092771
FindPrime Numbers	296.1871085	8.9	0.000379353
Random String Sorting	0.392310713	3971	0.000641904
Data Encryption	0.8766779	1383	0.000824779
Data Compression	0.034970332	28.5	1.003356768
Physics	17.81818182	181	0.000310069
Extended Instructions	0.758484696	740	0.001781646
Single Thread (MOps/Sec)	4.908772082	1533	0.000132888

CPUMark = 2149
As manual calculation, CPU Mark = 1 / AVERAGE( 1 / (W * X))) * 1.65 = 14.71330232
The scaler: 2149/14.71330232 = 146.0583051

How to How to estimate the correct CPU Mark?
Is the scaler a fixed value? or depend on different values produced by different devices?
Thanks.

[David (PassMark)]

I don't think you did the calculation correctly. I got 7024 for PC1 using the PT10 formula.

But the algorithm for the overall machine's rating (the PassMark rating) doesn't let one super fast component dominate the result. And conversely one super weak component can hold back the whole machine. It favours balanced machines.

So a score of 40,000 for the CPUMark is something around 4x a typical average CPU (sold over the last few years). But a 2DMark score of 770 is probably around the average for run of the mill machine. Big part of the problem is that hardware vendors haven't improved the 2D performance very much at all of the last 15 years. But CPUs and 3D performance has increased a lot.
I agree at some point we'll need to rebalance it.

[Ausfarmerjoe66]

Passmark Rating = 1 / (((1 / (10185.8 * 0.396566187)) + (1 / (812.1 * 3.178718116)) + (1 / (4411 * 2.525195879)) + (1 / (2546 * 1.757085479)) + (1 / (4968 * 1.668158805)))/5) = 4677.595655

This is a non-sensical rating and far too biased towards 2D tests. Take for example the following 2 scenarios

PC 1
CPU 19,000
2D 940
3D 16,000
Mem 3000
Disk 20000


Passmark rating = 7083

PC 2
CPU 40,000
2D 770
3D 25,000
Mem 3000
Disk 20000

Passmark rating = 7070

Can't tell me PC2 is not an overwhelmingly better PC than PC1 in any objective measure.

[Richard (PassMark)]

Any other formula used for mobile devices? If so, how it is supposed to compare them?

Only the CPU and Memory Tests has been standardize across platforms and the CPU Mark and Memory Mark can be compared directly with other PerformanceTests editions. The remaining tests (Disk, 2D and 3D) are not the same on the Mobile vs the Windows release of PerformanceTest and cannot be compared directly.

[Asen]

I applied this formula to Windows machine and everything worked out, but on Android devices is not:

1 / (((1 / (3953 * 0.396566187)) + (1 / (19535 * 3.178718116)) + (1 / (23146 * 2.525195879)) + (1 / (13265 * 1.757085479)) + (1 / (16953 * 1.668158805)))/5) = 6672

So, as you can see,

6672 != 7700

Any other formula used for mobile devices? If so, how it is supposed to compare them?

[Tim (PassMark)]

CPU_ENCRYPTION is MBytes / sec ( / 1048576), so using the value of 274.279038977543 you should see the same CPU mark.

It looks like the Linux version is not displaying the results in the expected formats.

CPU_mm: 143.85203527202719
CPU_sse: 600.24699137061884
CPU_fma: 0
CPU_avx: 1206.7660770010648
CPU_avx512: 0
m_CPU_enc_SHA: 171444498.41289219
m_CPU_enc_AES: 537332151.83488178
m_CPU_enc_ECDSA: 154030602.47697806

These are the scores for sub tests that are run, they are not required for calculating the score from the test results but are saved for reference.
(For example the Encryption test has 3 sub-tests to test different types of encryption and hashing algorithms).

[Rogelio Montanana]

Hi:

I've been caculating the CPU Mark and Memory Mark values from individual test results for Windows and Linux systems using the formulas published in this forum.

For the Windows case I have been mostly successful, since I have been able to reproduce the values shown within a 0.1-0.2 % error margin, which may be due to the fact that values are reported without decimals and sometimes they have only two or three significant figures.

But for the Linux case, in spite of having all the values with eight significant figures, for the CPU Mark the values I get are systematically 1-2 % over the reported values, and for the Memory mark my values are systematically below the reported values in all cases. It seems that I am missing something, but I don’t know what it is.

For example, in one execution I get the the following output:

Results:
Results Complete: true
NumTestProcesses: 1
CPU_INTEGER_MATH: 3473.6986666666667
CPU_FLOATINGPOINT_MATH: 1801.5623821969807
CPU_PRIME: 11088582.155407717
CPU_SORTING: 2372572.6501833266
CPU_ENCRYPTION: 287602417.57491738
CPU_COMPRESSION: 16487851.534536928
CPU_SINGLETHREAD: 1516.0919716574317
CPU_PHYSICS: 164.93171801707402
CPU_MATRIX_MULT_SSE: 903.50653418584182
CPU_mm: 143.85203527202719
CPU_sse: 600.24699137061884
CPU_fma: 0
CPU_avx: 1206.7660770010648
CPU_avx512: 0
m_CPU_enc_SHA: 171444498.41289219
m_CPU_enc_AES: 537332151.83488178
m_CPU_enc_ECDSA: 154030602.47697806
ME_ALLOC_S: 592.03581337674734
ME_READ_S: 18783.400195312501
ME_READ_L: 11171.9287109375
ME_WRITE: 7835.724609375
ME_LARGE: 778.16666666666663
ME_LATENCY: 43.892771756731001
ME_THREADED: 16525.2412109375
SUMM_CPU: 1122.8392054406484
SUMM_ME: 1057.6267754975174

Now I apply the CPU Mark formula using the following values:

CPU_INTEGER_MATH: 3473.698667
CPU_FLOATINGPOINT_MATH: 1801.562382
CPU_PRIME: 11.08858216
CPU_SORTING: 2372.57265
CPU_ENCRYPTION: 287.6024176
CPU_COMPRESSION: 16487.85153
CPU_SINGLETHREAD: 1516.091972
CPU_PHYSICS: 164.931718
CPU_MATRIX_MULT_SSE: 903.5065342

(observe that the values in yellow have been modified from the original)

With this I get CPU Mark 1143, which is 1.8 % higher than the SUMM_CPU value reported in the test (1123).

For the Memory mark, using the previously listed values in the formula I get 950, which is 10 % lower than the SUMM_MEM value in the report (105.

It seems that I am missing something. Maybe there is a different formula for the linux test?

Also, I see in the list the variables:

CPU_mm: 143.85203527202719
CPU_sse: 600.24699137061884
CPU_fma: 0
CPU_avx: 1206.7660770010648
CPU_avx512: 0
m_CPU_enc_SHA: 171444498.41289219
m_CPU_enc_AES: 537332151.83488178
m_CPU_enc_ECDSA: 154030602.47697806

which don’t appear in the windows test and are not used in the formula. Should they be also included somehow? What do they mean?

Thanks,
Rogelio.
Hi:

I've been caculating the CPU Mark and Memory Mark values from individual test results for Windows and Linux systems using the formulas published in this forum.

For the Windows case I have been mostly successful, since I have been able to reproduce the values shown within a 0.1-0.2 % error margin, which may be due to the fact that values are reported without decimals and sometimes they have only two or three significant figures.

But for the Linux case, in spite of having all the values with eight significant figures, for the CPU Mark the values I get are systematically 1-2 % over the reported values, and for the Memory mark my values are systematically below the reported values in all cases. It seems that I am missing something, but I don’t know what it is.

For example, in one execution I get the the following output:

Results:
Results Complete: true
NumTestProcesses: 1
CPU_INTEGER_MATH: 3473.6986666666667
CPU_FLOATINGPOINT_MATH: 1801.5623821969807
CPU_PRIME: 11088582.155407717
CPU_SORTING: 2372572.6501833266
CPU_ENCRYPTION: 287602417.57491738
CPU_COMPRESSION: 16487851.534536928
CPU_SINGLETHREAD: 1516.0919716574317
CPU_PHYSICS: 164.93171801707402
CPU_MATRIX_MULT_SSE: 903.50653418584182
CPU_mm: 143.85203527202719
CPU_sse: 600.24699137061884
CPU_fma: 0
CPU_avx: 1206.7660770010648
CPU_avx512: 0
m_CPU_enc_SHA: 171444498.41289219
m_CPU_enc_AES: 537332151.83488178
m_CPU_enc_ECDSA: 154030602.47697806
ME_ALLOC_S: 592.03581337674734
ME_READ_S: 18783.400195312501
ME_READ_L: 11171.9287109375
ME_WRITE: 7835.724609375
ME_LARGE: 778.16666666666663
ME_LATENCY: 43.892771756731001
ME_THREADED: 16525.2412109375
SUMM_CPU: 1122.8392054406484
SUMM_ME: 1057.6267754975174

Now I apply the CPU Mark formula using the following values:

CPU_INTEGER_MATH: 3473.698667
CPU_FLOATINGPOINT_MATH: 1801.562382
CPU_PRIME: 11.08858216
CPU_SORTING: 2372.57265
CPU_ENCRYPTION: 287.6024176
CPU_COMPRESSION: 16487.85153
CPU_SINGLETHREAD: 1516.091972
CPU_PHYSICS: 164.931718
CPU_MATRIX_MULT_SSE: 903.5065342

(observe that the values in bold have been scaled from the original)

With this I get CPU Mark 1143, which is 1.8 % higher than the SUMM_CPU value reported in the test (1123).

For the Memory mark, using the previously listed values in the formula I get 950, which is 10 % lower than the SUMM_MEM value in the report (105.

It seems that I am missing something. Maybe there is a different formula for the linux test?

Also, I see in the list the variables:

CPU_mm: 143.85203527202719
CPU_sse: 600.24699137061884
CPU_fma: 0
CPU_avx: 1206.7660770010648
CPU_avx512: 0
m_CPU_enc_SHA: 171444498.41289219
m_CPU_enc_AES: 537332151.83488178
m_CPU_enc_ECDSA: 154030602.47697806

which don’t appear in the windows test and are not used in the formula. Should they be also included somehow? What do they mean?

Thanks,
Rogelio.

[vincent911001]

Hi Tim,

Thanks a lot for your help. Appreciate your effort in showing me the calculation in such a clear way.

Vincent