What is each part of "Memory test" in Linux PerformanceTest doing?

[j7168908jx]

Hello,

I want to know what is each part of the memory test in Linux PerformanceTest doing?

I've got some weird result

I had a dual EPYC 9754 with 16x 64G 4800MHz Ubuntu system, and I ran the test, got

Memory Read Cached ~ 23G/s
Memory Read Uncached ~ 23G/s
Memory Write ~ 23G/s
Memory Threaded ~720G/s

and I just installed another 8 DIMMs, making a 24x 64G 4800MHz with exactly the same brand and part number memory sticks.

The test now has

Memory Read Cached ~ 23G/s
Memory Read Uncached ~ 23G/s
Memory Write ~ 11G/s
Memory Threaded ~910G/s


I want to know why this is happening. The `memory write` part drops about 50%.

Thanks for any help in advance!

[David (PassMark)]

From the Help file in PerformanceTest:

This suite contains a number of tests that exercise the memory sub-system of the computer (Random Access Memory- RAM). All tests use a combination of 32-bit and 64-bit data when reading or writing from or to RAM. Database Operations This makes heavy use of an in memory sqllite datbase to test the performance of memory in maintaining large structures of data in a database. A test thread is started for each available physical CPU core. Read Cached This test measures the time taken to read a small block of memory. The block is small enough to be held entirely in cache (if one is present) Read Uncached This test measures the time taken to read a large block of memory. The block is too large to be held in cache (512 MB). Write This test measures the time taken to write information into memory. Available RAM This test simply takes a measurement of how much memory is currently available for applications to use. This is not the total amount of memory in the system, memory in use by other applications is not counted, but it may include memory in use by the system cache that will immediately be made available should an application need it. Latency Time it takes for a single byte of memory to be transferred to the CPU for processing. Measured in nanoseconds, lower values are better. Threaded Nearly identical operations to the Read Uncached test, however being performed by separate threads simultaneously to test how well memory copes with multiple concurrent accesses. This test is run using a range of threads up the maximum available (eg 2, 4, 8 etc depending on available core count) and uses highest score.

It is bit strange that the cached and uncached results are the same. I suspect the cache is so large in these high end CPUs that a good portion of the RAM is still cached for the uncached test.

The write value dropping is also slightly strange, but it matches some other results we have for this CPU. So it isn't just your machine. Very likely it is a NUMA effect with the Dual CPUs or the extra RAM had some impact on the way the cache works in these CPUs. It is very common to get semi random memory performance with NUMA. It is big downside with dual CPU configurations.

[j7168908jx]

Thank you for your kindly reply! I would like to know if the test for memory read uncached will be updated in the next version. The EPYC 9754 has 128M L2 and 256M L3 cache for each CPU.

[David (PassMark)]

Read uncached is single threaded so only 1 CPU/Core will be used. The memory block used is already double the cache size, but maybe the pre-fetch is fast and the massive 12 channel memory controller also helps.

Each of the CPU cores in these Epyc CPUs are relatively slow. So likely they can't max out the cache and 12 channel RAM. (and this also explains why the threaded test reports huge numbers). These Epyc CPUs only start to make some sense when you have dozens of jobs running simultaneously.

But we'll have a look at all the tests when we do the next major release.