Announcement

Collapse
No announcement yet.

PerformanceTest 9.0 score variations between boards with the same spec

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • PerformanceTest 9.0 score variations between boards with the same spec

    Hi,

    We ran PerformanceTest 9.0 on 3 boards of the same model (same CPU, BIOS settings and peripheral devices). However, there are both inter-board and intra-board (3 trials per board) variations of the scores, especially the CPU Mark, usually the 1st trial being the lowest. How large is the typical deviation for the scores from PerformanceTest 9.0?

    Thank you!
    Board #1 Board #2 Board #3
    H/W Version ZR0 ZR0 ZR0 ZR0 ZR0 ZR0 ZR0 ZR0 ZR0
    BIOS Version v0.0.3 v0.0.3 v0.0.3 v0.0.3 v0.0.3 v0.0.3 v0.0.3 v0.0.3 v0.0.3
    EC Version 90726T01 90726T01 90726T01 90726T01 90726T01 90726T01 90726T01 90726T01 90726T01
    CPU i7-8665UE i7-8665UE i7-8665UE i7-8665UE i7-8665UE i7-8665UE i7-8665UE i7-8665UE i7-8665UE
    Memory J3 8G
    (WARIS2133)
    J3 8G
    (WARIS2133)
    J3 8G
    (WARIS2133)
    J3 8G
    (WARIS2133)
    J3 8G
    (WARIS2133)
    J3 8G
    (WARIS2133)
    J3 8G
    (WARIS2133)
    J3 8G
    (WARIS2133)
    J3 8G
    (WARIS2133)
    Clockspeed 1.7 GHz 1.7 GHz 1.7 GHz 1.7 GHz 1.7 GHz 1.7 GHz 1.7 GHz 1.7 GHz 1.7 GHz
    Turbo Speed Up to 4.4 GHz Up to 4.4 GHz Up to 4.4 GHz Up to 4.4 GHz Up to 4.4 GHz Up to 4.4 GHz Up to 4.4 GHz Up to 4.4 GHz Up to 4.4 GHz
    Core 4 4 4 4 4 4 4 4 4
    Storage Waris SSD
    128GB
    Waris SSD
    128GB
    Waris SSD
    128GB
    Waris SSD
    128GB
    Waris SSD
    128GB
    Waris SSD
    128GB
    Waris SSD
    128GB
    Waris SSD
    128GB
    Waris SSD
    128GB
    OS Win 10 64 bits Win 10 64 bits Win 10 64 bits Win 10 64 bits Win 10 64 bits Win 10 64 bits Win 10 64 bits Win 10 64 bits Win 10 64 bits
    PassMark Rating 3313 3338 3404 2955 3033 3006 3327 3380 3377
    CPU Mark 8438 8480 8438 7946 8365 8347 8254 8885 8818
    2D Graphics Mark 811 841 851 705 720 722 834 843 846
    3D Graphics Mark 1119 1125 1128 951 948 953 1122 1120 1126
    Memory Mark 2053 2078 2103 1841 1863 1859 2081 2089 2065
    Disk Mark 3127 2988 3145 2954 3201 2943 3098 3058 3068
    Best,
    Harry

  • #2
    Board #2 looks like it has slower RAM in it.

    Variations from one run to the next depend on what background tasks are running (often randomly) on the machine. e.g. having a windows update or Anti-virus update occur mid test might result in a large variation. Thermal throttling can also impact the results. As can if the machine is running on battery power (or has a low battery, depending on Windows power settings).

    On a modern Windows machine there is always a massive amount of background activity taking place. Most of is random when it occurs. This impacts the consistency of benchmarking.

    If you want to take a look download Microsoft's Process Monitor tool
    https://docs.microsoft.com/en-us/sys...nloads/procmon

    And also have a look at the background Network activity with Wireshark.
    https://www.wireshark.org/

    You'll be amazed what is going on without your knowledge.

    Comment


    • #3
      Hi David,

      The same RAM was used on all 3 boards as well as the heat sink. No battery was used, only AC source.The internet was not connected so no windows update was available. So i don't think these would be the source of variation.

      As for Windows background activities, is there any way of avoiding the possible variations from the background activities when using the PerformanceTest? Or is there a Linux-based PerformanceTest tool available?

      In addition, according to your experience, what is the approximate variation of the performance score? We just want to make sure that the variations we got are within the normal range.

      Thanks!

      Best,
      Harry

      Comment


      • #4
        Still looks like there was a difference in the RAM's performance to me. Maybe BIOS timing setting or the RAM not being in dual channel mode, or something else. Looking at the individual results might tell you more.

        The way to avoid some of the variability is to,
        A) Run longer tests AND/OR
        B) Run multiple test runs and take the max value.
        This can be done in an automated fashion from the Preferences window in PerformanceTest. e.g. do 10 runs on each platform.

        If you have a lot of time, you can also go through all the services and startup apps running in the background in Windows and disable a bunch of them. But it is hard to figure out which ones are essential and which are safe to disable.

        Linux also has a fair amount of background activity. It is just the way modern operating systems work.

        Comment


        • #5
          Hi David,

          We have double-checked that the testing environment (including RAM and BIOS settings) was all the same except for different boards of the same model. Currently, we are using the PerformanceTest as a part of the testing procedure to demonstrate our capabilities to our customers. Therefore, is there a final suggestion for how large would the variation (in the same board for example) typically be?

          Thanks!

          Best,
          Harry

          Comment


          • #6
            It depends on the particular test and the environment. There are valid factors (some mentioned above) that can cause a very large variation in scores.
            Disk systems (especially traditional HDD) can have large variations as performance depends on the placement of the files on the disk and fragmentation states. Which can vary from 1 minute to the next. Memory systems can also vary, NUMA & ECC error correction in particular can cause large differences on higher end systems as can the number of channels in use. Windows display settings and DPI settings can effect 2D performance. Drivers & SLI options 3D. The list is nearly endless.

            But you can collect the real life variance yourself from within the PerformanceTest software.

            Here for example is the real life distribution for the popular i7-8700K CPU in the CPUMark result (as of 23/Aug/2019).

            Click image for larger version

Name:	CPUMark-variance.png
Views:	424
Size:	14.8 KB
ID:	45263

            Here for example is the distribution for the popular Samsung 970 EVO SSD in the sequential read result.

            Click image for larger version

Name:	DiskRead-variance.png
Views:	362
Size:	16.3 KB
ID:	45264

            If you ran 10 tests and take the max result, in a very tightly controlled environment, then I would expect a variation of less that 3% in pretty much all of the tests.

            Comment


            • #7
              Hi David,

              Got it. So according to the CPU Mark figure you provided, it seems that even with the same CPU but different board configurations (such as more supported IO ports could result in decreased CPU performance, etc.), there would be a large variation in the test scores right?

              Comment


              • #8
                Yes, there can be a large variation between machines with the same CPU, for all the reasons stated above and more. e.g. Overclocking was not mentioned yet.

                There is another list of factors here that can effect performance.
                https://www.passmark.com/forum/perfo...-for-a-slow-pc

                Comment


                • #9
                  OK, thanks a lot for the quick response!

                  Comment

                  Working...
                  X