Tweet
PerformanceTest V10 has now been released, thanks to everyone who helped during the beta testing.
Current Version: V10 Beta Final
Date: 3/March/2020
Download link: PerformanceTest V10
Latest changes: See changes in beta 3.
Please contact us to report any issues found in this build.
Baseline submissions: Currently baselines can be submitted but the returned link to display them will not work (they are being stored in a different database to what is live on our current site).
What's new
Score rebalancing
• Due to the large amount of changes made to the 2D/3D/CPU/Disk tests all the calculated mark values have been rebalanced and scaled to be similar (but not exactly the same) to that of PerformanceTest 9.
• Individual test scores have not been scaled so a direct comparison cannot be made in many cases between version 9 and version 10.
Windows Support
• No longer supporting Windows version older than Vista.
Compiler updates
• We switched from using Visual Studio 2013 (for V9) to Visual Studio 2017 for V10 Alpha. Likely we will move to Visual Studio 2019 before the first beta release. 2019 is being tested at the moment. Newer compiler versions typically bring improved code optimisation and use of newer CPU instructions.
CPU Tests
• Enabled compiler optimisations for the CPU tests that had previously been disabled. This has resulted in better performance on newer CPUs when compared to older ones.
• Compression Test, Replace the existing algorithm with a new version that uses the Cypto++ Gzip library. Previous versions of PerformanceTest were using a Adaptive encoding algorithm, which gave good compression rates on text, but was't in common use. Zip is the defacto standard for real world data compression.
• Encryption test, removed Salsa and TwoFish from the sub tests that are run and replaced them with an ECDSA (Elliptic Curve Digital Signature Algorithm) sub test. Previously PerformanceTest V9 had 4 sub-tests. These were, TwoFish, AES, Salsa20 & SHA256. Now we have three sub-tests. AES, ECDSA & SHA256. These are all algorithms that are heavily used in the real world.
• Extended Instructions (SSE), added an AVX512 test (when available). AVX512 are a new set of CPU instructions that are in newer CPUs.
• Extended Instructions (SSE), fixed a pointer math issue in the test that was referencing incorrect memory locations. Accessing the correct data helped with data alignment and improved test throughput.
• Extended Instructions (SSE), made some changes to the SSE/AVX/FMA tests for how the results are retrieved and stored for next calculation loop (using the _mm_storeu_ps and mm256_storeu_ps CPU instructions).
• Extended Instructions (SSE), removed custom aligned vector/matrix class and switched to standard vector/matrix class. Changed how matrix data is loaded before test (using the _mm256_broadcast_ps CPU instruction).
• Integer Math, made some changes to add better support for out of order execution. This makes the algorithm less linear and gives modern CPUs the change to get more calculations done in parallel.
• Physics test. Updated Bullet physics engine to version 2.88. Previously were were using 2.83.
• Added a Cross-platform mark to the CPU test, made up of the Integer, Floating point, sorting and prime test scores. This will be calculated when loading a V9 baseline in V10 if the required scores are available. This cross platform score is not used when generating the overall CPU mark as it based on previously run tests. We envisage that we'll use these results in the future for cross platform comparisons (x86 PCs vs ARM based mobile devices).
2D Tests
• Added a new “PDF Render” test that will open a PDF and scroll through the available pages
• Changed default test size to 1920 x 1080 this tend to place more load on the video card than was previously the case. So frame rates are lower across the board compared to V9. Tests will scale down in size to 1024x768 and a penalty will be applied and will not run at lower resolutions than this.
• Direct 2D, increased amount of objects displayed during test.
• Windows Interface test, increased size of dialog and number of controls on dialog.
3D Tests
• DX9, changed default resolution to 1920x1080, test will be scaled down and a penalty applied if it cannot be run at this resolution.
• DX9, increased default Anti Aliasing level to 8, changed skybox and number of objects in scene. This was done with the aim of making the test less CPU bound.
• DX10, increased default resolution to 1920x1080, test will be scaled down and a penalty applied if it cannot be run at this resolution. This adds load to the video card.
• DX10, increased number of islands and meteors during test
• DX10, fixed a bug when enumerating display modes for the DX10 test where no compatible card would be found if there was a large amount (>500) of display modes
• GPU Compute , replaced the Bittonic sort test with an NBody Gravity test. Nobody knew what a Bittonic sort was or how it applied to the real world. NBody simulations on the other hand are a fairly common scientific application and they make a good visual impact.
• GPU Compute, increased default size for sub tests to 1920x1080
• GPU Compute, OpenCL test, made some changes to particle size and variables used for calculations so more is happening during the test.
• Started allowing ”Microsoft RemoteFX Graphics Device” for 3D support on some VMs
Disk tests
• Changed name of “Random Seek” test to “32KQD20” to better represent the test (using 32K block size with a queue depth of 20)
• Added a new “4KQD1” test (using 4k block size with a queue depth of 1)
• Increased test files size to 400MB for write test, 800MB for read test (traditional hard drive). If drive is an SSD then it is 1GB for write test and 2GB for read test.
• Removed the CD test. Surely nobody was using this anymore?
Advanced Physics Test
• Added option to allow resolution to be selected
• Updated Bullet physics engine to version 2.88. Previously were were using 2.83.
Advanced Network Test
• Added threading and the new Windows RIO sockets option. RIO is Registered Input/Output (RIO) API Extensions. These changes were made to achieve higher throughput & lower latency, particularly on 10Gb+ networks. You should now be able to really push the limits of your networking gear with these changes.
Advanced Disk Test
• Made some changes to try and stop crashes during the advanced disk IOPS test when the hard drive is failing/responding abnormally
• Added temperature collection to test results. Can now choose to display the temperature or the latency heat map when displaying a graph. This has become important as some SSDs throttle down their speed under high temperatures.
NEW Advanced CPU Test
• Added an advanced CPU test, this allows individual CPU tests to be run from 1 thread up to a specified number of threads and then the results graphed.
NEW Advanced Internet Speed test
• Added an implementation of M-Labs (https://www.measurementlab.net) internet speed test that will connect to their servers, performance a 10 second upload test, a 10 second download test and then display the results from that test.
Localisation
• Added a section in the installer to allow selection of a supported language, on install first launch PT will now choose that language by default.
Reporting
• Added percentile options to exported text, formatted text and HTML reports.
Baseline Management
• Added a way of tracking submitted baselines and displaying them in the advanced baseline management dialog under the “Uploaded” tab
Scripting
• Changed "Result Date" to be local time instead of UTC for consistency (windows install time was already local time)
• Added HIDEBASELINES option to only add current computer results to exported results files
• HTML report output, fixed a bug where the Unicode BOM was not being correctly written to the start of the file if the file was opened in append mode and didn't already exist
Misc
• Chart display, added percentage difference to baselines when a score the "this computer" is available. Also added option in the preferences to enable/disable.
• Fixed an incorrect error message in the advanced drive performance test when a selected disk did not have enough free space
• Fixed a bug when saving results to an image, the scrollbar width was being applied and causing some result to be hidden.
• Added a gray rectangle to custom list view header so the boundary that can be used to resize the columns is highlighted
Current Version: V10 Beta Final
Date: 3/March/2020
Download link: PerformanceTest V10
Latest changes: See changes in beta 3.
Please contact us to report any issues found in this build.
Baseline submissions: Currently baselines can be submitted but the returned link to display them will not work (they are being stored in a different database to what is live on our current site).
What's new
Score rebalancing
• Due to the large amount of changes made to the 2D/3D/CPU/Disk tests all the calculated mark values have been rebalanced and scaled to be similar (but not exactly the same) to that of PerformanceTest 9.
• Individual test scores have not been scaled so a direct comparison cannot be made in many cases between version 9 and version 10.
Windows Support
• No longer supporting Windows version older than Vista.
Compiler updates
• We switched from using Visual Studio 2013 (for V9) to Visual Studio 2017 for V10 Alpha. Likely we will move to Visual Studio 2019 before the first beta release. 2019 is being tested at the moment. Newer compiler versions typically bring improved code optimisation and use of newer CPU instructions.
CPU Tests
• Enabled compiler optimisations for the CPU tests that had previously been disabled. This has resulted in better performance on newer CPUs when compared to older ones.
• Compression Test, Replace the existing algorithm with a new version that uses the Cypto++ Gzip library. Previous versions of PerformanceTest were using a Adaptive encoding algorithm, which gave good compression rates on text, but was't in common use. Zip is the defacto standard for real world data compression.
• Encryption test, removed Salsa and TwoFish from the sub tests that are run and replaced them with an ECDSA (Elliptic Curve Digital Signature Algorithm) sub test. Previously PerformanceTest V9 had 4 sub-tests. These were, TwoFish, AES, Salsa20 & SHA256. Now we have three sub-tests. AES, ECDSA & SHA256. These are all algorithms that are heavily used in the real world.
• Extended Instructions (SSE), added an AVX512 test (when available). AVX512 are a new set of CPU instructions that are in newer CPUs.
• Extended Instructions (SSE), fixed a pointer math issue in the test that was referencing incorrect memory locations. Accessing the correct data helped with data alignment and improved test throughput.
• Extended Instructions (SSE), made some changes to the SSE/AVX/FMA tests for how the results are retrieved and stored for next calculation loop (using the _mm_storeu_ps and mm256_storeu_ps CPU instructions).
• Extended Instructions (SSE), removed custom aligned vector/matrix class and switched to standard vector/matrix class. Changed how matrix data is loaded before test (using the _mm256_broadcast_ps CPU instruction).
• Integer Math, made some changes to add better support for out of order execution. This makes the algorithm less linear and gives modern CPUs the change to get more calculations done in parallel.
• Physics test. Updated Bullet physics engine to version 2.88. Previously were were using 2.83.
• Added a Cross-platform mark to the CPU test, made up of the Integer, Floating point, sorting and prime test scores. This will be calculated when loading a V9 baseline in V10 if the required scores are available. This cross platform score is not used when generating the overall CPU mark as it based on previously run tests. We envisage that we'll use these results in the future for cross platform comparisons (x86 PCs vs ARM based mobile devices).
2D Tests
• Added a new “PDF Render” test that will open a PDF and scroll through the available pages
• Changed default test size to 1920 x 1080 this tend to place more load on the video card than was previously the case. So frame rates are lower across the board compared to V9. Tests will scale down in size to 1024x768 and a penalty will be applied and will not run at lower resolutions than this.
• Direct 2D, increased amount of objects displayed during test.
• Windows Interface test, increased size of dialog and number of controls on dialog.
3D Tests
• DX9, changed default resolution to 1920x1080, test will be scaled down and a penalty applied if it cannot be run at this resolution.
• DX9, increased default Anti Aliasing level to 8, changed skybox and number of objects in scene. This was done with the aim of making the test less CPU bound.
• DX10, increased default resolution to 1920x1080, test will be scaled down and a penalty applied if it cannot be run at this resolution. This adds load to the video card.
• DX10, increased number of islands and meteors during test
• DX10, fixed a bug when enumerating display modes for the DX10 test where no compatible card would be found if there was a large amount (>500) of display modes
• GPU Compute , replaced the Bittonic sort test with an NBody Gravity test. Nobody knew what a Bittonic sort was or how it applied to the real world. NBody simulations on the other hand are a fairly common scientific application and they make a good visual impact.
• GPU Compute, increased default size for sub tests to 1920x1080
• GPU Compute, OpenCL test, made some changes to particle size and variables used for calculations so more is happening during the test.
• Started allowing ”Microsoft RemoteFX Graphics Device” for 3D support on some VMs
Disk tests
• Changed name of “Random Seek” test to “32KQD20” to better represent the test (using 32K block size with a queue depth of 20)
• Added a new “4KQD1” test (using 4k block size with a queue depth of 1)
• Increased test files size to 400MB for write test, 800MB for read test (traditional hard drive). If drive is an SSD then it is 1GB for write test and 2GB for read test.
• Removed the CD test. Surely nobody was using this anymore?
Advanced Physics Test
• Added option to allow resolution to be selected
• Updated Bullet physics engine to version 2.88. Previously were were using 2.83.
Advanced Network Test
• Added threading and the new Windows RIO sockets option. RIO is Registered Input/Output (RIO) API Extensions. These changes were made to achieve higher throughput & lower latency, particularly on 10Gb+ networks. You should now be able to really push the limits of your networking gear with these changes.
Advanced Disk Test
• Made some changes to try and stop crashes during the advanced disk IOPS test when the hard drive is failing/responding abnormally
• Added temperature collection to test results. Can now choose to display the temperature or the latency heat map when displaying a graph. This has become important as some SSDs throttle down their speed under high temperatures.
NEW Advanced CPU Test
• Added an advanced CPU test, this allows individual CPU tests to be run from 1 thread up to a specified number of threads and then the results graphed.
NEW Advanced Internet Speed test
• Added an implementation of M-Labs (https://www.measurementlab.net) internet speed test that will connect to their servers, performance a 10 second upload test, a 10 second download test and then display the results from that test.
Localisation
• Added a section in the installer to allow selection of a supported language, on install first launch PT will now choose that language by default.
Reporting
• Added percentile options to exported text, formatted text and HTML reports.
Baseline Management
• Added a way of tracking submitted baselines and displaying them in the advanced baseline management dialog under the “Uploaded” tab
Scripting
• Changed "Result Date" to be local time instead of UTC for consistency (windows install time was already local time)
• Added HIDEBASELINES option to only add current computer results to exported results files
• HTML report output, fixed a bug where the Unicode BOM was not being correctly written to the start of the file if the file was opened in append mode and didn't already exist
Misc
• Chart display, added percentage difference to baselines when a score the "this computer" is available. Also added option in the preferences to enable/disable.
• Fixed an incorrect error message in the advanced drive performance test when a selected disk did not have enough free space
• Fixed a bug when saving results to an image, the scrollbar width was being applied and causing some result to be hidden.
• Added a gray rectangle to custom list view header so the boundary that can be used to resize the columns is highlighted
Comment