PerformanceTest V10 Beta Release

[Tim (PassMark)]

Changes in Beta 1

Note: The search script has been swapped over to a new version for this V10 release, at the moment searching for baselines will no longer return results from V9. This will be changed in future to return results from both V10 and V9 databases.

Advanced Memory Test
• Updated latency test to use same settings as standard test, running random range latency test for 32KB, 64Kb and 8MB ranges and then averaging the results.

Memory Tests
• Latency test, now will take measurements based on 32KB, 64KB and 8MB ranges (previously was just 64KB) and use the average value of the three tests for the score

2D Tests
• Added a new SVG image rendering test
• Rebalanced 2D Mark score due to new tests

Scripting
• Fixed 2 extra line breaks being added at end of records for REPORTSUMMARYCSV command
• Fixed a bug preventing the PDF test from running when scripted
• Added HIDEBASLINES flag to REPORTSUMMARYCSV script command

Misc
• Fixed some preferences dialog alignments
• CSV export, fixed an issue where multiple "unknown disk" entries could be output and change column ordering

[rofl]

Testing is going well with Beta 1, no failures so far over several runs on multiple systems with fresh 1909 installed. (no longer requires VC++ x86 to be manually installed)


Biggest ask at this point is not stopping at the results screen after every run while scripting multiple runs.

[Tim (PassMark)]

Changes in Beta 2

Baseline management
• Re-enabled choosing of colours for loaded baselines, this will only affect the bar graph colour of the baseline and the text colour will not change

Memory Tests
• Latency, changed lower range to 16KB instead of 32KB to give a broader range of samples.
• Database, removed a temporary debugging measure to limit to 2 threads.
• Threaded, fixed an integer overflow when calculating max amount of memory to use with a large number of test threads

Scripting
• Fixed an issue preventing the SVG test from being called correctly from as script
• Stopped submit baseline window being displayed when a script is executed

[rofl]

Beta 2 - if I launch my script the app flashes the test launch screen multiple times and they seem to instantly close. The tests never run.
Generic.ptscript.txt

[Tim (PassMark)]

Beta 2 - if I launch my script the app flashes the test launch screen multiple times and they seem to instantly close. The tests never run.
[ATTACH]n46607[/ATTACH]

We've just updated the beta 2 download to fix an issue with the RUN_ALL script command.

[rofl]

Seems to be working, thanks for the significant QOL improvement!

[Tim (PassMark)]

Changes in Beta 3

Memory Tests
• Moved the Database Operations test (based on SQLite 3.3.1.1) into separate executable to avoid slowdown when re-running the test on the high thread count systems. Slow down after multiple runs of the test was determined to be due to poor performance of the standard memory allocation heap functions. Starting a new process (instead of threads) fixed the problem and allowed the test to maintain constant performance over multiple runs. (Hard to understand how the SQLite developers hadn't already noticed and corrected this).

[rofl]

question and a request:

Q: Want to be sure the CPU tests are fully utilizing CPU for benchmarking, and a standard script command will accommodate any CPU of varying clock and core count.

SETDURATION 15
SETCPUTESTPROCESSES 2
RUN ALL

Are those 3 commands sufficient for any CPU type or are there any other switches needed to ensure max CPU utilization in testing?


Request: When writing an output to a hosted file location it seems it will keep the file open and locked after writing. This is an issue if you would like to have two machines running the test with the intention of both of them writing to the same file. Any way we could have multiple tests writing results to the same hosted file? (or better way to have a combined static output location)