Tweet
I recently compared my PassMark rating (2310916) of 12,810 to someone else's score of 15,559. (2300126) Funny thing is, my PC spanked theirs in every individual benchmark score, despite their higher overall rating. What’s up with that?
-
-
Maybe you read the result wrong. Baseline 2310916 seems to have a rating of 18,089.
(also there was a few results where baseline 2300126 was better, for example a couple of the disk results)
-
Here are a few screen shotsComment
-
I see at least part of the problem.
Benchmark baseline 2310916 was done with V10 of the PerformanceTest software (from a few years ago).
Benchmark baseline 2300126 was done with V11.
So not strictly an apples to apples comparison, as there are different benchmarks in each version.
Comment
-
I actually did notice the version difference but was more focused on the submission dates and didn’t realize different versions would make such a big difference. The question is, why let older versions run at all? Why not force an update or, at the very least, inform the user of potential discrepancies that could occur between PassMark scores and software versions?Comment
-
"(also there was a few results where baseline 2300126 was better, for example a couple of the disk results)"
With the updated Passmark version my PC smashed all of those disk results by a huge margin
Comment
-
The current V11 release doesn't run on older systems (for example we no longer test or support on Win7 or
Win8. So we make the old releases available for people with old hardware.
It would be kind of rude for older software to disable itself and not run.
Comparisons between versions are difficult to make. We re-scale V10 results in the PerformanceTest V11 software to make them comparable.
From the V11 release notes
Comment
-
While I appreciate the effort to maintain compatibility for older systems, I feel my original concern may have been misunderstood.
My suggestion wasn’t to disable older software arbitrarily, which indeed could inconvenience users with legacy hardware. Rather, my point was about enhancing user communication to prevent confusion regarding performance results and software scaling. If comparisons between versions are inherently inconsistent due to re-scaling or updates, wouldn't it be beneficial to:- Notify Users: Prompt users of older versions with a clear message indicating that results may not align with the latest standards and advising an update for accurate comparisons.
- Force Manual Acknowledgment: If users opt to continue with the older version, require a confirmation acknowledging potential discrepancies.
- Flag Results: Add a watermark or disclaimer in the generated results, explicitly stating the version used and its implications on comparability.
These steps could help users make informed decisions and avoid scenarios where outdated results cause confusion or misinterpretations, as happened in my case. I believe this approach would strike a balance between respecting users with older systems and ensuring clarity for everyone.
Comment
-
Old software isn't aware of what software there is available in the future. So if you downloaded the install package 3 years ago, we don't have any control over it. Yes, there is an option in the software to check if a new version available, but you didn't use it, I assume.
In many cases the changes between versions are minor and all the benchmark results are comparable, or very nearly comparable.
Version numbers are already noted in the results. You said yourself you noticed the version difference.
There is an element of common sense required as well.
Comment
-
Your response, comes across as dismissive and unnecessarily condescending. Suggesting that I lacked "common sense" in identifying discrepancies between versions undermines the very purpose of benchmarking software: to provide clarity and reliability, not confusion.
You mentioned that "old software isn't aware of what software there is available in the future." With respect, this is not a matter of foresight but of functionality. Pop-up reminders or update notifications are standard practices in modern software development and could easily alert users when a newer version is available. Expecting users to manually check for updates when the software itself could streamline this process is unreasonable.
The difference between scores, 12,810 in the older version and 18,495 in the new, is substantial, not minor. Expecting users to intuitively grasp such changes without proper communication reflects poorly on the software's usability.
A simple alert or disclaimer would have sufficed. Blaming the user, or implying a lack of foresight in the software, does little to address the issue. I trust you will consider this feedback more seriously in the future.
Comment
-
We think it is annoying when software self-updates, or pops up messages every few weeks.
So we made it a manual update check (under the Help menu).
We've made a change to the web site today to make it more obvious when you are looking at older V10 results.
Example page
https://www.passmark.com/baselines/V...d=231091615958
Comment
Comment