Strange results, 2D, Intel G43 chipset, possible bug in testing

[srr2]

I'm getting some very odd-looking and erratic results with two tests in the 2D suite on Intel G43 chipsets.

I'm comparing results on three different machines:

1) Q8400, G43 onboard GFX, 4GB RAM, Win7x64, Latest Intel driver, default timings (no overclock), ASRock Mobo.

2) Same as 1, except 6GB RAM and Intel Mobo.

3) Q9550, Nvidia 220GT, 8GB RAM, Win7x64, recent (but not latest) Nvidia driver, CPU overclocked, but default timings on everything else including GFX, Gigabyte Mobo.

As you can see, all are socket 775 with generally similar properties.

Most all the benchmark scores make sense and are proportional to expected system capabilities, except for the 2D suite of tests. Systems 1 and 2 are getting enormous and wildly-variable scores in "2D Complex Vectors" and "2D Image Rendering". System 3, with the Nvidia card is scoring far lower than the other two.

Specifically:
System 1: C.V. Run 1 - 873, Run 2 - 9308
System 2: C.V. Run 1 - 3059
System 3: C.V. Run 1 - 123

System 1: I.R. Run 1 - 14159 Run 2 - 7236
System 2: I.R. Run 1 - 3461
System 3: I.R. Run 1 - 422

I thought that perhaps the HPET might have something to do with this, but there are only negligible differences in the scores, with and without the HPET in a test I ran on System 2.

Now the problem ought to be obvious -- the Intel G43 chips are racking up scores that appear (to me) to be way out of line, and are varying wildly from trial to trial. The scores on from the Nvidia card are consistent from trial to trial to under 1% variation. Should these Intel G43 chipsets outperform a dedicated graphics card by these kinds of margins on these two tests? I doubt it. So what's the explanation? Here are two different sets of hardware, same OS, same chipset, same driver etc, both of which are achieving what appear to be impossible scores compared to a competent lower-end dedicated graphics card. BTW, the Nvidia easily outscores the G43s on all 3D tests. On the other 2D tests the scores are in the same ballpark.

Looks to me like there could be a bug in the benchmark test in systems where G43 chipsets are used.

[David (PassMark)]

You didn't mention what video was in system 2. You said system 2 is the same as system 1 except for the RAM and motherboard.
A single run across the 3 systems isn't really enough to justify claims of wildly-variable scores. There could have easily be some background activity taking place on system 1, than effected the score. Many dedicated graphics cards aren't particularly good with 2D and results are very dependent on the drivers in use, and the system settings.

[srr2]

What is unclear about "Same as 1"? Did you even read my post? The drivers ARE THE SAME.

The whole point of this is the wild variations in those two measurements on the G43 chipset. How can anyone justify a variation from 873 to 14159 measurement in the same hardware? Or justify a 10:1 variation from one run to the next? The resulting numbers are SO far off that they skew the whole system number. You tell me -- what other system, much less a G43 equipped system, scores 14159 in 2D Image Rendering? Huh? Which one?

No, it's not background processes or other things. There's nothing else going on in these except usual Windows 7 background stuff.

Fix it. Don't fix it. Up to you. I don't care. It's obvious that something's wrong here, and if you want to charge money for this stuff, IMO, it ought to deliver plausible results.

Ehhhhhh, nevermind. Forget it. I'm not about to argue with anyone. There's something clearly wrong here and if I were trying to sell benchmark software, I'd want to look into it. I'm done now.

[David (PassMark)]

You initially said system 2 had a different motherboard, so I assumed is must be different. So I just wanted to check what was in each machine and the that the results are repeatable before spending any time looking into it.

So I did some searching in our database, 0.02% of the ~500,000 results we have collected have stupidly high looking results for Image Rendering & 2D Complex Vectors.

Looking at the effected machines, the issue is not restricted to any particular CPU, motherboard, video card, software version, device driver nor operating system.

These tests are pretty simple. More or less they are like,

StartClock()
StartLoop
DisplayImage()
Loop Until Time is up.
StopClock()
Result = Number_Of_images / Time.

What we believe is happening is that the step to display the image on the screen is never actually executed by the device driver. There is a few plausible scenarios when this might happen. Some examples are, A) Connecting to the machine via some remote desktop products where screen updates are dropped in order to reduce the amount of data. B) Instances where the test window from PerformanceTest becomes obscured by another window. Once the Windows is hidden no further rendering in the window is required. We have taken some steps in the software to stop the window being obscured while a test is running, but it can still happen. C) There is some 3rd party software interfering with the display.

Running longer tests in PerformanceTest increases the chance of some external event.