Consumer Security Products Performance Benchmark Doubts

[BearMan]

Hi,

I'm a regular reader of your perfomance reports and I wish question you about some doubts which I have found in the last one .

Firstly, I have noticed about that you have reduced the amount of products: In the previous report (February 2011) they were 14 and in this one there's only 5. Why have you chosen these ones excluding other ones?

My second question its more specific; it's about "Benchmark 4 – Memory Usage during System Idle" test. I saw (comparing the results with February's report) that the amount of RAM consumed by the products has increased considerably (excepting in the case of Avast!). Did you change your methodology for this test? I have readed the proper section but aparently nothing has changed since the previous report.

In addition, I will like to question you about which performance counter did you monitor to calculate the results of this test; "Working Set" of each process, and then make a sum of all Product's processes?

It's possible to download "perflog++" from anywhere?

Thanks for your helping!

Regards.

[David (PassMark)]

Why have you chosen these ones excluding other ones?

We in fact did the same last year. Tested products as they released their 2012 products and did several editions of the report. So there will be more products later in the year.

The RAM usage during idle test was more or less the same. Without doing a detailed investigation the only factor that comes to mind is the O/S change. We were using
Windows 7 Ultimate (64-bit) SP0 last year, but we are now on SP1. It is possible there might have been a change in the test order. e.g. looking at idle RAM usage towards the end of the run rather than near the start. Again, it would need some investigation. But as we aren't comparing last years results to this years in the report each report should stand on its own.

Yes, working set was used. Data was collected via a program not via the WMI performance counters.
perflog++ isn't available, but check out this MSDN page about GetProcessMemoryInfo(),
http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx