Advanced network testing issues

[Ian (PassMark)]

Currently BurnInTest v6.0 is planned for early 2008. Prices are yet to be finalized. Historically, we have priced upgrades at 50% of the current retail price, with free upgrades for any licenses purchased within 6 months of the product release.

I will email you some Comark specific information.

Regards,
Ian

[Comark Corp]

The auto detection of NIC hardware could be improved for the case where "Test all NICs" is specified and there is more than 1 NIC. It is however worth noting that auto detection by its nature includes hardware that can be seen by the software. In some of the cases you describe, I suspect that the NIC would not be visible to Windows.

Actually, in order for Windows to "see" the NIC, all that it requires is a functioning PCI interface. If there is ANY defect between the NIC and it's external RJ45 (or other) interface, The only thing Windows would be able to detect would be that the cable is disconnected, whether it can send and receive packets, etc. It would still show up in the device manager, and be listed as functioning.

As such we believe the safest way to "Test all NICs" is for the user to specify the number of NIC's.

I agree. If there is any missing hardware in the system I would find it best if BurninTest would catch it. This holds true for serial, parallel and other devices as well.

While we have not yet done the design for the changes, we are thinking of making improvements along the following lines:
1) Change the test such that each IP address correlates to a NIC to be tested. Hence if there are 4 IP addresses specified in Preferences->Network, then there will be an attempt to test 4 NIC's.
2) Change the test around such that the Network 1, Network 2, ..., Network 4 displayed in the main window reflects the NIC under test, rather than the IP address destination.

We have started work on V6.0 of BurnInTest and when we have made the above changes I will send it to you.

Regards,
Ian

Thanks. I have two additional questions then:

1. What would be the upgrade cost for us? (I believe we have 40 some-odd licenses at this point.)

2. Do you anticipate this will happen before the end of the year?

Thanks again.

[Ian (PassMark)]

The auto detection of NIC hardware could be improved for the case where "Test all NICs" is specified and there is more than 1 NIC. It is however worth noting that auto detection by its nature includes hardware that can be seen by the software. In some of the cases you describe, I suspect that the NIC would not be visible to Windows. As such we believe the safest way to "Test all NICs" is for the user to specify the number of NIC's. While we have not yet done the design for the changes, we are thinking of making improvements along the following lines:
1) Change the test such that each IP address correlates to a NIC to be tested. Hence if there are 4 IP addresses specified in Preferences->Network, then there will be an attempt to test 4 NIC's.
2) Change the test around such that the Network 1, Network 2, ..., Network 4 displayed in the main window reflects the NIC under test, rather than the IP address destination.

We have started work on V6.0 of BurnInTest and when we have made the above changes I will send it to you.

Regards,
Ian

[Comark Corp]

Sorry I did not respond to your first (new) post.

If "Test all available NIC's" is checked then BurnInTest v5.3 will automatically route the test traffic over all Network adapters that are available and enabled on the system.

To be considered available, the NIC needs to have an IP address assigned. So, you are correct that NIC's without an IP address will not be included in the test. So you should connect the NIC's before testing.

This increases the risk of operator error allowing a system to pass, as well as the chance that a NIC port that is not functioning at all, will test as good. For instance, in a system with two or more NICs, if one of the following conditions is true for any one of the NIC ports:

1. The NIC cable is defective.
2. The NIC connector is damaged and not making contact with the cable.
3. The transformer is defective or completely missing from the system under test
4. There is a complete failure of the NIC chip's interface.
5. There are one or more open etches between the NIC chip and the transformer, or the transformer and the NIC connector.

The test will pass. I could come up with more reasons, but I think that illustrates my point fairly well.

Note: If there are no Network test operations, an error will be raised (e.g. no NICs are tested).

We will investigate changing this behavior in the next major version of BurnInTest.

Regards,
Ian

Thanks. So, when you say you will investigate changing this behavior in the next major version, does that mean that the failures I mention above will get caught? Also, the next major revision - does this mean 6.X or 5.X? Do you by any chance have a timeline for this?

Thanks again

[Ian (PassMark)]

Sorry I did not respond to your first (new) post.

If "Test all available NIC's" is checked then BurnInTest v5.3 will automatically route the test traffic over all Network adapters that are available and enabled on the system.

To be considered available, the NIC needs to have an IP address assigned. So, you are correct that NIC's without an IP address will not be included in the test. So you should connect the NIC's before testing.

Note: If there are no Network test operations, an error will be raised (e.g. no NICs are tested).

We will investigate changing this behavior in the next major version of BurnInTest.

Regards,
Ian

[Comark Corp]

Hello Ian,

Looks like this version (5.3 build 1010) doesn't work under certain conditions:
If I have one of the NICs disconnected *before* I start burnin with the standard networking test, and "Test all available NICs" checkbox enabled, the test passes. Doh!

Thanks!

Soo... Can anybody confirm this?

Thanks!

[Comark Corp]

Jay,

32-bit BurnInTest Professional V5.3.1007 with the Advanced Network changes that are outlined in this thread is now available from:
http://www.passmark.com/download/bit_download.htm

The 32-bit standard and 64-bit versions are not yet available and are planned for later this week.

Once again, sorry for the delay.

Regards,
Ian

Hello Ian,

Looks like this version (5.3 build 1010) doesn't work under certain conditions:
If I have one of the NICs disconnected *before* I start burnin with the standard networking test, and "Test all available NICs" checkbox enabled, the test passes. Doh!

Thanks!

[Ian (PassMark)]

Jay,

32-bit BurnInTest Professional V5.3.1007 with the Advanced Network changes that are outlined in this thread is now available from:
http://www.passmark.com/download/bit_download.htm

The 32-bit standard and 64-bit versions are not yet available and are planned for later this week.

Once again, sorry for the delay.

Regards,
Ian

[Ian (PassMark)]

Jay,

Thanks for the feedback. We have corrected this issue and plan to release a new build of BurnInTest at the end of the week.

Regards,
Ian

[Comark Corp]

Jay,

You can download a Beta of BurnIntest Pro with the updates to the Network test from here:
http://www.passmark.com/ftp/bitpro5.3.1006.beta1.exe

Please note that the software is a Beta release and I would say that the Advanced Network test component is pre-Beta.

Please let me know if you have problems with this build.
Regards,
Ian

Hello Ian,

The only thing I've noticed so far is the check box for "high bad packet ratio generates an error" is non-functional. Otherwise, this looks much better.

Thanks again!

[Ian (PassMark)]

Jay,

You can download a Beta of BurnIntest Pro with the updates to the Network test from here:
http://www.passmark.com/ftp/bitpro5.3.1006.beta1.exe

Please note that the software is a Beta release and I would say that the Advanced Network test component is pre-Beta.

Please let me know if you have problems with this build.
Regards,
Ian

[Comark Corp]

Hello Ian, thanks for the detailed update. Yes, I would like to download a beta version of this for testing. Is there some way I could download? Our mail server has a nasty habit of stripping executables, so I don't think you could email it to me direct. If this is not acceptable, you could send it to jayw_comark@yahoo.com - however I'm not sure what the maximum attachment size is. Thanks again for your help!

[Ian (PassMark)]

Jay,

Again, for what it is worth, my apologies. We have spent a lot of time working on this over the last 4 weeks. We have changed both the standard and advanced network tests but are still having some trouble getting the Advanced Network test to effectively manage some of the more obscure cases (such as not reporting false errors when you have PC's with Gigabit NICs and greatly different CPU speeds). If you asked me last week, I would have said it should be ready in about a week, however I would have said that the week before that, and then the same the week before that. Again, for what it is worth, my current estimate is 2 to 3 weeks.

In any case I will summarize what's happening below (for an upcoming release):

Standard Network Test:
We have increased the number of IP addresses from 4 to 6.
We have added an option "Test all available NICs", which will force traffic down every system NIC with a basic algorithm of NIC1 to IP Address 1, NIC2 to IP Address 2 etc.
I can send you a Beta of this if you would like.

Advanced Network test:
We have greatly simplified this test.
We have removed the UDP and FTP options. The Standard Network test can be used as a UDP test.
We have removed the Advanced Network test specific logging, and included all relevant logging in the standard BurnInTest logging mechanism.
We have replaced the complicated dynamic balancing of any system NIC to any Endpoint NIC with a simpler static allocation on test startup.
We have changed the error detection mechanism to detect errors much more quickly.
We have re-worked the errors reported.
We have changed the CPU throttling mechanism to reduce the CPU load.
I believe these changes will resolve the issues you have reported.

Regards,
Ian

[Comark Corp]

OK, it's been three months - any update yet?

With version 5.3 (1005) the dual NIC problem still exists, but single NIC problems are handled better than with version 5.1 (1014). Testing two or more NICs is of great concern for us. Is there a possibility of getting this resolved with a reasonable NRE charge?

Thanks.

No answer?

How about with an unreasonable NRE charge?

[Comark Corp]

I would guess 1 - 3 months.

Ian (PassMark)

OK, it's been three months - any update yet?

With version 5.3 (1005) the dual NIC problem still exists, but single NIC problems are handled better than with version 5.1 (1014). Testing two or more NICs is of great concern for us. Is there a possibility of getting this resolved with a reasonable NRE charge?

Thanks.