Announcement

Collapse
No announcement yet.

Advanced network testing issues

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Advanced network testing issues

    Hello, we are looking into testing systems with 2 or more NIC interfaces, and have found the following possible issues with the testing:

    (UUT with two ethernet interfaces and running Windows 2000 SP4)
    1. Loading .bitcfg with two ethernet interfaces (but one connection is unplugged), Bit silently deselects the disconnected interface. This should be flagged as an error.
    2. Disconnecting (and leaving it this way) the second ethernet interface (as detected by Bit) while testing is not tagged as an error.
    3. Endpoint.exe is CPU intensive - is there a recommended server configuration for testing many systems concurrently?

    We are using version 5.1 (1014)

    Thanks for any help.
    Jay W.
    Diagnostic Engineer
    Comark Corporation
    93 West St.
    Medfield, MA 02052
    http://www.comarkcorp.com

  • #2
    I will start by saying that the Advanced Network test was initially written for a specific purpose. As a result, the advanced network test is actually more complicated than it probably needs to be. We plan to modify the Advanced Network test in V5.2 due around the end of this month to remove some of this complexity (that we believe is not being used by our customers and that is just confusing).

    As part of this V5.2 work will will investigate your post further. My initial comments are:
    1) Yes an error should be raised.
    2) This should raise an error now, there may be a case when then does not occur - we'll investigate further.
    3) We will investigate reducing the CPU utilization of Endpoint. The best configuration would depend on exactly what you are testing. If you are testing the 2 NICs on the 'BurnInTest' system (and not the intermediate network gear or endpoint systems) then a single Endpoint would suffice, but 2 may allow a higher loading level. If you are looking at testing a many servers (eg. 10 servers running BurnInTest's advanced network test), then I would probably advise that you don't do this (if you do want to do this let me know as this is a candidate for removal from the BurnInTest Advanced Network test).

    Best regardes,
    Ian

    Comment


    • #3
      Hello Ian,

      All we are looking to do is to test multi-homed systems. There are configurations of ours that may include 6 or more ethernet interfaces.

      As for version 5.2 - do you have any idea when this may come out? If we need to do ISO9001:2000 validation ourselves (see my other new thread regarding this that I posted today) I would not want to have to validate 5.1 (1014) now, only to have to do it all over again when 5.2 is released.

      Thanks again.
      Jay W.
      Diagnostic Engineer
      Comark Corporation
      93 West St.
      Medfield, MA 02052
      http://www.comarkcorp.com

      Comment


      • #4
        Sorry for the delay in responding. We plan to release V5.2 at the end of this week, 2 February. Unfortunately, it may not include the Advanced Networking work I mentioned. I expect this may in a patch release after v5.2.

        Sorry.

        Best regards,
        Ian

        Comment


        • #5
          Originally posted by Ian (PassMark) View Post
          Sorry for the delay in responding. We plan to release V5.2 at the end of this week, 2 February. Unfortunately, it may not include the Advanced Networking work I mentioned. I expect this may in a patch release after v5.2.

          Sorry.

          Best regards,
          Ian
          Thanks Ian.

          Noted. I did test 5.2 and found out that the problem still exists. Do you have any idea when you might be patching this? It would seem that we are shipping more boards with dual NICs and this is becoming a critical problem for us.

          Thanks again.
          Jay W.
          Diagnostic Engineer
          Comark Corporation
          93 West St.
          Medfield, MA 02052
          http://www.comarkcorp.com

          Comment


          • #6
            Originally posted by Comark Corp View Post
            Do you have any idea when you might be patching this? It would seem that we are shipping more boards with dual NICs and this is becoming a critical problem for us.

            Thanks again.
            Bump.

            Even a guess would be better than nothing.

            Thanks.
            Jay W.
            Diagnostic Engineer
            Comark Corporation
            93 West St.
            Medfield, MA 02052
            http://www.comarkcorp.com

            Comment


            • #7
              Additional information regarding the network test.

              In the past, I wrote a program that handled multi-homed systems by using the socket bind function for the clients. This allows binding clients to a specific network interface. If this step is not performed the connect call uses the wildcard (zeroes) for the local IP address.

              Comment


              • #8
                I would guess 1 - 3 months.

                Ian (PassMark)

                Comment


                • #9
                  Originally posted by Ian (PassMark) View Post
                  I would guess 1 - 3 months.

                  Ian (PassMark)
                  OK, it's been three months - any update yet?

                  With version 5.3 (1005) the dual NIC problem still exists, but single NIC problems are handled better than with version 5.1 (1014). Testing two or more NICs is of great concern for us. Is there a possibility of getting this resolved with a reasonable NRE charge?

                  Thanks.
                  Jay W.
                  Diagnostic Engineer
                  Comark Corporation
                  93 West St.
                  Medfield, MA 02052
                  http://www.comarkcorp.com

                  Comment


                  • #10
                    Originally posted by Comark Corp View Post
                    OK, it's been three months - any update yet?

                    With version 5.3 (1005) the dual NIC problem still exists, but single NIC problems are handled better than with version 5.1 (1014). Testing two or more NICs is of great concern for us. Is there a possibility of getting this resolved with a reasonable NRE charge?

                    Thanks.
                    No answer?

                    How about with an unreasonable NRE charge?
                    Jay W.
                    Diagnostic Engineer
                    Comark Corporation
                    93 West St.
                    Medfield, MA 02052
                    http://www.comarkcorp.com

                    Comment


                    • #11
                      Jay,

                      Again, for what it is worth, my apologies. We have spent a lot of time working on this over the last 4 weeks. We have changed both the standard and advanced network tests but are still having some trouble getting the Advanced Network test to effectively manage some of the more obscure cases (such as not reporting false errors when you have PC's with Gigabit NICs and greatly different CPU speeds). If you asked me last week, I would have said it should be ready in about a week, however I would have said that the week before that, and then the same the week before that. Again, for what it is worth, my current estimate is 2 to 3 weeks.

                      In any case I will summarize what's happening below (for an upcoming release):

                      Standard Network Test:
                      We have increased the number of IP addresses from 4 to 6.
                      We have added an option "Test all available NICs", which will force traffic down every system NIC with a basic algorithm of NIC1 to IP Address 1, NIC2 to IP Address 2 etc.
                      I can send you a Beta of this if you would like.

                      Advanced Network test:
                      We have greatly simplified this test.
                      We have removed the UDP and FTP options. The Standard Network test can be used as a UDP test.
                      We have removed the Advanced Network test specific logging, and included all relevant logging in the standard BurnInTest logging mechanism.
                      We have replaced the complicated dynamic balancing of any system NIC to any Endpoint NIC with a simpler static allocation on test startup.
                      We have changed the error detection mechanism to detect errors much more quickly.
                      We have re-worked the errors reported.
                      We have changed the CPU throttling mechanism to reduce the CPU load.
                      I believe these changes will resolve the issues you have reported.

                      Regards,
                      Ian

                      Comment


                      • #12
                        Hello Ian, thanks for the detailed update. Yes, I would like to download a beta version of this for testing. Is there some way I could download? Our mail server has a nasty habit of stripping executables, so I don't think you could email it to me direct. If this is not acceptable, you could send it to jayw_comark@yahoo.com - however I'm not sure what the maximum attachment size is. Thanks again for your help!
                        Jay W.
                        Diagnostic Engineer
                        Comark Corporation
                        93 West St.
                        Medfield, MA 02052
                        http://www.comarkcorp.com

                        Comment


                        • #13
                          Jay,

                          You can download a Beta of BurnIntest Pro with the updates to the Network test from here:
                          http://www.passmark.com/ftp/bitpro5.3.1006.beta1.exe

                          Please note that the software is a Beta release and I would say that the Advanced Network test component is pre-Beta.

                          Please let me know if you have problems with this build.
                          Regards,
                          Ian

                          Comment


                          • #14
                            Originally posted by Ian (PassMark) View Post
                            Jay,

                            You can download a Beta of BurnIntest Pro with the updates to the Network test from here:
                            http://www.passmark.com/ftp/bitpro5.3.1006.beta1.exe

                            Please note that the software is a Beta release and I would say that the Advanced Network test component is pre-Beta.

                            Please let me know if you have problems with this build.
                            Regards,
                            Ian
                            Hello Ian,

                            The only thing I've noticed so far is the check box for "high bad packet ratio generates an error" is non-functional. Otherwise, this looks much better.

                            Thanks again!
                            Jay W.
                            Diagnostic Engineer
                            Comark Corporation
                            93 West St.
                            Medfield, MA 02052
                            http://www.comarkcorp.com

                            Comment


                            • #15
                              Jay,

                              Thanks for the feedback. We have corrected this issue and plan to release a new build of BurnInTest at the end of the week.

                              Regards,
                              Ian

                              Comment

                              Working...
                              X