V5.1.0 test fail on Bay Trail

[stevensxl]

logfile:https://drive.google.com/file/d/0Bxm_1t0QPEJIV05VX3ZBZ2MxWXM/edit?usp=sharing
I think the previous hang on test 2 issue has been solved, because this time we passed 3 test cycles without error. But the test was running very slow and it finally hang on test 7 of the last cycle. The "slow" I just mentioned, I mean during the test, system will hang on some random stages, the time is stopped counting too. But when I press ESC, I can call out the menu and then press 0 test will back to running for a while. So the whole test cost me more than 4 hours.

[keith]

Looks like we are making progress. It is likely the 'hang' is due to constantly writing to the log file, significantly slowing down the memory tests. Since the workaround seems to be working, the additional logging can be removed. Please give this build a try and let us know if the results:

http://www.passmark.com/ftp/memtest8....1.1b.1012.zip

[stevensxl]

test failed, hang on test 3
log file: https://drive.google.com/file/d/0Bxm...it?usp=sharing

[keith]

Looks like this 'hang' was caused by the same firmware multiprocessor issue, but on a separate occasion before a test is run. The same workaround has been applied. Please try this build to see if this issue occurs again:

http://www.passmark.com/ftp/memtest8....1.1b.1013.zip

[stevensxl]

The same issue happened.
https://drive.google.com/file/d/0Bxm_1t0QPEJIanZONFlDRFFaMWc/edit?usp=sharing

[keith]

It seems to hang while trying to enable the cache on the cores, so we made another slight adjustment. If it still hangs, we may have to think of something else.

http://www.passmark.com/ftp/memtest8....1.1b.1014.zip

[stevensxl]

It works! Completed all tests without any errors or hangs. But I checked the log file, there are some invalid parameters warning, not sure whether these are errors, please check. Thanks for your great support.
log: https://drive.google.com/file/d/0Bxm_1t0QPEJINXlJcWZCR0FFVzg/edit?usp=sharing

[keith]

Good to hear, thanks for the quick turnaround.

The warning messages you see in the log indicate that MemTest86 encountered issues with the multiprocessor subsystem in the UEFI firmware, and that the workaround has been applied correspondingly. There are still issues with the firmware that will require an update from the manufacturer, but this build will allow the memory tests to continue without hanging.

[stevensxl]

Do you mean the UEFI BIOS need to be updated? So with the workaround you provided, looks like it ignored some compatibility issues with the UEFI BIOS, then whether it really completed the memory test? If I pass the memory test, does that mean everything is ok for memory part design?

[David (PassMark)]

It looks like there is a bug in your UEFI BIOS. No other machine we had seen had this problem. The bug prevented the CPU cores notifying UEFI when a thread had finished. So it looked like threads were still running..... forever.

We put in a work around to force the threads to finish and the memory tests can now execute correctly. But the your UEFI implementation is still buggy in our opinion and is the real cause of the problem (and might cause other problems with other applications).

[stevensxl]

We will start to work on BIOS side try to fix the thread issue. Would you please tell me what registers are you probing to get the results?
By the way, the original issue was hang on test 2, then we got the hang on test 3 issue after Keith's debug. So are these two issues caused by the same threads error in BIOS? Or hang on test 2 issue was something else and it's fixed?

[keith]

We call the UEFI BIOS firmware's StartupThisAP() API to run our memory test on each CPU thread.

StartupThisAP (  IN  EFI_MP_SERVICES_PROTOCOL  *This,
  IN  EFI_AP_PROCEDURE          Procedure,
  IN  UINTN                     ProcessorNumber,
  IN  EFI_EVENT                 WaitEvent               OPTIONAL,
  IN  UINTN                     TimeoutInMicroseconds,
  IN  VOID                      *ProcedureArgument      OPTIONAL,
  OUT BOOLEAN                   *Finished               OPTIONAL
  );

We specify a 'WaitEvent' parameter in order to run in non-blocking mode, and wait on this event for the thread to finish execution. In your case, however, the firmware did not properly signal the event, causing MemTest86 to wait indefinitely or "hang". Through debugging we found that the threads did indeed finish execution.

We call this API function in separate instances, which is why you saw the hang on test 2 and test 3

[stevensxl]

Hi Keith,
We have BIOS team is working on the BIOS now. Currently they want to know how do you call function StartupAllAps() in Memtest. Would you please provide this part of code as you did for StartupThisAp() before? Thanks a lot

[keith]

Hi Keith,
We have BIOS team is working on the BIOS now. Currently they want to know how do you call function StartupAllAps() in Memtest. Would you please provide this part of code as you did for StartupThisAp() before? Thanks a lot

Yes, we had been calling StartupAllAPs() though we have since removed it and switched over to StartupThisAp() for future builds.

typedef
EFI_STATUS
StartupAllAPs(
  IN  EFI_MP_SERVICES_PROTOCOL  *This,
  IN  EFI_AP_PROCEDURE          Procedure,
  IN  BOOLEAN                   SingleThread,
  IN  EFI_EVENT                 WaitEvent               OPTIONAL,
  IN  UINTN                     TimeoutInMicroSeconds,
  IN  VOID                      *ProcedureArgument      OPTIONAL,
  OUT UINTN                     **FailedCpuList         OPTIONAL
  );

Like StartupThisAp(), we specify a 'WaitEvent' parameter in order to run in non-blocking mode, and wait on this event for all threads to finish execution.

[stevensxl]

Thanks, Our BIOS team is able to reproduce the issue and they reviewed their code with regards to the StartupAllAps()/StartupThisAp() implementation. As per their observation, when a user tries to execute async operation by passing down Event parameter, the event is trigged only when the timeout happens. Can you tell me which value is passed down?

Thanks.