Problem with USB loopback test in machine

[ChrisA]

We have a machine at a customer where we have some issues with spontaneous USB plugs and unplugs and we do not know why because we do not see this in other machines (we found this 2x until now). We use a common express PC module with an Intel Celeron N3060 CPU which has 5 USB 2.0 ports. However the module has 8 USB2.0 ports so the PC module uses a USB2514U hub (microchip) to extend the 5 CPU ports to 8 external ports. Until now we see these unplugs and plugs specific at the ports on the hub while we do not unplug or plug anything at the USB ports because everything is always connected with the different USB ports like touch panel and barcode reader. We already replaced USB cables but the cables were OK. We ordered a few PM065 analyzers to test the ports at the customer and the analyzer shows an error during the unplug moment. We repeated the test with a new PC module with an Intel ATOM X5 CPU that has 8 USB ports at the CPU so that module has no hub. With the new module we do not see this issue (so the cables are OK because the speed test was running at max 480Mb/sec) but we have built several machines with the present PC module so we have to know what could be the cause of this?
Below the logging of the analyzer we got from our engineer this morning who visited this customer and tested with several PM065 analyzers (we have 3 external devices connected with cables so we connected the analyzers at the end of the cable after disconnecting the touch panels and barcode reader. One of the analyzers was connected with a port from the hub (USB2514 hub at PC module). At the unplug moment the following text is displayed at the analyzer. I can not find the meaning of these error codes so maybe someone knows what the meaning of these errors could be or maybe someone an idea of the root cause?

PM065 Plug loopback test with its extension cable on X28-USB#5 (USB port at USB2514 hub on PC Module)
A few lines with normal loopback data
Wed Jul 6 14:04:22 2022: Loopback - Loop 348051 - 348060, Avg Speed 258.0 Mb/s (32.3 MB/s)
Wed Jul 6 14:04:22 2022: Loopback - Loop 348061 - 348070, Avg Speed 264.0 Mb/s (33.0 MB/s)
Wed Jul 6 14:04:22 2022: Loopback - Loop 348071 - 348080, Avg Speed 263.3 Mb/s (32.9 MB/s)

Wed Jul 6 14:04:24 2022: ERROR - Write failed : 995
Wed Jul 6 14:04:24 2022: ERROR - The I/O operation has been aborted because of either a thread exit or an application request.
Wed Jul 6 14:04:24 2022: ERROR - Overlapped I/O operation is in progress.
Wed Jul 6 14:04:24 2022: ERROR - USB device driver error reported: 0xc0000120
Wed Jul 6 14:04:24 2022: ERROR - The I/O request was canceled.
Wed Jul 6 14:04:24 2022: ERROR - USB request status reported: 0xc0010000
Wed Jul 6 14:04:24 2022: ERROR - Endpt UsbdStatus: [state=STALLED status=UNKNOWN]

Then the next error, which repeats many times until the test is stopped by the analyzer.
Wed Jul 6 14:04:24 2022: Loopback - Loop 348073 - 348082, Failed
Wed Jul 6 14:04:24 2022: ERROR - Write failed : 170
Wed Jul 6 14:04:24 2022: ERROR - The requested resource is in use.
Wed Jul 6 14:04:24 2022: ERROR - Overlapped I/O operation is in progress.


And the results lines:
Wed Jul 6 14:04:24 2022: ERROR - Test stopped due to device removal
Wed Jul 6 14:04:24 2022: ERROR - RESULTS:
Wed Jul 6 14:04:24 2022: ERROR - Test Start time: Wed Jul 6 13:36:25 2022
Wed Jul 6 14:04:24 2022: ERROR - Duration: 000h 27m 59s
Wed Jul 6 14:04:24 2022: ERROR - Maximum Data Rate: 278.4 Mb/s (34.8 MB/s)
Wed Jul 6 14:04:24 2022: ERROR - Minimum Data Rate: 15.1 Mb/s (1.9 MB/s)
Wed Jul 6 14:04:24 2022: ERROR - Average Rate: 253.6 Mb/s (31.7 MB/s)
Wed Jul 6 14:04:24 2022: ERROR - 348097 operations.
Wed Jul 6 14:04:24 2022: ERROR - 16 error(s).
Wed Jul 6 14:04:24 2022: ERROR - 16 write error(s). 0 read error(s). 0 verification error(s).
Wed Jul 6 14:04:24 2022: ERROR - 0 physical layer error(s).
Wed Jul 6 14:04:24 2022: ERROR - 0 link layer error(s).
Wed Jul 6 14:04:24 2022: Loopback - END OF USB LOOPBACK TEST
Wed Jul 6 14:04:27 2022: Information- PMU356VGNO (HighSpeed 480Mb/s) added

And below the second analyzer we connected to another USB2.0 port that is not connected with the hub on the PC module
(NOTE: this analyzer shows no errors but shows a message that the other analyzer was plugged again after the error)

PM065 plug loopback test with its extension cable on X27-USB#2 (USB port not at hub but directly from CPU)

A few lines with normal loopback data Wed Jul 6 14:04:25 2022: Loopback - Loop 435651 - 435660, Avg Speed 264.8 Mb/s (33.1 MB/s) Wed Jul 6 14:04:25 2022: Loopback - Loop 435661 - 435670, Avg Speed 256.5 Mb/s (32.1 MB/s) Wed Jul 6 14:04:25 2022: Loopback - Loop 435671 - 435680, Avg Speed 221.3 Mb/s (27.7 MB/s) Also this analyzer reports that the other analyzer is plugged. Wed Jul 6 14:04:27 2022: Information- PMU356VGNO (HighSpeed 480Mb/s) added to list the logging continues on this analyzer. Wed Jul 6 14:04:27 2022: Loopback - Loop 435681 - 435690, Avg Speed 257.1 Mb/s (32.1 MB/s) Wed Jul 6 14:04:27 2022: Loopback - Loop 435691 - 435700, Avg Speed 266.4 Mb/s (33.3 MB/s) Wed Jul 6 14:04:27 2022: Loopback - Loop 435701 - 435710, Avg Speed 275.9 Mb/s (34.5 MB/s)

[David (PassMark)]

If I understand the problem, you are seeing USB disconnections from a PCIe connected USB2 expansion hub (manufacturer unknown)? But you don't see the same issue when the same devices are connected to motherboard USB ports.

the module has 8 USB2.0 ports

The USB2514 chip only claims to support 4 downstream ports. So not sure how you got to have 8 on a single card.

Error 0xc0000120 from the device driver is just, "STATUS_CANCELLED"
See,
https://docs.microsoft.com/en-us/ope...c-49e60bebca55

So not very descriptive. But it is hard for software to provide a good description of malfunctioning hardware.

Seems likely the expansion card (or it's device driver) is bad. I would suggest going back to the manufacturer and asking them about it.

PM065 plug loopback test with its extension cable

If you are using extension cables or very long cables, this can be a source of electrical problems. Reflections, low signal levels, external interference, timing issues, etc..

You could also try using the suspect USB hub card in another regular desktop machine.

Small chance it could be a PSU issues as well.