I2c timeout sda stuck error

Our team has working on developing a i2c driver (based on the i2c driver example created by Dryw Wade a number of years ago) that interfaces with the adafruit neodriver board. We’ve had reasonable success with it except that the control hub and the neodriver board seem to lose sync after a period of time ranging from 30 seconds to 5 minutes. It’s a little bit random but mostly related to sizes of i2c write messages and the frequency in which they are being sent across the sda line. log file reports an error of i2c_timeout_sda_stuck. Once it happens all communication to the device is lost until a hard reboot of the control hub occurs. Here’s a screen shot of the log, if thats at all useful. Looking for suggestions as to what might be going on, how to further troubleshoot it and/or recover i2c communication afterwards.

An image of the filtered log file from the REV log file viewer is unfortunately of little use. Can you please provide the log file in a Google Drive folder and provide a link to it?

Thanks!
-Danny

An SDA stuck error means that the (open collector) data line of the I2C bus is “stuck down” to ground by some other device on the bus. This can happen for various reasons, such as EMI causing a slave device to miss clock pulses and get wedged in a bad state where it will not let go of the data line. It could also be excess capacitance on the bus maybe. Hard to say without putting a scope on it. FWIW, when I was on a team I used an I2C to SPI bridge to push data to a Dotstar strip and it worked pretty well.

I suspect you might be able to recover the bus by unplugging and replugging the slave devices on said bus to get them out of the bad state without power cycling the entire hub.

If you can reliably reproduce it in 30 seconds to 5 minutes, you at least have a decent chance of catching what happened on a scope. I wish I could say the same for the UNKNOWN CAUSE error I ran into in Skystone

1 Like

Here’s the link to the log file. Thanks.

There shouldn’t be any other devices on the i2c bus. At least no external devices. For simplicity of testing, the only device connected to any of the i2c ports is the AdaFruit neoDriver.

I’ll be able to look the log over later today, but you’re not plugged into I2C Port 0 are you? There are more devices that don’t play nicely with other devices on I2C than I can count. The documentation usually warns of wedged I2C busses when issues arise. Remember that the IMU is on Port 0 too.

-Danny

I had to double check but the NeoDriver is connected to I2C Port 1.

Yup, that’s what the logs are telling me. But there’s also something I’m not quite sure I understand, why are there two drivers associated with the same device?

RobotCore: Adafruit WS28ll Driver                             led_strip                      USB (embedded); module 173; bus 1; addr7=0x60
RobotCore: Adafruit NeoDriver I2C to NeoPixel Driver Seesaw   led_strip                      USB (embedded); module 173; bus 1; addr7=0x60

Generally having more than one driver trying to access the same device is not really a recipe for a good time. You might be in some kind of deadlock situation where one driver grabs a resource that the other one wants, and the other driver grabs a resource that the first one will grab next, and they’re both stuck waiting for resources that will never be available - and thus the bus will appear wedged.

Can you provide any further insight into this?

-Danny

We’ve had two versions of the driver we developed as we have iterated. They are separate files and only one should be being configured and instantiated in the active opmode. I’ll investigate some more, but I thought that would have been from different executes of different opmode executions. We can definitely remove the other version altogether to avoid any potential confusion and see if that makes a difference.

There’s nothing else in the logs that point to anything relating to your device or driver. I don’t have a neopixel device, so I cannot help debug your driver. Sorry!

-Danny