I found there was a way to release the GIL inside Python ‘C’ libraries:
Py_BEGIN_ALLOW_THREADS // do something synchronous here like epoll_wait(); Py_END_ALLOW_THREADS
My variant of (and the mainline) GPIO library does this when it blocks – specifically when it’s waiting for the hardware interrupt pulse with GPIO.edge_detect_wait() (mine) and GPIO.wait_for_edge() (standard version). I wondered whether actually releasing the GIL at this point was bad as once released the Python interpreter then processes another 100 bytes of bitcode before releasing it again; I postulated this could possibly cause missed sensor reads.
So I updated my library code, and ran the thread+signal variant of the code – once more 800Hz, just like the serialized version. Oh well, it was worth a try.
For now then I’ve definitely run out of ideas about threading and have moved onto vibration damping; the point of 1kHz sensor reads is integrating out noise / vibration – if the vibration is physically damped, then the sensor read frequency becomes a lot less important. . More on that tomorrow, probably.