This week I successfully set up the bluetooth communication between the ESP32 and our python app, and incorporated the communication scheme into our app. In order for the bluetooth communication setup to start the same time we start the app, I use the python library qasync to combine two tasks into the same event loops. The first task is the bluetooth listening for creating a connection with our esp32, the second task is the PyQt app that lucy built that updates the GUI and reads hand gestures from the camera. After the bluetooth connection is established, the user can now play sounds using IMU motion detection, and do chord selection with the left hand with the computer camera.
For incorporating the IMU data, I’m using the notification scheme with Bluetooth Low Energy (BLE). This means one message is sent every time an event is signaled, and this prompts a customizable handler to handle the notification. This is much simpler than a reader/writer scheme where we stream data. To force the handler to be promptly run in the event loop, I incorporate this notification as a signal inside the PyQt app. This allows the handler to be run inside the event loop first despite other tasks being scheduled already.