This week, I worked on completing the networking code for keystroke detection as well as completing a UI for users to attribute gestures to specific keystrokes. I came up with the idea of using spotlight search on the Mac to quickly open common websites for viewing movies (ex. Netflix, Hulu, YouTube), and I played around with polling and delays to get a really quick connection between the sensor movement and the keystrokes. By Rosina’s suggestion, I also added a longer delay when a gesture was registered compared to when no gesture was registered to avoid the system registering the same opcode multiple times on a single press.
One bug I had to fix dealt with the pyautogui hotkey function, which is supposed to handle keyboard shortcuts (like pressing ctrl+z). I noticed that it sometimes didn’t press the keys at the same exact time and the shortcuts were not always carried out. To fix this, I wrote my own version using other pyautogui functions that ensured that all keys stayed pressed at the same time. This likely is less optimized than the hotkey function, but the latency difference was undetectable, and it worked with good reliability. I also worked on a protocol for sending IMU data. If the fail-safe is triggered, while it stays triggered, I’ll continuously transmit IMU data and have the mouse move in the IMU data callback function. This failsafe condition will be checked on the ESP before any of the other sensors are checked to ensure that other sensors are ignored when the failsafe is pressed.
In terms of the Gantt chart, I am on track based on my tasks. I should be ready to integrate with Rosina’s part and hopefully we can have a finished product ready to user test by the end of Thanksgiving break.