Author: exg500

Accomplishments

This week, I worked with Caitlyn and Nick on user tests. We conducted multiple user tests and had them fill out feedback surveys on intuitiveness of the Kar and the haptic feedback. While we did some of these user tests, we noticed multiple bugs. For example: the Kar would keep running after Bluetooth disconnected, the BLE supervision timeout would be too high (and therefore the Kar would keep running for a while after it disconnected from the Kontroller), and the Kontroller wouldn’t reconnect/try to scan to connect to the Kar after a reset was triggered on either device. We addressed all of these issues, and we will continue polishing our system until demo day.

Finally, we conducted some BLE roundtrip and end-to-end latency tests. We met our end-to-end latency target of <50 ms. Here is a graph further showing our round-trip latency. Of course, there is some variance with latency as distance between the BLE devices increases, but the connection and latency of packets is reliable for a distance within 10 m.

Progress / Schedule

We have reached our MVP. We are now adding finishing touches, and polishing as much as possible before our demo day.

Next Steps

• Perform more user testing and collect more feedback.
• Fine-tune Kar reverse movement and program switch for reverse motion.

Accomplishments

This week, I completed some preliminary code that detects crashes on the Kar. Essentially, a crash is detected when the derivative of the IMU acceleration values reaches a threshold. This can be adjusted. Additionally, Caitlyn and I have completed haptic motor driver code for the Kontroller. We worked on mapping the range of motion for speed (0º to 90º) to an 8-bit value that varies the oscillations of the haptic motor. This was achieved via linear interpolation.

Progress / Schedule

We’re on track, and making progress with our Kontroller. Soon enough, we’ll have the PCB fully set and we’ll move away from the breadboard implementation.

Next Steps

Next week, I’ll be doing some testing on the haptic motor system, to ensure that it responds properly in edge case scenarios. Additionally, I will work on optimizing the BLE code, and take measurements throughout, to be ready to discuss tradeoffs.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

Becoming familiar with the STM32CubeIDE was a new tool that took some time to getting used to, but it paid off in the end. It’s easy to want to create your own development environment because some of the IDE features are questionable and unorthodox for an IDE, but the autogenerated code for the HAL (Hardware Abstraction Layer) is very robust and easy to use for user programs.

Additionally, it takes some time to learn how to properly set up the debugger for the IDE. For example, you need to specify the debugging interface (JTAG/SWD), modify some syscalls (like sys_write and sys_read) so you get UART redirection for print debugging, ensure you carefully set compiler flags so you have the necessary debug symbols, etc. But once everything is fully set up, debugging becomes very streamlined.

Finally, version control support on the STM32CubeIDE isn’t the best either. STM32 projects could become quite bloated (especially if you have multiple peripherals and wired/wireless communication protocols enabled). Therefore it becomes hard to keep track of what files to push to a shared repo. Additionally, the STM32CubeIDE is picky with the IO configuration files, and it has prevented us from easily continuing someone’s work on another machine. Thankfully, we’ve learned over time exactly which files to commit, and how to commit IO configuration files so that we don’t get error messages like “Error: this project is already open” when opening up projects.

Accomplishments

This week I primarily worked on fine tuning the speed control on the Kontroller and getting the Kar IMU (BNO005) to work for haptic motor data on the glove. Caitlyn and I completed a logarithmic mapping between the finger angle and PWM value. Fine tuning the parameters for this logarithmic relationship helped us eliminate the range of the speed “deadzone” (where the Kar is in the resting/brake position) and increase the range corresponding to PWM values that will result in significant acceleration of the Kar. Additionally, we were able to configure a new IMU for the Kar that communicates with the STM32 via I2C. This will be crucial for crash detection and sending haptic feedback to the Kontroller. Currently we are simply redirecting the gyro/acceleration values to UART. However, the next step would be to send a haptic vibration flag over BLE when high jerk (high change in acceleration over time) and gyro spikes are detected (which could likely indicate a crash).

Progress / Schedule

We’re on track for the most part. I was hoping to test the total latency of sending Kar motion commands over BLE this week but I was not able to. Early this week, I will be doing that using my camera’s slow motion feature and blinking an LED.

Next Steps / Deliverables

Besides measuring latency of commands, I will work with Caitlyn to wrap up the haptic feedback integration. I will also continue fine tuning angle to PWM conversion by experimenting with different parameters, and hopefully conduct some user tests to gauge the opinion of others’ on the user inferface.

Accomplishments

This week, I worked on flashing and testing code on the Kontroller PCB to control the Kar with BLE. Additionally, I’ve been doing some fine tuning with the IMU code that Caitlyn wrote, and smoothing out transitions to/from dead zones. This is important to refine, given that we don’t expect full 360º rotations, but rather 90º-120º rotations for both the ICM and MPU located on different parts of the wearable device. Other than that, we were able to finally test driving the Kar on the ground, and evade obstacles by steering and varying acceleration.

Progress / Schedule

We are on track, and we have a working Kar PCB to demo for our interim demo. There’s also progress with the haptic motor, and we’ve been experimenting with different vibration modes. My biggest concern now is getting the backwards PWM motion working for the Kar. Because the servo steering provides a limited range of motion, backwards movement on the Kar would be necessary to have the user complete tracks or get past obstacles without external help. The Kar’s WP1625 ESC uses a FBR (Forwards/Brake/Reverse) mode, where 1000 µs – 1450 µs corresponds to backwards, roughly 1450 µs – 1550 µs corresponds to braking, and 1550 µs – 2000 µs corresponds to forwards, assuming a 50 Hz PWM signal. I’ll measure the PWM signal for backwards using an oscilloscope to make sure I’m generating the correct signal, and follow the troubleshooting steps for the WP1625 datasheet if I encounter further issues.

Next Steps / Deliverables

Next week, I will work on adding backwards motion to the Kar, and continue fine-tuning the roll, yaw, and pitch conversion to angles. If I have extra time, I’ll help with the haptic motor integration.

Accomplishments

This week, I was able to implement and test the majority of the car movements for our final demo (forwards with varying increasing/decreasing speeds via keyboard inputs, and full 180º motion on the IMU for steering). Most of my time was spent understanding PWM signals for standard RC ESCs. Since our STM32 produces a 32 MHz HSE clock signal, I used a prescaler value of 32 and a period of 20,000 to obtain a 50 Hz (20 ms period) PWM rate. Additionally, standard RC ESCs operate with a 1-2 ms HIGH every 20 ms, so we start with 1.5 ms for both the motor and servo control signals to start the car at a neutral position (this is also what our reset state will change to). Additionally, I used Caitlyn’s IMU code and linearly interpolated her IMU roll data between 1-2 ms to create a smooth mapping for the servo steering.

Progress / Schedule

I’m on track with the schedule. I’ve been having some difficulty getting the wheels to spin backwards. I believe there’s a sequence of PWM signals that need to be sent to get the Kar moving backwards. However, the WP1625 brushed motor user guide is quite short and I can’t seem to find much information online about the sequence. I will continue working on that this week, and work with Caitlyn to map other IMU motions between the Kontroller and Kar in parallel. Other than that, I’m feeling confident about our upcoming demo, given that we’re already seeing smooth Kar motion in action.

Next Steps / Deliverables

• Get backwards motions working for the Kar’s wheels.
• Fine-tune the steering for the Kar and integrate a PID control loop for speed stabilization.
• Implement haptic motor feedback for the Kontroller.

Accomplishments

This week, I worked on adding an LED characteristic for BLE on the STM32. This allowed me to verify that I could send data from the ESP32 to the STM32 besides simply establishing a GATT client/server connection. Additionally, I spent some time reading the Donkeycar’s documentation to begin writing code to move the brushless DC and servo motors.

Contrary to what I thought, the STM32 doesn’t directly send the PWM signals to the motors. The Donkeycar’s expansion board has a PCA9685 controller that sends servo signals to the motors via I2C. I then spent some time modifying the STM32 IOC file to provide the HAL for I2C with 7-bit slave addresses.

Progress / Schedule

Currently, I am on track with my tasks. I’ll ensure to have functional motor code this week, so that Caitlyn can integrate her IMU logic with the system once basic movement recognition is finished. In parallel, I will be trying to flash my code onto the board’s as Nick finishes verifying them.

Next Steps / Deliverables

• Finish writing I2C setup code to move Donkeycar’s wheels.
• Integrate simple IMU motions for forwards/backwards movement on the car.
• Flash STM32 and ESP32 code on PCB boards after verification is done.

Acomplishments

The past 2 weeks, I worked on and completed the P2P BLE setup between the STM32 and the ESP32 microcontrollers. In our BLE communication architecture, the STM32 is a GATT server, and the ESP32 is a GATT client. At a high level, as a GATT client, the STM32 advertises and exposes its services; the ESP32 scans and connects to the server and writes/reads data. Both devices initialize the BLE stack on their second CPU core, which support a variety of low power modes. This is ideal, given that our filtering/processing tasks are likely going to consume a significant amount of power.

As for our implementation/process, we ended up not using an RTOS on our STM32. The BLE setup code included on STM32CubeIDE uses a sequencer for BLE that conflicts with some of the tasks on FreeRTOS for example. While we found a workaround for it, we thought it would be more productive to simply not use an RTOS, instead of going through the hassle of fixing linker errors and carefully managing task priorities between sequencer and RTOS tasks. Additionally, there are relatively few subroutines that the STM32 has to execute compared to the ESP32. We were, however, able to get BLE working on its own FreeRTOS task on the ESP32. Fortunately, not much new code had to be written for both microcontrollers. Most time was spent properly configuring the IOC config file for the STM32 and editing a few things like UUID and advertising name. For the ESP32, I was able to reuse most of the GATT Client example code from Espressif. I simply had to modify it so that it would scan for the server with the proper device/characteristic UUIDs and the correct remote device name.

For verification, I used multiple debug logs and redirected them to UART for verification of the existence of the successful case logs. I was also able to set a supervision timeout of 2000 ms for the connection and got the correct disconnection status codes when I disconnected the ESP32 from power.

Progress / Schedule

I’m currently on track with my schedule. The next step would be to integrate Caitlyn’s work with the IMU sensor data, and to test the BLE code once Nick is able to verify the PCBs. During the following days, I will be using an oscilloscope to test BLE latency and taking measurements, so we have something to compare against once we start implementing optimizations.

Deliverables / Next Steps

• Take latency measurements for BLE and SPI/I2C.
• Flash code into PCBs once they’re verified.
• Write mock code to simulate sensor activity and communication.

Accomplishments

This week, I worked with Caitlyn for some of the FreeRTOS integration with the STM32WB55RG Nucleo board. However, I spent most of my time working setting up P2P BLE communication between the STM32 and ESP32.

We’re using the STM32CubeIDE for development on our STM32 board. This allows you to autogenerate configurations so that you don’t have to manually do things like writing a bootloader/RTOS, writing drivers for GPIO and communication protocols like UART, I2C, SPI, BLE, etc. However, there was an issue with getting both RTOS and P2P BLE communication working on the STM32. This is because the P2P code uses a sequencer that conflicts with the RTOS’s scheduler, and therefore the IDE cannot autogenerate code to support both. I saw that at least 2 ST employees have acknowledged this limitation in these 2 ST community forum posts: see post 1 and post 2.

Regardless, with the help of some of the documentation from one of the ST employees, I was able to replace the P2P sequencer code with similar RTOS kernel function calls. So far, the code at least builds and compiles fine. I have documented some of my learnings along the way, as well as the communication architecture for BLE between the ESP and STM in this Google doc.

Progress / Schedule

Currently, I would be a bit behind schedule. The FreeRTOS and P2P setup on both devices has been a challenge, as well as getting accustomed to the new development environment for the microcontrollers. However, I have learned how to debug on both devices via UART and GDB (at least the STM32IDE has a pretty decent GUI for GDB+print debugging), and I have also learned more about the BLE protocol.

In parallel, Caitlyn and I will be testing out some of the IMU sensors and haptic motors.

Deliverables / Next Steps

Next week, I will:

• Work on sending a stream of bytes from one MCU to another and validating with UART with minicom on my machine.
• Test BLE latency between the 2 devices using an oscilloscope via a sequence of GPIO set commands.
• Help with the setup and integration of sensors/motors.

Accomplished

So far, I’ve been working on setting up a development environment for the XIAO ESP32S3 that works on both macOS and Windows. We have decided to move forward with the Espressif-IDF (Espressif-IoT Development Framework), given that Espressif Systems is the company that makes the ESP32 and there is extensive amounts of support online through documentation and forums.

There is a Docker image that is built and published by Espressif on their official Docker Hub account. This makes building ESP32 code seamless across different operating systems. I wrote the Dev Container configuration file for the ESP32 which uses the Espressif image. However, I personally develop on macOS, and have found that the flashing process specifically is easier to do locally, given that Docker spins up a Linux VM and does not run on the macOS kernel. Windows can bypass this with WSL and USB passthrough, on the other hand.

We accomplished writing a simple program that flashes an onboard LED via simple GPIO set calls and delays. The program also launches an additional thread that prints to a serial monitor via UART. This basic multithreading program was possible due to FreeRTOS running on the board after being flashed (it is the default with Espressif-IDF).

Progress / Schedule

According to the gant chart, I completed my part of the “FreeRTOS Setup for ESP32/STM32”. However, “Bluetooth Communication Implementation” is still work-in-progress, and I could not fully finish it this week, so it will carry on to next week.

Most of my time spent went into setting up development environments for microcontrollers, and figuring out the correct software dependencies for our project. This video in particular was pretty helpful in getting set up for macOS. Some code, along with our PCB schematics/layout are in this GitHub repository. Overall, I’m mostly on track. I don’t have an official task for next week, which gives me some slack for Bluetooth communication. I’ll also help Caitlyn with STM32 code.

Deliverables / Next Steps

Next week, I will:

• Finish pushing some of the ESP32 setup code to our GitHub repo.
• Start implementing with wireless communication with 2 microcontrollers.