Category: Caitlyn’s Status Reports

Accomplished

Over the past two weeks, I worked with Enrique and Nick to fine-tune the driving and steering behaviors of our Kar. The first thing we worked on fine-tuning was the haptic feedback, where the haptic intensity is determined by the drive throttle. If the driver is accelerating at full throttle, they will feel considerable haptic feedback on the hand compared to if they only slightly accelerating.

We also worked on developing and fine-tuning user control, experimenting with driving with and without backwards, and the different hand motions/gestures that users could test with for feedback. I created our user feedback form and we have started receiving responses regarding the intuitiveness, haptic feedback, and overall performance of our system.

Progress / Schedule

Our team has finished MVP and are now focusing on fine-tuning and improving current controls based on the user feedback.

Deliverables / Next Steps

Before demo day on Monday:

• perform more user feedback tests.
• fine-tune Kar steering and acceleration control and haptic feedback.
• integrate user feedback into Kar and Kontroller.

Accomplished

This week, I started integrating the haptic motor controller on the glove controller. Since both the haptic controller and the MPU6050 communicate over I2C, I had to ensure that both worked individually after porting over the code and checking that they worked together as well. I worked on translating IMU speed to haptic buzz intensity while Enrique worked on crash detection on the Kar.

Progress / Schedule

I am on schedule to completing the MVP. Enrique and I have begun working together to integrate haptic feedback and the Kar IMU.

Deliverables / Next Steps

Next week, I will:

• implement a mapping between the Kar IMU’s gyroscope values and haptic feedback.
• fine-tune Kar control.
• help bring up firmware on the glove PCB.

New tools and new knowledge

Accomplished

This week, I worked with Enrique to implement logarithmic scaling for IMU control on the glove. Previously, we had linear scaling which gave us too much precision at lower angles, which is where the motor deadzone was. This means that it was harder to start the motor at lower speeds. To resolve this issue, we applied a logarithmic function so that movement is more “snappy” at low inputs and gradual at high inputs.

We also worked together to enable the BNO055 IMU on the STM32 on the Kar over I2C. This required enabling I2C on the STM32 and writing up a driver to read the IMU inputs to be converted into roll.

Progress / Schedule

I am on schedule to completing the MVP. Enrique and I have begun working together to integrate haptic feedback and the Kar IMU.

Deliverables / Next Steps

Next week, I will:

• come up with and implement a good mapping between the Kar IMU’s gyroscope values and haptic feedback.
• fine-tune Kar control.

Accomplished

This week, I worked on prototyping the MPU6050 to ensure that we can still get yaw, pitch, and roll values for it over I2C. After prototyping this, Enrique integrated the MPU6050 with the current Glove Kontroller that already has the ESP32 communicating over SPI with the ICM20498.

I also worked on generating different types of feedback on the haptic motor controller over I2C. I experimented with different waveforms that were already implemented on the controller, and manually tried to generate my own waveforms. Some important waveforms we may look into for the mapping between Kar motion and haptic feedback include ramp up/down and strong buzz feedback.

Progress / Schedule

I am on track as I have finished prototyping my subsystems (IMU + haptic motor controller), and will be working with Enrique to integrate the haptic feedback and fine-tune Kar control.

Deliverables / Next Steps

Next week, I will:

• implement the MPU6050 on the Kar and come up with a good mapping between the Kar IMU’s gyroscope values and haptic feedback.
• fine-tune Kar control.

Accomplished

This week, I implemented the firmware for reading data measurements from the ICM20948 IMU sensor over SPI. At first, I measured only the gyroscope values and not the accelerometer and magnetometer. However, after integrating the angular velocity over time to find absolute heading, we noticed significant drift in the gyroscope. After some research, I decided to implement the Madgwick filter for a better estimate of yaw, pitch, and roll, since it does better against drift by combining data from the gyroscope, accelerometer, and magnetometer. However, after some testing, I realized it would be best to leave out the magnetometer, and base the estimate on just gyroscope and accelerometer data instead.

After getting a more consistent reading of yaw, pitch, and roll, Enrique and I worked together to map roll to servo angle PWM, and integrated the IMU firmware with his bluetooth code to send servo PWM high-times to the STM32. The result was being able to control the servo on the Kar by the rolling angle of the IMU.

Progress / Schedule

I am back on track as Enrique and I have started working together to integrate our individual subsystems on the Kar. Our plan is to finish prototyping the MVP so we can migrate the firmware to the final hardware PCBs and begin testing.

Deliverables / Next Steps

Next week, I will:

• currently we plan on doing very simple linear mapping between IMU roll and pitch and Kar steering and drive, however we will need to look into a more optimal and realistic mapping.
• continue communicating with the haptic controller over I2C and generating different types of feedback.

Accomplished

This week I received the actual IMU (ICM20948) that we will use on the glove kontroller. I have started looking into existing ESP-IDF drivers for this IMU and am currently experimenting with setting up the IMU with SPI.

Progress / Schedule

I am a bit behind schedule, but will be back on track next week.

Deliverables / Next Steps

Next week, I will:

• finish setting up the new IMU with SPI and Kalman filtering.
• figure out optimal mapping between IMU and kar steering/rotation.
• begin twiddling with the haptic controller and experiment with generating different types of feedback.

Accomplished

This week, I worked on setting up the initial framework for development on the ESP32. This included updating the README on GitHub to document how to connect to the ESP32 for development and setting up the file directory structure. We will modularize our code by creating files for each task or subsystem on the ESP32, with the main.c handling creation and scheduling of all tasks.

Another task I worked on this week was getting raw IMU data from the MPU6050 (which we are currently using as a substitute for the actual IMU we will use). Once we order the actual IMU, I can ideally quickly update the code to process the new IMU’s data since most of the interface will be the same. I also implemented the FIR filter which simply outputs filtered data of the most recent raw gyro data.

Progress / Schedule

I am still a bit behind schedule, but now that I have implemented one filtering method (FIR), I think implementing Kalman Filtering will be a bit more straightforward. I have also started thinking about how to translate the gyro z-axis into the kar’s steering/rotation, which will have to be done through integration of the z-axis rotational acceleration.  I have updated the Gantt chart, and believe that I can still catch up with my goal of finishing up the IMU subsystem by next week.

Deliverables / Next Steps

Next week, I will:

• implement Kalman filtering for IMU.
• figure out optimal mapping between IMU and kar steering/rotation.
• begin twiddling with the haptic controller and experiment with generating different types of feedback.

Accomplished

This week, I worked with Enrique to set up FreeRTOS on the NUCLEO-WB55RG board that we will be primarily using for kar prototyping. We followed a tutorial that I linked two weeks ago, however instead of setting up FreeRTOS in VSCode which I worked on last week, we decided to set it up in STM32CubeIDE. After setting up LEDs, redirecting printf to Serial Wire View (SVW), and FreeRTOS, we were successfully able to create two tasks where one toggles an LED and the other prints to the serial monitor. We decided to stick with using the ST IDE for setting up the environment because there are more resources online using this setup, and we will probably stick with using the ST IDE for the remainder of the STM32 development.

I also worked on a more complete architecture diagram of our system that was used in the design presentation. For this diagram, I communicated with Nick on where the hardware is and communicated with Enrique on what the communication protocols look like between components.

Progress / Schedule

I am a bit behind schedule, since I was supposed to finish FreeRTOS setup for the STM32 last week and finish up researching IMU algorithms this week. Since I have just received the ESP32-S3 microcontroller and found an additional MPU6050 that I can use, I will focus on getting the IMU reliable, accurate, and consistent. I have updated the Gantt chart, and believe that I can still catch up with my goal of finishing up the IMU subsystem by next week.

Deliverables / Next Steps

Next week, I will:

• learn how to use the ESP32-S3 for IMU.
• research the FIR and Kalman filters for the IMU.
• collect measurements from the IMU and process them through filters.
• set up FreeRTOS on my own Nucleo STM32 (not the NUCLEO-WB55RG) on STM32CubeIDE since it looks like we will be doing all development on there.

Accomplished

This week, I followed multiple tutorials to set up FreeRTOS and the beginning of what our team’s workflow would look like on the STM32.

Using an STM32 Nucleo board that our team had on hand, the STM32F401xE, I used the STM32CubeMX software to enable FreeRTOS on the board. This software has a user interface for easily using methods in the FreeRTOS library, which I experimented with to create two simple tasks. I was able to change the priority of tasks and figure out other settings like configuring the heap size. Since STM32CubeMX is more for configuration and code generation, I couldn’t figure out how to compile and deploy the code without installing STM32CubeIDE, so I moved onto setting up the STM32 development environment on VSCode.

Recently STM32 developed an extension on VSCode called “STM32Cube for Visual Studio Code” which integrates build systems like CMake so users can compile STM32 projects within VSCode. With this extension, I was able to compile and deploy the STM32CubeMX-generated FreeRTOS firmware onto my STM32 Nucleo board.

Progress / Schedule

I will need to spend a bit more time making the STM32 development workflow more seamless and integrated, since it took me some time to install and debug drivers for the STM32 Nucleo board. This way it will be easier for the team to work on different parts of the STM32 software concurrently.

Otherwise, I am mostly on schedule but will have to prepare and do some research for collecting IMU measurements and experimenting with different filters, namely the FIR and Kalman filters.

Deliverables / Next Steps

Next week, I will:

• push the STM32 development environment up to the team GitHub and work with Enrique to set up a more seamless workflow for working on the STM32.
• outline and plan what types of tasks we will need on the TX and RX MCUs. This will include identifying what initialization and run will look like for each task, as well as task priority.
• research the FIR and Kalman filters for the IMU.

Accomplished

The high-level architecture of our project will involve the ESP32-S3 acting mainly as the data source and an STM32W series board acting as the data receiver. Both of these microcontrollers will need some sort of RTOS running to manage complexity and ensure the system is responsive and reliable

This week, I looked into how we would set up FreeRTOS on each microcontroller. We want to use FreeRTOS since it is a small and open-source real-time operating system that can simplify task management and scheduling for both of our MCUs.

I found that the development framework for ESP32-S3 is already built upon FreeRTOS, so it is possible to just use features like tasks, queues, and semaphores in our code development. FreeRTOS also already supports use on certain development boards such as STM32s, so I watched a video on how we would begin writing tasks and handle scheduling on STM32 after some setup on STM32CubeIDE, which is an IDE for STM boards.
Here is the link for the tutorial for next steps.

Progress / Schedule

My progress is on schedule, as we wanted to spend this week investigating FreeRTOS compatibility and setup on the microcontrollers. This will end up extending into next week of actually setting up our development environments and FreeRTOS on the hardware.

Deliverables / Next Steps

Next week, I will:

• follow the tutorial to setup FreeRTOS for an STM32 board that we have readily available such as the STM32F401xE.
• outline and plan what types of tasks we will need on the TX and RX MCUs. This will include identifying what initialization and run will look like for each task, as well as task priority.
• set up a Git repository for the team so that we can begin collaborative development.