This Two Week’s Work

• Worked on the UI code (see “State of the UI” for a description of what happened)
• Worked on final presentation slides
• Conducted some testing with my team

Schedule

Out of the two deliverables from last the last status update, I completed one (in green) and did not finish the other one (in red).

1. Complete implementation of turn signal control
2. Complete radar and UI integration

Upcoming Deliverables

The deliverables for this week are as follows:

1. Get magnetometer into working state for final demo
2. Make alerts on UI less noisy
3. Complete more tests

Special Questions

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks?

Here are some tools that I had to learn for this project:

• Figma – the website that I used to mockup the UI before I implemented it.
• ElectronJS and associated tooling (e.g., Vite) – ElectronJS is the framework that I used to build the UI in. I’ve used HTML, JavaScript, and CSS, but never the ElectronJS framework itself. Additionally I had to learn how to use the associated tooling, such as how to package the application and use the build system. Some additional libraries I learned to use include:
  • onoff – GPIO control in JavaScript
  • i2c-bus – I2C control in JavaScript

Some other specific knowledge I got included:

• Figuring out how to use the magnetometer over I2C and how to calibrate things

What learning strategies did you use to acquire this new knowledge?

In general for tools, I learned things very informally (e.g., without watching a tutorial or reading a textbook). For Figma, I essentially just messed around with it and looked up things as needed. Figma’s official documentation was useful along with videos.

For Electron, I used their documentation and Googled things when I got stuck. Reading Github issues and StackOverflow answers was also very helpful.

For learning how to use the magnetometer, reading the MMC5603 datasheet and looking at Adafruit’s C++ library implementation was really helpful. And of course, lots of trial and error. For calibration, reading blog posts and seeing sample code proved to be very useful

For Raspberry Pi-related issues, I read posts from peoples’ blogs, the Raspberry Pi forums, questions and answers on the StackExchange network, and reading the documentation.

State of the UI

Here’s a more detailed rundown of what the changes were to the UI since the last update

Integration with radar

The implementation work was done by the previous status report, but in these two weeks Jack got the front radar working and we verified that radar and UI integration is working for both front and rear radars.

Making the UI less “flickery”

When we were testing, we noticed that the system is very “flickery” in the sense that the blind spot indicators, range indicators, and alerts can change their values very rapidly. For example, the blind spot indicators would show up and disappear pretty quickly which is not a good experience for the user.

To fix this, I implemented a deactivation delay for the blind spot indicator where each blind spot indicator must stay activated for at least 2 seconds after the last detection of a vehicle in the side lane before it is allowed to deactivate.

For the range indicator, I ended up implementing something akin to a throttling system where updates to the range indicator are throttled to 1 Hz (although in testing it before writing this I think there might be a slight bug that occasionally allows faster updates to go through).

The collision warnings are still too sensitive right now. I’ll have to think of some way to make them less sensitive.

Adventures in magnetometer calibration

In the last status report, I talked about an issue where rotating the magnetometer 90 degrees physically only reports 60 degrees. I spent a significant amount of time the last two weeks trying to calibrate the magnetometer to see if I could improve this.

In my first attempt, I used some Python calibration code that was modified from code on this Adafruit page (plotting code taken from this Jupyter notebook from Adafruit) to determine the hard iron offsets. That gave me some calibration values, except when I applied the calibration values in code the magnetometer broke even more where rotating the magnetometer by a lot only changed the heading by a few degrees.

Then, I attempted to perform a more advanced calibration using a software called MotionCal (which calculates both hard and soft iron offsets). I spent a decent amount of time getting it to build on the RPi 4 and modifying the code to take input from my Python script that interfaces with the magnetometer instead of reading from a serial port as MotionCal originally wants. This StackOverflow answer had code that I used for bridging between my Python script that interfaced with the magnetometer and MotionCal. My original idea was to create a virtual serial port that MotionCal could use directly so I didn’t have to modify MotionCal, but it turns out this solution didn’t create the type of device MotionCal wanted so I ended up having to modify MotionCal too. In the end, I effectively ended up piping data from the Python script to MotionCal.

I also used this StackExchange answer that had sample code that was helpful in determining what to send from the Python script and in what format (e.g., multiplying by 10 for the raw values) for MotionCal to work, and it was also helpful for learning how to apply the soft iron offsets once I obtained them from MotionCal.

With all these changes, I still didn’t have any improvement – rotating the magnetometer a lot would not result in a significant change in the heading :(. So, as of this writing, the magnetometer is still not functional as a compass. Oddly enough, reverting my changes seemingly no longer allows the magnetometer to work as before all this (where rotating 90 degrees is needed to report 60 degrees). I’ll need to spend some time before the demo getting this fixed, but I may just use my workaround mentioned in the last status report instead of trying to get calibration to work.

Personally, I’m starting to think that something’s wrong with our magnetometer. The plots I get when calibrating are very different from what Adafruit gets (see animation in top left corner), where my code draws three circles very far apart instead of overlapping like they get.

It turns out I actually have an IMU (MPU6050) from a previous class that I took so I could’ve used that instead of the magnetometer, but I think at this point it may be too late to switch. I’ll have to look into how much effort it’ll take to get the IMU up and running instead, but from what I remember from trying to use the IMU to calculate heading in my previous class, I will also need to do calibration for it too which might be difficult.

 

This Week’s Work

• Implemented physical control of turn signal lights
• Implemented auto-cancellation using magnetometer
• Started integration with Jack’s radar code

For details, please see the “State of the UI” section below.

Schedule

Out of the two deliverables from last week, I partially completed one (in orange) and did not make any progress on the last task (in red).

1. Complete implementation of turn signal control
2. Complete radar implementation

The only real work remaining for the turn signal stuff on my end is to mitigate the issue where we need to physically turn the compass 90 degrees for it to report a change of 60 degrees – I have a potential workaround (see below)

However, I am behind on the radar implementation task. I will check with Jack on whether this is something he can take on, since theoretically we can reuse the stuff he does for the RCW specifically for the FCW. I’m also realizing it is somewhat ill-defined what radar implementation is – I think a good definition at this point is to have code to filter the points from the radar, identify which of them correspond to a vehicle directly in front (for FCW), and send the distance and relative velocity of that car to the UI.

Upcoming Deliverables

The deliverables for this week are similar to last week’s, except changing “radar implementation” to “radar and UI integration”.

1. Complete implementation of turn signal control
2. Complete radar and UI integration

State of the UI

Here’s a more detailed rundown of what the state of the UI is as of the time of this writing.

Turn Signal Lights

I implemented code to drive the actual turn signal lights through GPIO pins which in turn control transistor gates. There is a minor issue where the on-screen turn signal indicator isn’t synchronized with the physical turn signal light, but for now it should be OK.

Turn Signal Auto-Cancellation

For our turn signal auto-cancellation system, we are using the Adafruit MMC5603 Triple-Axis Magnetometer as a compass. As a quick refresher, the idea of using the compass is that when we activate a turn signal, we record the current bicycle heading. Then, once the bicycle turns beyond a certain angle (currently targeting +- 60 degrees of the starting heading) we automatically turn off the turn signals.

Adafruit provides libraries for Arduino and Python, but since the UI code is written in Javascript there weren’t any pre-written libraries. Therefore, I ended up writing my own code to interface with it.

As mentioned last week, I’m using the i2c-bus library to communicate with the MMC5603 chip over I2C. With help from the MMC5603 datasheet and Adafruit’s C++ library for the MMC5603, we now have support for obtaining the raw magnetic field measurements from the MMC5603.

To convert those raw readings into a heading, I implemented the algorithm described in this Digilent article.

There is an issue where rotating the magnetometer reports 60 degrees of angle change only when it’s actually rotated 90 degrees. We can work around this in code right now by setting a lower threshold to auto-cancel the turn signal (perhaps 40 degrees so it’ll turn off when the bicycle rotates 60 degrees physically), but we may need calibration. From what I briefly read online though, this won’t be a simple process.

I also implemented the ability to override the bike angle to some fake angle in the debug menu. Here’s a video I recorded (a CMU account is required) showing the overriding angle feature and the turn signal auto-cancellation.

Integration with radar

The UI side is ready to receive real radar data! The general overview is that Jack’s code will do all the data processing, identify one vehicle each in front, behind, left, and right, and send their distances and velocities in a JSON string to my program. We will be using named pipes, created using mkfifo.

On the UI side, I based my code off of this StackOverflow answer. There was a slight hiccup initially when we were testing passing data from Python to JavaScript when the Python side seemed to buffer multiple JSON strings before actually sending them over the pipe, resulting in a crash on the Javascript side as it received a single string with multiple objects at the top level which isn’t valid JSON. We fixed this by inserting a flush call after sending each string on the Python side.