Author: zhuoran2

This week I mostly worked on the design report and integrating Raspberry Pi with depth camera imaging.

For the design report, I wrote the sections of abstract, introduction, use case and design requirements, bill of material, schedule, and related work. While writing the report, we realized that we need to order a belt and more vibration motors and I put in the request.

For depth camera sensing, I looked into Oak-D’s examples of edge detection, stereo depth with Ning and tested out their effects after integration into our raspberry pi main. Our initial approach will be forwarding the stereo depth processed frames, which shows depth difference without noisy features like printed text on the surfaces, to the edge detection module. The implementation will be left to after spring break.

 

After spring break, I will further work on integrations with Raspberry Pi and design of sensor data modeling.

This week we made the milestone of successful integration of our sensors, actuator, and controller boards. The system is minimally functional in that

• raspberry pi can take in data from depth camera
• Arduino can make the coin motor vibrate with different intensities given multiple ultrasonic sensor reading
• through serial communication, raspberry pi can send Arduino control data while Arduino can send raspberry pi sensor data

Next week, we will look into belt design, plan out more details in our design report, and do more system integration and algorithmic modeling incorporates different data cohesively.

 

This week I mostly worked with Ning on Raspberry Pi integration with the Oak-D depth camera.  We first successfully registered our Raspberry Pi as a CMU device and managed to SSH into it from laptop, then downloaded all the Oak-D dependencies to run its “Hello world” script and was able to activate the camera and take in data with Raspberry Pi. I also started integrating oak-d skeleton code to the raspberry pi main file. Meanwhile, Alex helped me set up VS Code ssh integration to allow a smooth experience with editing Raspberry Pi files.

Next week, I will further integrate Oak-D pipeline into the raspberry pi main and look into belt design with the team.

This week I was focused on testing and integrating Arduino with the HC-SR04 ultrasonic sensor and the vibration motor.

From the ultrasonic sensor test, I got a sense of the frequency and format of the collected data  and decided that it is feasible to process the data points with methods like moving average to adjust data rate as to suit transmission to Raspberry Pi while discovering that our original range specified in quantitative requirement has to be reduced from 5m to 4m due to the sensor’s tested accuracy. The sensor can provide feedback at a rate of more than 100hz, which is way more than what we need(our team will likely work with 20hz or 30hz). Furthermore, I tested the connection with 3 different sensors at the same time with a motherboard, and the connection is able to process data from all 3 sensors according to will, which proves that our original design with multiple sensor is feasible.

Meanwhile, for the vibration motor, I was able to control its vibration intensity with Arduino’s analog pin which is otherwise not feasible with Raspberry Pi. I tested runing several different disks with different intensity at the same time, and it was sucessful.

Another concern that was addressed this week was the pin assignment for the arduino UNO. We were originally woried that the UNO will be insufficient for our number of sensors and disk motors, but after examining the size of the motors and sensors, we have decided to limit the number of sensors used to 6 maximum. In which case, we will need 1 DIGITAL for sensor control(we operate all 6 sensors at the same time), 6 DIGITAL PINS for sensor feedback, 6 PWM pins for vibration motor control, which is exaclty the number of digital pins provided by the UNO model. Thus we will be sticking with the arduino UNO board.

Next week, I will integrate the control interface for multiple sensors and vibration motors in arduino. Further more, I will work with Kelton to implement a basic control script in python to run the sensors and vibration disk through communication between the raspberry pi and arduino.

This week my work is centered around reviewing our design and checking if any technical incompatibility arises while building.

First, I drew the detailed block diagram, incorporated our revised requirements and designs based on TA and professor feedback (e.g. using Arduino to gather sensor data and control vibration motor while using Raspberry Pi for depth camera and control logic) into the design review slides. Then to verify design with the newly arrived parts, I helped Alex with how to test HC-SR04 ultrasonic sensor with Arduino and set up Raspberry Pi with help from Ning.

Next week, I will work on connecting Raspberry Pi online to test its integration with Arduino and the Oak-D depth camera.

This week I peer reviewed other projects, collected the requested Raspberry Pi board, created our git repo, looked into how to set up Raspberry Pi, and ordered a Raspberry Pi SD Card.

This week, we finalized the general idea of our capstone project. Initially, we were weighing between building an embedded system that makes food or drinks e.g, eggs, cocktail, boba tea or something computer-vision based to guide the blind. After hours of discussion and feedback from the faculty, we settled on building a smart wearable belt that can give the visually-impaired users intuitively actionable feedback.

With this idea, we delved deeper and fleshed out more details : naming the project The Bat Belt as the product involves echolocation; comparing existing tech gadgets built by university lab, startups and another capstone team to the traditional cane and our concept; establishing a wider obstacle detection range with more granular haptic feedback as one of our core appeals …

We also recognize the challenges and risks that lie ahead (such as incompatible sensor and microcontroller, software integration with depth camera, mapping multiple sensor data algorithmically into a cohesive feedback) and planned accordingly as detailed in our proposal.

This week I mostly worked on the proposal presentation with my teammates, researching and debating about problem definition, solution approach, sensor tradeoffs etc as we go. Additionally, I placed the orders for ultrasonic sensors and raspberry pi board and worked on formatting of the website. This coming week, hopefully as the parts arrive, I can get a sense of the data collected.