Wearable belt to help the visually impaired
For this week I got a deeper understanding of the DepthAI Python API. I have successfully implemented code with stereo depth camera and edge detector enabled. I have also read a paper on ground-level obstacle detection, but I have not integrated the algorithm into the system.
Next week I will work on other possibilities of the stereo depth camera and integration of obstacle detection/threat classification.
This week, I focused on finishing up the arduino interface. I remodeled our pin assignment, since I found a way to use the analog pins provided to manage the trigger pins. As a result, we now have 18 pins in total that we can use, which satisfies our requirement. Furthermore, I was able to finish up the arduino interface and tested on operating 3 sensors and vibrators at the same time.I have concluded our work on arduino for now, and next week, I will focus on the python script in raspberrypi.
This week I reviewed comments of our design report and prototyped the obstacle alert model based on ultrasonic distance and speed. The model has 3 alert levels(high, medium, low) that correspond to vibration intensities, given conditions of speed>3m/s or distance <= 1m, speed>2m/s or distance <= 2m, and speed>1m/s or distance <= 4.5m respectively. The model is further tested with multiple ultrasonic sensors and vibration motors.
For Arduino and Raspberry Pi integration, an API is established with semantics of activateSensor, modifyVibrator <num 1-6><level 0-3>, deactivateVibrator <num 1-6> and deactivateSensor.
For control logic, a primitive alert model based on obstacle’s ultrasonic distance and speed is developed. The model has 3 alert levels(high, medium, low) that correspond to vibration intensities, given conditions of speed>3m/s or distance <= 1m, speed>2m/s or distance <= 2m, and speed>1m/s or distance <= 4.5m respectively.
The above integration and control logic are tested to work with multiple ultrasonic sensors and vibration motors.
This week I mostly worked on the design report and integrating Raspberry Pi with depth camera imaging.
For the design report, I wrote the sections of abstract, introduction, use case and design requirements, bill of material, schedule, and related work. While writing the report, we realized that we need to order a belt and more vibration motors and I put in the request.
For depth camera sensing, I looked into Oak-D’s examples of edge detection, stereo depth with Ning and tested out their effects after integration into our raspberry pi main. Our initial approach will be forwarding the stereo depth processed frames, which shows depth difference without noisy features like printed text on the surfaces, to the edge detection module. The implementation will be left to after spring break.
After spring break, I will further work on integrations with Raspberry Pi and design of sensor data modeling.
This week, my main focus is expanding our efforts on the arduino, and providing an arduino script that would be controlled using serial communication with python script. The system works well with a single sensor single vibration unit model, I was able to control the vibration and the on and off of the sensors through writing serial messages to the arduino. Furthermore, I was able to write a basic python script, that would communicate with the arduino to turn on the vibration unit whenever the sensors senses object <30cm, and turn off otherwise. We ran the script on raspberry pi and it worked well. I also tested connection and controlling multiple sensors with the same trigger pin, and it also worked.
One particular challenge I noticed was using pulseIn to receive multiple sensor information at the same time. In which I will have to read the source code and implement my own pulseIn
Next week, I will be increasing the scale of the arduino system to include all sensors and vibration units.
This week, I focused on working with the integration of OAK-D depth camera and the Raspberry Pi. We have successfully connected the Raspberry Pi to CMU-DEVICE and installed proper environment to run sample scripts from OAK-D documentation.
Next week my work will focus on finding appropriate NN model for the OAK-D camera as well as explore the necessity of running the system in headless/standalone mode.
This week we made the milestone of successful integration of our sensors, actuator, and controller boards. The system is minimally functional in that
Next week, we will look into belt design, plan out more details in our design report, and do more system integration and algorithmic modeling incorporates different data cohesively.
This week I mostly worked with Ning on Raspberry Pi integration with the Oak-D depth camera. We first successfully registered our Raspberry Pi as a CMU device and managed to SSH into it from laptop, then downloaded all the Oak-D dependencies to run its “Hello world” script and was able to activate the camera and take in data with Raspberry Pi. I also started integrating oak-d skeleton code to the raspberry pi main file. Meanwhile, Alex helped me set up VS Code ssh integration to allow a smooth experience with editing Raspberry Pi files.
Next week, I will further integrate Oak-D pipeline into the raspberry pi main and look into belt design with the team.
This week my work is mainly on learning to work with the Luxonis OAK-D depth camera. I have successfully run a program of object detection with pre-existing NN models (on my PC; re-establishment of the proper environment on the Raspberry Pi is expected). I also worked on the initialization of Raspberry Pi 4 with Kelton and discussed the block diagram of our system with the team.
Next week my work will be focused on finding (or possibly designing) appropriate algorithms for our purpose, as well as integrating the OAK-D depth camera with Raspberry Pi.