What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?
- We are currently using the software solution, which is the distance estimation feature in the OR model, for the proximity module. Although it has some decent accuracy with all measurements to be within +- 30cm, the uncertainty is around 20%, which may jeopardize the success of the use case requirement of the detection distance. This risk can be mitigated by using an Arduino board to connect to the Jetson and the ultrasonic sensor to get an accurate distance. However, this alternative will increase the weight of the product, which can go over the use case requirement of the weight of the device, and have latency in data transmission. It will also increase the development time to transfer the distance data to the Jetson. This is the tradeoff we still need to consider: accuracy vs. weight and latency.
- After connecting the camera module and the OR model, we realized that there is a latency for every frame, possibly due to the recognition delay. Therefore, even if the camera is turned to a different object, the Jetson outputs the correct object around 5 seconds after the change. This can crucially jeopardize the success of the project because we had set the use case requirement to be less than 2.5 seconds of recognition delay. The risk can be mitigated by using an alternative method of capturing frames. A screen capture can be used instead of the video stream, which can potentially resolve the delay issue. However, the problem with this method is that the process of Jetson Nano running the program of a camera capture, transferring the information to the model, and deleting the history of the captured frame can take more time than the current delay. This alternative solution can also delay the product delivery due to more time necessary for the modification of the program.
Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?
Besides the change from using an ultrasonic sensor integration to using the DE feature for the proximity module, no design change has been made.
Provide an updated schedule if changes have occurred.
Josh and Shakthi will work on integration and testing of the headless device. Meanwhile, Meera will work on the box for the device. Consequently, Josh, Meera, and Shakthi will conduct user testing and work on the final demo.
List all unit tests and overall system test carried out for experimentation of the system.
| Testing | Metrics | Result |
| Object Recognition Model | > 70% on identifying an object | 95% (38/40 images, 5 objects) |
| Distance Estimation Feature | ± 30cm of actual object distance | Tests done on 4 different distances. Average of 21.5% uncertainty within ± 30cm |
| Text-to-speech Module | user-testing for surrounding sounds 20 trials each object | 100% (20/20 person, 20/20 couch, 20/20 chair, 20/20 cat, 20/20 cellphone) |
| Vibration Module | > 95% accuracy on vibration | 100% (20/20 on person, 20/20 on nothing) |
| Device Controls (buttons) | 100% accuracy on controls | 100% (20/20 on button A, 20/20 on button B) |
| Module Integration (weight) | < 450g on the overall product weight | 192g (device) + 209g (battery) = 401g < 450g |
| Recognition Delay | < 2.5s to recognize an object | ~8 seconds delay for 20 seconds testing.
Frame delay due to the latency of the OR model |
List any findings and design changes made from your analysis of test results and other data obtained from the experimentation.
- Chose pre-trained model instead of trained model with indoor object dataset
| Model | Real Objects | Detected | Falsely detected | Percentage (%) |
| Pre-trained | 58 | 49 | 5 | 84.4 |
| Trained | 58 | 21 | 4 | 36.2 |
- Chose Distance Estimation feature in the OR model instead of ultrasonic sensor
- Ultrasonic sensor does not work well with Jetson Nano
- DE feature rarely goes over ± 30cm, although some calibration is necessary
| Actual (m) | Detected (m) | Off (m) |
| 1.80 | 1.82 | + 0.02 |
| 1.20 | 0.89 | – 0.31 |
| 0.20 | 0.38 | + 0.18 |
| 2.2 | 1.94 | – 0.26 |