We finally have a decent replacement camera, not the camera module 3 which is still not here, but an Arducam IMX519 Raspberry Pi camera which meets all the requirements we had except the IR functionality which won’t affect our demo. As such, at this point, the biggest risk is the time crunch and to finish all deliverables on time.
No changes were made to the existing design of the system other than the use of this new camera. It was needed since our raspberry pi camera module 3 is not here and our demo/final deliverables are coming up. The costs are minimal since the camera is easy to replace in our system pipeline, and did not cost too much money.
Currently the next things to do would be to integrate the new camera and fully test its capabilities. In addition to that, work is going to be needed in making our poster and final deliverables.
Testing we have done:
Unit tests:
- Cooling and power supply
- We ran the raspberry pi for 55 minutes with the full implementation
- We saw the temperature stay below 75C, avoiding thermal throttling
- Power supply kept our device stable the whole time
- Both met our requirements
- ML Models
- We evaluated the model pipeline on images at different distances (5m, 10m) and weather/lighting conditions (day, night, rain, night+rain) of ideal image quality
- We got a 98.6% precision and 79.1% recall, meeting our 90% precision requirement but not our 85% recall requirement
- We will adjust with cutoff confidence levels to try and trade some of the precision for recall to meet both requirements
- GPS Accuracy:
- We got GPS location at 10 locations we know the true coordinates of outside (ex. CMU Flagpole) and compared it with the GPS reading at these locations
- All results were in 31m, well within our 200m requirement
- Edge-Cloud Communication:
- We ran 15 simulated matches on the raspberry pi, which should show up properly in supabase and 15 updates to the database should be added to the Edge database
- This should update 100% of the time which we observed in our testing
System tests:
- Timing requirement:
- We ran the raspberry pi for 55 minutes with the full implementation
- All images were checked within our 40 second requirement
- Full system performance test:
- We evaluated the model pipeline on images at different distances (5m, 10m) and weather/lighting conditions (day, night, rain, night+rain) of our camera’s quality (the old webcam at the time)
- 64.7% precision and 23.6% recall observed, well below our 90% and 85% targets respectively
- We have a new camera to test, which should have much better results