Progress
This week we did some user position testing, but the error was kind of high for people who were somewhat far away and people in more extreme angles. Also, if one is too close to the camera, the camera will not catch the person’s face. More on testing is in the section below.
I also worked on getting the Raspberry Pi to work. I finally got it to build the pyrealsense2 module from source (this build alone took multiple hours, and it had to all be built consecutively because apparently it was getting errors from previous partially built builds). So I had to remove and just completely restart a build. After it built, I had to debug why the instructions on the Intel RealSense Github (on how to build it on Raspbian) wasn’t working. Apparently although the librealsense.so file was getting put into usr/local/lib, the pyrealsense2.so file wasn’t. Although it seems simply in hindsight, I was very confused when trying to figure this out. What I did instead is set $PYTHONPATH in the wrappers/python folder, which had pyrealsense2.so. I also discovered something else after that – To find the latitude/longitude of an address, we use geopy/Nominatim, where geopy sends a GET request to some Nominatim server. However, I was getting that these requests were timing out because apparently, Nominatim has a limit on how many requests you can do, and it’s plausible that this is partly tracked using IP address, and someone at CMU used up all our requests. This is for sure the cause of the issue because once I used a VPN, this issue disappeared. I’m still thinking about how to avoid this (use a different library, make our user_agent’s name more unique with a random number), but once this issue is solved, all that we should have to do to finish with the Pi is to work out which port we should connect the Arduino and make the changes to the lines using the Serial package.
Testing
We did some distance testing this week, but found that for people in more extreme angles or extreme distances, the distance calculation is not super accurate. The way we did these tests were by using an image and the depth data to calculate the user position, and then physically measuring the distance to the person. Ideally we would want a percent error of below 10%, but this might not be achievable, partly due to previously mentioned issues, partly due to it being kinda hard to accurately measure the distance to the person with just one measuring stick, as it’s kinda hard to keep the stick straight and perpendicular to a specific spot in the air. Also, OpenCV sometimes interprets some objects as people, when this is clearly inaccurate. For the distance calculations, some online sources say it is plausible that using a radians/pixel approach is not linear, in other words, it may not be acceptable just to divide horizontal FOV by the number of horizontal pixels. However, I’m not actually sure if this is the case, as it is hard to get data on, nor am I sure how to correct this if this is the case. In the future we plan to do more integrated testing and individual testing.
Schedule
We are a little behind schedule, as basically everything works, but the Raspberry Pi’s code needs to be a bit edited to account for timeout issues with Nominatim. However at this stage in time, we should be solely focused on testing. But because we aren’t very far behind, we can focus most of next week on testing.
Next Steps
The goal for next week is to finish up with working with the Pi, and spend most of next week testing.