I finished the frontend of the heat map for the pet activity logs. I switched the heat map library I was using to jsheatmap as it was compatible with TypeScript and had documentation that was easy to understand and implement with. As well, I confirmed a data format between the CV and web application for the pet activity logs. The CV would send an array containing the amount of frames a pet stayed in a specific cell of the room that goes in order from the top left to the bottom right of the room. This array is sent to the web application, and the web application would keep the tally of the amount a pet spent in a specific cell of the room. Also, we have integrated this task between the CV and the web application.
To implement notifications for a pet entering the forbidden zone, I had to learn about polling, which is making requests to the backend every few seconds to see if the notification flag, which is turned to true by the CV as it makes a POST request whenever the pet moves into the forbidden zone, is set to true. Once the notification flag is true, then a popup is created on the frontend to notify the user that the pet has entered the forbidden zone. After trying a few methods of polling, I found a custom React hook called useInterval that was able to do the task above. We have successfully tested sending a notification from the CV to the web application.
The last task I have started but have not finished is displaying live video feed of the room to the user. We based our implementation on a previous implementation that send a constant loop of arrays of byte data representing the image taken from the CV to the web application locally, but we ran into problems with encoding and decoding the arrays, especially during GET and POST requests.
I am slightly behind schedule as I should have finished displaying the live video feed. I should be able to finish up this task next week as it is a matter of encoding and decoding the images correctly while the data is being passed on between the GET and POST requests.
In addition to finishing up displaying the live video feed of the room, I hope to add authentication by using Google OAuth2.0, and I hope to clean up the user flow of the website and make the website more intuitive, appealing, and user friendly. Lastly, I look to deploy the frontend after all tasks are finished.
The above picture uses fake data, but it is an accurate depiction of the heat map.
The above picture is what is displayed on the frontend when a pet enters the forbidden zone.
This is code currently in progress for displaying live video feed.