Posted on

Rebecca’s Status Report for 4/1/2023

This has not been my most productive week to date, as I tend to spend a lot of time working on Saturday, but was quite busy today with personal business – most of my work time will be offloaded to Sunday, but thus I don’t have major progress to report in this blog. However, I’m content with the status of my portion. There’s one small bug I’m still trying to work out in the CV code, where sometimes when multiple bounding boxes overlap, the smaller bounding box is favored. Other than resolving this, I would say that the tracking and motion detection are working exactly as I had planned. I had hoped to post a video, but my computer kept crashing when I tried to edit the video and I didn’t have time before midnight to figure that out. Perhaps I will edit to add this later. Other than some tweaking of the core CV code, I have spent this week preparing to start integration for the interim demo. Specifically, I’ve written rough drafts of code for where the CV will connect to other pieces (i.e. forbidden zones and tracking activity), and done a bit of research into how we will go about connecting the web app to the CV when they’re hosted on two separate computers (likely sockets).

Other than that one last stubborn little bug, I would say that the core CV is done, which is nice and on schedule. However, we are moderately behind on really being able to dive into integration, including as it relates to the CV. The plan to remedy this is that 1) it will be our primary focus from here on out and 2) we will begin meeting in person, starting tomorrow.

In the next week, it will be full steam ahead on integration. First and foremost, the goal will be to get the CV communicating with the Web App for the interim demo on Wednesday. Depending on how Max is doing, we may try to incorporate the ML as well, though likely still keeping strictly on laptops for the time being. In the latter half of the week (once we can hopefully establish exactly how we will be moving forward with the project), I should be able to turn my focus back to the hardware we’ll be using, as this has slightly fallen on the back burner due to the uncertainty around our project.

Posted on

Max’s Status Report for 4/1/2023

This week I was able to continue work on the transfer layers, however, due to some serious health complications, I was fairly limited in my time to work on this. Due to this setback, our team is looking into integration alternatives as we look to start integration this weekend. Aside from this, though setback due to health, I am only slightly behind schedule.

Posted on

Brandon’s Status Report for 4/1/2023

I focused on making the task on my web application ready for integration with other parts of the system. Specifically, I accomplished allowing get requests for the pet images and forbidden zones based on the pet name specified in the get request so the ML and CV are able to get this data respectively. As well, I created a dashboard to give the user easy access to creating new forbidden zones and upload pets images for a new pet and view already entered forbidden zones and pet images for existing pets.

I am changing my schedule such that I will be working on tasks that are most relevant for what we will be integrating in our in person meetings. Hence, I will focus on pet activity logs next after finishing integrating forbidden zones as this is one of the core tasks that we hope to integrate between all sub systems.

I hope to finish displaying the frontend of the pet activity logs and work on the integration process for the pet activity logs between the web application and the CV.

Posted on

Team’s Status Report for 4/1/2023

Risks and mitigation

Currently, the most significant risk is that we may be unable to incorporate the Machine Learning part into the system due to personal reasons. Hence, we would be unable to identify different pets in the room and the creation of the different forbidden zones for each pet would not make sense anymore. To mitigate this risk, each person is implementing a different part of the project. So, we could integrate the Web App and CV only if the Machine Learning would not be available for integration, and we could add in the Machine Learning portion if it is finished before the final demo. If the system only includes the Web App and CV, then the creation of one forbidden zone for all pets and detecting if any pet enters the forbidden zone would make the most sense as we would be unable to differentiate the pets without Machine Learning.

Project Changes (why & cost)

If we are unable to add in the Machine Learning part of the project, then we would not have to use a Jetson, which is most beneficial for training the machine learning model, to support integrating the system, which would reduce the cost of the project significantly as the Jetson costs $150 and the internet adapter needed for the Jetson costs $15-30. We would start with two computers (one for Web App and one for CV) to start integrating the system. If we can integrate Machine Learning into the project, then there would be no major changes.

Schedule Changes

We are going to change the order of integration from what was listed on the schedule. We plan to start with integrating Web App and CV, specifically sending forbidden zone data chosen by the user from the Web App for the CV to detect pets that violate the forbidden zone boundaries using sockets for the interim demo. We hope to integrate parts of the Machine Learning into the project for the interim demo as Max said he is feeling better enough to help with integrating.



Posted on

Brandon’s Status Report for 3/25/2023

This week I worked on creating the frontend and backend of a user designating the forbidden zone for a pet. I learned how to use state and props in React so that I could keep track of the state of each forbidden zone cell. For my forbidden zone example, I created a 10×10 grid (sizing will most likely be changed), and I keep track of the position in the x and y directions and if a user has clicked the cell (which represents a forbidden spot or not). Once a user submits their forbidden zone data, then it sent to the database that is managed by the Django REST framework. A picture of the frontend for the forbidden zone task is shown below.

I was hoping to looking into how to display a frontend for the pet activity logs using the heat map library described from last week, so I am currently behind. I hope to do this for next week and make sure everything is stable for the interim demo.

Like I mentioned above, I hope to create a frontend for displaying pet activity logs using a heat map. As well, I hope to make sure everything I currently have is working for the interim demo, and work on integrate some parts of my web application with other parts of the project.

Posted on

Rebecca’s Status Report for 3/25/2023

The CV code is pretty much done. All that is left is a bit more tweaking/debugging, and of course integration with the ML (which will be more so a team activity, so that will happen when Max is ready).

My progress with regards to the CV is pretty much on track, other than the little bit of tweaking I’d still like to do. However we are fairly behind on learning to work with the hardware, so that will be almost entirely the focus for this upcoming week. I plan to work on basic set-up for the Jetson, and hopefully we can accomplish enough this week to be back on schedule for fuller integration next week.

In the next week, I intend to focus on setting up the Jetson. I have found a nice step by step for general set up the developer kit which I hope will be doable with minimal struggles. I aspire to start on this early in the week (whenever we get the Jetson), so that we can discuss any major issues in mandatory lab on Wednesday. I also intend to finish the refinements on the CV, hopefully tonight or tomorrow. Then in the later part of the week, we can look at hooking up the camera with the Jetson.

Posted on

Max’s Status Report for 3/25/2023

This week I was finally able to produce some results for the simplified ML classifier, and assisted my team in making the decision to go forward with implementing the Jetson over the Raspberry Pi. I have moved onto finalizing the additional transfer layers that will be used to create our final classifier, which is on schedule. I have also been doing more research on potential for parallelization through NVIDIA options available through the Jetson. This puts me back on schedule. In addition, the NVIDIA research is to help avoid any complications that may arise from implementing the classifier as I am, and provide shortcuts that will enable me to speedup the classifier if we run into future problems with our speed. All together, this puts me on schedule.

Posted on

Team Status Report for 3/25/2023

  1. Risks and mitigation

As we are changing the primary hardware of our project – to an objectively more complex system that we also have no experience with – our main risk at this point will be in using the Jetson Nano. The mitigation for this risk will be to ask others with more experience when we have issues using the Jetson, i.e. course staff and/or other teams. 

  1. Project Changes (why & cost)

Using a Jetson Nano will require us to buy some form of internet adapter, which seems like it will use another ~$15-30 of budget for a relatively cheap adapter. It also changes our overall proposed product cost requirement from <$100 to around $180-200 in the current market (i.e., post pandemic), but since our primary hardware is coming from inventory this will have minimal impact on our $600 project budget. Speaker requirements may also change (or be scrapped altogether), but this is tbd as it’s lower on our list of priorities. The reason for this change is because the ML requires greater computational power than an RPi has, and a Jetson seems like an easier/safer option than trying to run the ML on the server.

  1. Schedule Changes

Everyone will be devoting a lot of time this week to learning the Jetson Nano system. In particular, we must figure out the general setup, how to connect it to the internet, and potentially look into how to connect it to a speaker. This puts us a bit behind on what we had hoped for in terms of starting to integrate our pieces with the hardware/each other, but if we can get the basics figured out this week then we should be on track to flesh it out more fully next week, and begin testing shortly thereafter.

Posted on

Brandon’s Status Report for 3/18/2023

This work, I worked on the ethics assignment, looked into how to deploy the machine learning aspect of the project onto an Amazon EC2 instance, and worked on implementing choosing forbidden zones as a user and showing pet activity logs to the user. As our team has been investigating where to run the machine learning, such as running it on the web server, I looked into how matching learning can be run on the Amazon EC2 server as I have the most experience with this platform. From my research, there seems to multiple examples of other machine learning models being deployed on Amazon EC2 using a deep learning amazon machine image, so I believe that this is a possible idea. For implementing pet activity logs as a heat map, I have found out that I can use a library called heatmap.js that will allow me to create heat maps based on data taken from OpenCV.

I am behind on work due to homework in other classes, but I will be making sure that all the core tasks for the web application are functional before the interim demo.

I hope to finish implementing forbidden zones and showing pet activity logs in the next week. As well, I hope to create a user interface design based on my user flow diagram I created.

Posted on

Rebecca’s Status Report for 3/18/2023

Aside from working on the ethics assignment, I’ve spent this week working on basic RPi setup and the mostly fleshed out CV algorithm. Basic RPi setup seems to have been successful, but I have yet to run it on CV-ish examples as we have yet to get a camera for the RPi, unfortunately. I have most of a ‘rough draft’ for the CV algorithm for our project, but am still debugging and filling in a couple blanks – mostly where different pieces (i.e. motion detection versus tracking versus overlap) meet up. I hope to have a full working draft by the end of tomorrow (Sunday 3/19).

My progress is slightly behind – I haven’t been able to work with Pi as fully as I had hoped by this point due to lacking a bit of necessary hardware, and the draft for the overall CV isn’t quite complete. However, if I can finish the CV draft by tomorrow this will put me at a good point. With regards to the Pi, it’s likely that we’ll go ahead with the testing we had hoped for this week regardless, so barring any significant problems that arise when we try to do this, that should also put me back on track for working with the hardware. We also intend to explore other routes for running the ML, so it’s likely that I’ll be doing additional work this week researching/experimenting with those.

In the next week, I plan to have a fully working draft of the CV portion that I can run on my laptop (by early in the week, so I can use the latter half of the week to refine it a bit). I also hope that we will be able to begin some integration testing of the ML algorithm on the Pi or other platforms, though this will also depend on where my team mates are at.