Brandon’s Status Report for 3/25/2023

This week I worked on creating the frontend and backend of a user designating the forbidden zone for a pet. I learned how to use state and props in React so that I could keep track of the state of each forbidden zone cell. For my forbidden zone example, I created a 10×10 grid (sizing will most likely be changed), and I keep track of the position in the x and y directions and if a user has clicked the cell (which represents a forbidden spot or not). Once a user submits their forbidden zone data, then it sent to the database that is managed by the Django REST framework. A picture of the frontend for the forbidden zone task is shown below.

I was hoping to looking into how to display a frontend for the pet activity logs using the heat map library described from last week, so I am currently behind. I hope to do this for next week and make sure everything is stable for the interim demo.

Like I mentioned above, I hope to create a frontend for displaying pet activity logs using a heat map. As well, I hope to make sure everything I currently have is working for the interim demo, and work on integrate some parts of my web application with other parts of the project.

Rebecca’s Status Report for 3/25/2023

The CV code is pretty much done. All that is left is a bit more tweaking/debugging, and of course integration with the ML (which will be more so a team activity, so that will happen when Max is ready).

My progress with regards to the CV is pretty much on track, other than the little bit of tweaking I’d still like to do. However we are fairly behind on learning to work with the hardware, so that will be almost entirely the focus for this upcoming week. I plan to work on basic set-up for the Jetson, and hopefully we can accomplish enough this week to be back on schedule for fuller integration next week.

In the next week, I intend to focus on setting up the Jetson. I have found a nice step by step for general set up the developer kit which I hope will be doable with minimal struggles. I aspire to start on this early in the week (whenever we get the Jetson), so that we can discuss any major issues in mandatory lab on Wednesday. I also intend to finish the refinements on the CV, hopefully tonight or tomorrow. Then in the later part of the week, we can look at hooking up the camera with the Jetson.

Max’s Status Report for 3/25/2023

This week I was finally able to produce some results for the simplified ML classifier, and assisted my team in making the decision to go forward with implementing the Jetson over the Raspberry Pi. I have moved onto finalizing the additional transfer layers that will be used to create our final classifier, which is on schedule. I have also been doing more research on potential for parallelization through NVIDIA options available through the Jetson. This puts me back on schedule. In addition, the NVIDIA research is to help avoid any complications that may arise from implementing the classifier as I am, and provide shortcuts that will enable me to speedup the classifier if we run into future problems with our speed. All together, this puts me on schedule.

Team Status Report for 3/25/2023

  1. Risks and mitigation

As we are changing the primary hardware of our project – to an objectively more complex system that we also have no experience with – our main risk at this point will be in using the Jetson Nano. The mitigation for this risk will be to ask others with more experience when we have issues using the Jetson, i.e. course staff and/or other teams. 

  1. Project Changes (why & cost)

Using a Jetson Nano will require us to buy some form of internet adapter, which seems like it will use another ~$15-30 of budget for a relatively cheap adapter. It also changes our overall proposed product cost requirement from <$100 to around $180-200 in the current market (i.e., post pandemic), but since our primary hardware is coming from inventory this will have minimal impact on our $600 project budget. Speaker requirements may also change (or be scrapped altogether), but this is tbd as it’s lower on our list of priorities. The reason for this change is because the ML requires greater computational power than an RPi has, and a Jetson seems like an easier/safer option than trying to run the ML on the server.

  1. Schedule Changes

Everyone will be devoting a lot of time this week to learning the Jetson Nano system. In particular, we must figure out the general setup, how to connect it to the internet, and potentially look into how to connect it to a speaker. This puts us a bit behind on what we had hoped for in terms of starting to integrate our pieces with the hardware/each other, but if we can get the basics figured out this week then we should be on track to flesh it out more fully next week, and begin testing shortly thereafter.

Brandon’s Status Report for 3/18/2023

This work, I worked on the ethics assignment, looked into how to deploy the machine learning aspect of the project onto an Amazon EC2 instance, and worked on implementing choosing forbidden zones as a user and showing pet activity logs to the user. As our team has been investigating where to run the machine learning, such as running it on the web server, I looked into how matching learning can be run on the Amazon EC2 server as I have the most experience with this platform. From my research, there seems to multiple examples of other machine learning models being deployed on Amazon EC2 using a deep learning amazon machine image, so I believe that this is a possible idea. For implementing pet activity logs as a heat map, I have found out that I can use a library called heatmap.js that will allow me to create heat maps based on data taken from OpenCV.

I am behind on work due to homework in other classes, but I will be making sure that all the core tasks for the web application are functional before the interim demo.

I hope to finish implementing forbidden zones and showing pet activity logs in the next week. As well, I hope to create a user interface design based on my user flow diagram I created.

Rebecca’s Status Report for 3/18/2023

Aside from working on the ethics assignment, I’ve spent this week working on basic RPi setup and the mostly fleshed out CV algorithm. Basic RPi setup seems to have been successful, but I have yet to run it on CV-ish examples as we have yet to get a camera for the RPi, unfortunately. I have most of a ‘rough draft’ for the CV algorithm for our project, but am still debugging and filling in a couple blanks – mostly where different pieces (i.e. motion detection versus tracking versus overlap) meet up. I hope to have a full working draft by the end of tomorrow (Sunday 3/19).

My progress is slightly behind – I haven’t been able to work with Pi as fully as I had hoped by this point due to lacking a bit of necessary hardware, and the draft for the overall CV isn’t quite complete. However, if I can finish the CV draft by tomorrow this will put me at a good point. With regards to the Pi, it’s likely that we’ll go ahead with the testing we had hoped for this week regardless, so barring any significant problems that arise when we try to do this, that should also put me back on track for working with the hardware. We also intend to explore other routes for running the ML, so it’s likely that I’ll be doing additional work this week researching/experimenting with those.

In the next week, I plan to have a fully working draft of the CV portion that I can run on my laptop (by early in the week, so I can use the latter half of the week to refine it a bit). I also hope that we will be able to begin some integration testing of the ML algorithm on the Pi or other platforms, though this will also depend on where my team mates are at.

Team Status Report 3/18/2023

  • Risks and mitigation

At this point, our primary risk relates to the ML, and our ability to run that. Since this will be the most computationally intensive part of our project, making sure that we have enough power to run that algorithm has been our main concern, so that we achieve the time requirements we set out for ourselves. At this point, we are experimenting with/considering 3 different methods to run our ML: 1. Running identification on the pi. This seems the riskiest, and is something we hope to firmly rule in/out by the end of this weekend or beginning of next week. 2. Running identification on the web server, i.e. completely off of the hardware. This is generally the largest unknown, and so it is something we will be doing much more research on in the next few days, but it would have the advantage of significantly reducing the complexity of what’s running on our hardware (and allow us to drive the theoretical consumer cost down even further). 3. Running identification on a Jetson Nano. This is the ‘safest option’, and so it is something that we intend to be prepared for by getting a Jetson from inventory ASAP and trying to learn its setup. However it also adds a bit of cost to the project, as well as a significant amount of theoretical consumer cost, which is why we still intend to look into options 1 and 2. We hope that looking into multiple options will allow us to mitigate the potential risk that one or several of the options proves infeasible. The primary work required to evaluate each involves setting up the platforms and having an operational CNN. 

  • Project Changes

We are considering where to run the ML, as detailed above. If we find that we can run it on the Pi, then this wouldn’t be a change. If we end up running it on the server, we will need to change the general structure of how the pi communicates with the web app, but otherwise this shouldn’t be a large change to the project as a whole. If we find that our only choice is to run it on a Jetson, this will be the largest change to the project – we can get a Nano from inventory without needing to use our budget, but would probably also have to acquire a wifi adapter of some sort, which would require a bit of budget. The current impact of the changes we are considering would mostly affect communication (if we intend to host an instance of the ML separate from the hardware) and obviously our hardware.

  • Schedule Changes

Trying to get the CV/ML code to run on hardware was always scheduled for this week, so this should stay as intended. There will be additional research needed at the beginning of this week to hopefully narrow down the feasibility. We also intend to acquire a Nano to experiment with ASAP. 

Max’s Status Report for 3/18/2023

This week I have been working on a simplified version of our final ML classifier in order to start performing preliminary speed and efficiency tests for the CNN. Our group is still bouncing around a few options in terms of where we are going to host the CNN, whether its on a Pi, Jetson, or externally. Since we are still deciding, this has put some extra stress on my portion of the project to produce some testing results, so although I was originally on track, I am a little behind due to the decision our team has made to explore potentially different implementations of our system. However, I am now much more accustomed to InceptionV4 and Tensor-flow, and have a better understanding of the transfer layers I will be implementing, but have focused on creating a classifier without extensive transfer layers in order to start platform testing.

Brandon’s Status Report for 3/4/2023

During this week, I spent most of my time working on the design review document with my team. For example, we figured out the details for the testing, verification, and validation as this was one of the points that wasn’t fully addressed in the presentation. As well, I fixed the block diagram for the web application side so that it aligns more with what a block diagram should look like. In addition, I added a flow diagram of the user experience on the web application into the design review document. I have looked into how I can allow users to choose forbidden zones on the web application, but I have not implemented this yet.

I am currently behind schedule, but I will hope to make up the task of implementing the frontend and backend for a user choosing forbidden zones this week.

I hope to fully implement choosing forbidden zones on the frontend and storing this data on the backend. As well, I hope to start looking at how to display pet activity logs on the frontend using data that would be given by the Raspberry Pi.

Max’s Status Report for 3/4/2023

Our group was finishing up our design report this week. However, since our design choices and implementation for my ML portion of the project have not changed much since the start, while I was primarily finishing up the design report with my teammates, I was also able to continue to implement my InceptionV4 architecture.

As for schedule, I am on track with where I wanted to be. This week I continued my work on the InceptionV4 architecture and I am on track for that task of making a simple classifier. I was also able to better flesh out the additional transfer layers that will be required.