Author: mstark

I spent the majority of this week working on the Kalman filter for our model output, conducting tests, and working on our upcoming deadlines. Here is what I did this past week:

• • Made and integrated Kalman filter
    • I worked independently on writing Kalman filter code for the model output
    • The goal of the system is to reduce noise associated with model output
    • The system achieves the desired effect of reducing unnecessary state transitions and reducing noise
    • Still needs to be tuned for video and final demo
  • Set up and tested system in demo room
    • Made sure that the jetson will be able to set up on ethernet in our demo room
  • Began work on poster and video
    • Added poster components related to the web app subsystem
  • Conducted user tests with 3 people
    • Had them use the system and give feedback on the UI and system usabiliity/practicality
  • Worked with Kapil on Jetson BLE integration. Researched a few options and ultimately found a suitable usb dongle for Bluetooth.

Goals for next week:

• Tune Kalman filter
• Further prepare system for video and demo
• Finish final deadlines and objectives

This week, I focused on preparation for the final presentation, testing, and output control ideation. Here’s what I accomplished:

This week’s progress:

• Testing
  • Ran tests on API latency using python test automation.  
  • Found that the API speed is well within our latency requirement (in good network conditions).
  • Ran latency tests on the integrated system, capturing the time between image capture and system output. 
  • Found that our overall system is also well within our latency requirement of 350 ms, in fact operating consistently under 200 ms. 
  • Wrote out user testing feedback form for user testing next week.
• Final presentation
  • Worked on our final presentation, particularly the testing, tradeoff, and system solution sections.
  • Wrote notes and rehearsed the presentation in lead up to class.
• Control, confidence score Ideation
  • In an effort to enhance our systems confidence score efficacy, I decided to integrate a control system.
  • I plan to use a Kalman filter to regulate the display of the system output in order for account for the noise present in the system output.
  • By using the model output probabilistic weights, I will try to analyze the output and make a noise adjusted likelihood estimate.
  • I will implement this with numpy on the Jetson side of the system and update the API in conjuction with this.

Goals for Next Week:

• Implement kalman filter on Jetson.
• Assist Kapil with BLE Jetson integration for the Adafruit bracelet.
• Continue user testing and integrate changes into UI in conjunction with feedback. 
• Work on poster and finalize system presentation for final demo.

I’d say we’re on track for final demo and final assignments.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

Images:

This week, I focused on integration tasks to prepare the system for the demo. Here’s what I accomplished:

This week’s progress:

• Jetson and Adafruit UART Integration
  • Worked with Kapil to implement UART communication between the Jetson and Adafruit
  • Helped write, run, and debug the code required to communicate via the Jetson’s UART pins.
  • Created and executed test scripts for this communication, eventually achieving functionality with the output of the model.
• Model Deployment on Jetson
  • Resolved compatibility challenges related to leveraging the Jetson’s GPU for efficient model execution. 
  • Successfully installed the necessary packages and verified the model running in real-time using a webcam.
• System Integration
  • Made changes to the model to integrate it with the API and UART communication, ensuring smooth output transmission.
  • Finalized the Jetson setup for the demo: the model now runs locally on the Jetson and transmits outputs as specified in the project write-up.

Goals for Next Week:

• Collaborate with Kapil on Bluetooth integration between the Jetson and the bracelet/Adafruit.
• Work with Noah to improve the model’s efficiency and reduce latency.
• Conduct tests for API latency to ensure real-time responsiveness.
• Begin user and user experience testing to evaluate the system’s performance and usability.

I’d say we’re on track and I feel good about the state of our project going into the demo on Monday.

UART (bracelet) working alongside API (iPad):

I forgot to capture photos of the model on Jetson/whole system in action.

Accomplishments: This week, I completed deployment using Gunicorn and Nginx, which is now set up to handle concurrent requests effectively and provides a stable environment for our application. The website is ready to be viewed on the ipad and used in the system. Additionally, I designed the Rest API that will connect to the model, facilitating data transmission from the Nvidia Jetson to the website. This api is made using the python requests package, and integrated into the Django web app. Through Ajax, the display can make real-time updates on the display. Note that while the API is designed and integrated into the webapp, but it hasn’t been tested yet.

Next Steps: On schedule, my goal next week is to integrate and run both the API and the facial recognition model directly on the Jetson. This milestone will be essential to finalize the whole system’s integration and confirm that all components work cohesively to deliver consistent, real-time data flow and processing. We will also begin testing the latency of the API to ensure fluid user experience, and consistent timing between the iPad and bracelet. From there, I will move to working on adding a watch/timer display to the interface to show time elapsed, and add a bar to show confidence. These UI enhancements will aid in the usability of the display.

1. What did you personally accomplish this week on the project?

This week, I made substantial progress on our application by setting up an EC2 instance and deploying the application on it. This involved handling critical tasks like configuring the server, running application migrations, and integrating various components to ensure the system functions cohesively.

I also refined the application mocks, which are now available for review in the team report, and consolidated the app to be more lightweight and efficient to run. The application is almost fully operational, and I am close to making it accessible via a URL, enabling team members and future users to access it directly.

2. Is your progress on schedule or behind?

My progress is on schedule. Deploying the application to EC2 and reaching this stage without major issues is a significant milestone, and I am pleased with the current pace.

3. What deliverables do you hope to complete in the next week?

Next week, my primary focus will be A. getting the application fully accessible via web domain, and B. setting up the API call from the NVIDIA Jetson to the server, which will control the server’s output based on input from the Jetson device. Additionally, I plan to establish the Jetson’s TCP connection and enhance the web interface by incorporating features like bar charts to display confidence levels and a stopwatch to track time elapsed per detected emotion.

Mason’s Status Report for 10/20

This week, I focused on significant front-end enhancements for the EmotiSense web app. These updates improve user interaction and provide better feedback on emotion recognition results. Specific updates are:

• Added individual views for each of the six emotions (happiness, sadness, surprise, anger, fear, and neutral), allowing users to see a dynamic visual representation for each detected emotion.
• Implemented a display for the confidence score alongside the emotion, which helps users gauge the accuracy of the detection.
• Created logic to track and display the time elapsed from the last emotion transition, offering insight into how quickly emotions change during a session.

Additionally, I spent time researching how to make API calls directly from the Nvidia Jetson Xavier AGX. This research focused on how the Jetson communicates with cloud services and how it handles real-time emotion data processing efficiently. Key learnings included optimizing the use of TCP/IP protocols and managing data transmission with minimal latency.

Is your progress on schedule or behind?

My progress is on schedule. The front-end enhancements have been successfully implemented, and I have made significant progress in understanding how to integrate the Nvidia Jetson for real-time data transmission.

What deliverables do you hope to complete in the next week?

• Finalize API integration design to enable real-time data processing between the Nvidia Jetson Xavier AGX and the cloud-based web app.
• Conduct latency testing to ensure the system can handle real-time emotion data efficiently.
• Further enhance the user interface with responsive design elements and user feedback on system performance.