Team F4

Here is a link to our completed Final Video Demo. Enjoy!

Here is a pdf to our completed Final Report due 5/6/20.

Team_F4

Mary:

• Worked with Aria to integrate speaker into python scripts
• Worked with Aria to make python scripts faster & make sure individual’s names are not repeated within 15 seconds
• Completed web app user comfort tests
• Completed hardware user comfort tests

Angelo:

• Completed web app user comfort tests

Aria:

• Worked with Mary to integrate speaker into python scripts
• Worked with Mary to make python scripts faster & make sure individual’s names are not repeated within 15 seconds
• Completed web app user comfort tests

***This is status report due Sunday 4/26***

Mary:

• Successfully installed opencv and other installations necessary for Aria’s facial recognition script
• Can get encodings.pickle when new user is entered into database
• Tested detection script with image from my camera and it takes ~30 seconds to successfully output ‘mary’
• Then tested a new script that has a while(true) loop which consists of taking a picture, and performing facial recognition algorithm to determine who it is, and outputs user name or unknown. This also works with two people in view
• RPi is integrated with Angelo’s automatic downloads/addition of new people
• Attached camera to a pair of glasses

Angelo:

• Deployed to aws: http://ec2-3-83-242-73.compute-1.amazonaws.com/
• Changed Front End mistakes
• Added pictures to show the user
• Enrollment works with zip

Aria:

Mary:

• Took apart battery pack for speaker circuit… still testing
• Worked all week with Aria to follow different tutorials to download opencv with different virtual environments and no luck… continuing tomorrow

Angelo:

• Fixed logout issues in navbar.
• Added backend, user is able to add people and relationship then is able to upload pictures of each person.
• Pictures can be deleted after being added.
• Most now fix and make frontend of the devices page to make it user friendly.

Aria:

• Can now recognize multiple faces in an image!
• Can also now rerun encodings automatically whenever the training set directory is changed! Used pythons watchdog and subroutine library
• Also worked with Mary to try to get opencv on the rpi with no success. Tried all the tutorials we could find. Asked my brother for help, he tried for hours with us and helped explain kinda why we were having some problems we were having but couldn’t get us past one error. Asked Gauri’s group for help and they gave us a list of things they did, will try that soon.

Mary:

• Camera takes two pictures per seconds and saves pictures of raspberry pi

• Almost finished speaker circuit (ran into some issues with soldering => using breadboard and wires & missing battery for circuit)

• Downloaded dlib onto raspberry pi

• Set up a virtual environment on the raspberry pi

• Working with Aria to install opencv on raspberry pi

Angelo:

• Added Authorization and Authentication such that you must create an account and must be logged-in in order to access your profile which is where you can add people to recognize.
• Did more of the front end of the device page
• Started beck end models and forms. Stuck on having multiple pictures be part of a model.
• Javascript Drag and Drop.

Aria:

1. Working on having it recognize two people in a picture
2. Uploaded new test image
3. Working with Mary to get virtual environment set up so i can send her my files and integrate
4. Tried creating environment yaml file, Mary had trouble installing dependencies
5. Tried creating a spec_file.txt to create environment, Mary got slightly further but still had trouble with dependencies
6. Finally created environment from scratch and began installing all packages, having trouble installing opencv again, surprise surprise
7. Have another tutorial in mind to try, so we will try that

Mary:

• Set up camera module
• Taking pictures on RPi
• Working on taking 2 pictures per second for 10 seconds
• Looking for tutorial links for speaker

Angelo:

• Home Page has been updated a little bit with content
• Device Page has been started with its front end.
• Started designing backend of the web application.

Aria:

• put data in organized folders with appropriate names
• separated into test and training sets
• finished writing facial recognition code
• finished writing encodings code
• what the hell is a pickle file
• ah i dont have to worry about that because python has a pickle package that creates a pickle file for me
• trained encodings, for some reason it doesnt like jpegs but it worked anyway?
• tested on the test set and it works!!
• takes 10 seconds tho and i have no idea how to cut that down except for not using hog but i have to use hog because im using an RPi and it can’t handle anything more extensive

Mary:

• Set up camera
• Working on getting camera to take 2 pictures per second
• Mary’s Projected Schedule – Gantt Chart
• Updated Risk Management:
  • My RPi is separate from facial recognition RPi
  • Now I have to upload pictures taken from camera to web app so Aria can access data to perform facial detection algorithm.
  • Pool data will now consist of fewer people and no longer include Angelo and Aria since Mary has RPi
  • Still may have some skewed statistics, meaning our accuracy, false positive rate, and false negative rate
  • May include family members in data pool

Angelo:

• Continued doing Front End for the home page of the web application.
• Updated Website:
  • Better Home Page
  • Links will send to sections for now and won’t send to new page.
• Updated Gantt Chart to fit into new virtual schedule
  • Angelo’s Projected Gantt Chart

Aria:

• Started following tutorial
• Thought about how to set up rpi
• Found problems with having to set up rpi
• Do not have all the parts necessary to set up rpi
• Have not had any problems with facial recognition so far
  • Aria’s Projected Gantt Chart

 

Angelo

• Kept working on the front end of the site after settling in during the “stay at home” period.
• Will continue working on the web application from home.

Mary

• Had some delays as I had to pack up my belongings and move back to NJ
• Will continue working on getting the camera to take pictures and store them in the raspberry pi

Aria

• Spent time with my family and did not work on this
• Refactored project to work with new guidelines
• Waiting for new Raspberry Pi to come so I can do facial recognition on it

Refocus SOW

Mary:
– RPi working!
– Connected RPi to CMU-DEVICE wifi
– Researching how to download OpenCV to SD card
– Researching camera set-up tutorials

Aria:

– wrote design report

– researched more on time limits of humans for abilities like recognizing a familiar face, etc

– looking into how to incorporate opencv with dlib and rpi

– trying to crop and transform images in dlib to give to opencv

— need 96×96 image, eyes and nose in the same place

Angelo:

• worked on Front End for the web application
• worked on Design Document
• talked to TechSpark about 3D printing