Team A3

Below is a link to our final writeup detailing everything that happened over the semester 🙂 Happy reading!

Final report paper

Team

Accomplishments:

• Completed both the 2D and 3D code for facial recognition
• Integrated fully with Emily’s part
• Co-wrote final report and helped design slides for Emily’s presentation
• Tested the facial recognition algorithm to make sure that it works
• Integrated power circuits with rest of hardware
• Finished putting everything onto the door

Upcoming Work:

The group will probably meet one last time to make sure everything works in our final integration, and we’ll make the door look a little prettier. Need to make the pamphlets as well.

Jason Huang

Accomplishments:

• Improvement of integrating Emily’s 3D data into my code
  
  Upcoming Work

Will continue working with 3D data, and hopefully making the 3D facial detection a lot better at detecting fake faces.

Schedule

Going to work during Thanksgiving to continue fine-tuning the code.

 

Emily

Finally fixed the pi/camera issue, so was able to get more data to Jason finally! Also fixed the picture taking script so that the camera had some time to settle before taking the person’s picture. Other work this week included taking a trip to Home Depot with the gang and buying a door and deadbolt, so I’ve been fine tuning the servos to that specific deadbolt. Next week I’ll be working with Jason to write some scripts to integrate all the pieces of code we’ve written so that things can be called sequentially following the order of our intended control flow.

Joe

I spent a lot of time in the lab this week. The first thing that I was able to accomplish was to increase the switching speed back to 100khz. I realized that the physical distance between the two mosfets was causing the gate signal to be slightly offset, resulting in some oscillations in the output. I moved the both right next to each other on the breadboard and it was better. I also attached some heat sinks to the mosfets just in case. On the actual power transfer side, I am looking into changing around the topology. (parallel-parallel vs series-series, etc). Something very strange that I noticed that the function generator was able to have a much greater efficiency in power transfer than the switching circuit I was using. This is leading me to suspect that there is something in the internal impedance of the function generator that is helping transfer more power, or that there is maybe some kind of signal noise that is reducing the efficiency for the switching circuit. If I can figure out how to get similar efficiency as the function generator on my power circuit, I should be able to send more than enough power, it is just a matter of figuring out why this is the case. I have read many papers on improving the efficiency of inductive power transfer, and I believe that I am hitting most of the important aspects. I strongly suspect that it may be a small issue that is causing low efficiency. I obtained several large capacitors and a voltage regulator so that shouldn’t be an issue once the power can be transferred.

Team

Went to Home Depot and bought a door, door handle, and deadbolt!

Jason Huang

Accomplishments:

• Started integrating Emily’s 3D data into my code
• Still hoping to fine-tune some of the other parts of my code; can partially detect 3D faces, not 100% successful
  
  Upcoming Work

Will continue working with 3D data, and hopefully making the 3D facial detection a lot better at detecting fake faces.

Schedule

Made decent progress, but hope to move forward more as we are headed towards the end of the semester. In the next few weeks, our team will come together and figure out how to integrate our three parts together.

 

Emily

Things were going ok but then they were not! I managed to get a bit of facial data over to Jason, but then for some unknown reason either the RPi or the camera/camera firmware stopped working (some uvcvideo error) and the Pi can no longer detect the camera when running the picture taking script even though it’s technically there when I run “usb-devices” 🙁 Have been struggling with this for a while, but nothing yet. Might have to re-install everything on the Pi and hope for the best 🙁

In happier news, the servos turn a deadbolt now! I’ve also been working on designing the outer enclosure so that all the buttons are accessible and the camera gets enough field of view.

 

Joe

This week I was able to build the switching circuit. Theoretically I should be getting good power, but in reality I’m getting around 1/10th of our required amount of power. The coils are also coupling very strangely, as they seem to couple more intensely when further apart (around 2 door widths is where they are optimally coupled). Getting 30V, 10ma at that point with a 9k Ohm load. Short circuit current is 300mA. Theoretically the optimal power should be 1/2 open circuit voltage * short circuit current, so should be getting enough power, but that is not the case! Unsure what is going on at this point, so will need lots of help next week!

Jason Huang

Accomplishments:

• Updated codebase to detect facial curvature and three-dimensionality; work-in-progress
• Emily’s camera sort of works now; using 3D data from the camera to update my code as well
  
  Upcoming Work

Still trying to integrate my code with Emily’s facial data, and hopefully that will be done next week, since she has a better understanding of how to use the camera.

Schedule

I had a midterm this week, and an essay due by the end of the week, so I’ve been working on that a lot, which meant I was unable to work on capstone too much. The whole team will be meeting up next week to combine some of our work together.

 

Emily Wong

This week has been busy, with midterms and big homework assignments. However, I was able to get the proper python libraries to build on my computer and put point cloud data to .ply file for now. I have been working on building the python bindings on the raspberry pi. It seems that the raspberry pi does not have enough memory to support the compiling with the desktop at the same time. Even then, it often runs out of memory. I configured a 1Gb swap file to finally make it compile. This was already after I had tried to cross-compile using an ubuntu vm, which ended up being a cmake file disaster. For the servos, I was able to move around some weak servos at home using the metro. The bigger servos just came and I will be testing those, and starting to design the housing.

I am slightly behind due to work from other classes, and I hope to make up ground by finishing up integration with Jason. We have several meetings planned to get the 3D face detection up and running so we can do testing. As for the servos, after I make sure that the higher torque servos will work, I can start planning out the enclosure so that they actuate the deadbolt, and then most of our project will be done.

 

Joe Zhao

This week I ordered a bunch of more parts. I should have all the components I need to finish the inductive power. It’s hard to judge how efficient the power transfer is, given that the function generator seems to supply different max currents for different frequencies. While I have been waiting for components to arrive, I have been trying to characterize the max power transfer, and depending on the efficiency of the DC adapter I bought from amazon, I should be able to achieve about 2x the power requirement I had set during the design phase. Another aspect I have been working on is to design the circuit for the 2.1 mhm 555 timer to output at 100khz pwm. I believe that my math is correct, and once the timer arrives I can build the circuit and verify via oscilliscope.

The goal for next week is to spend most of my time building circuits, and have significant testing and data on those power systems.

 

Whole Team:

Despite having a lot of other work this week, we think that we made a decent amount of progress. 2/3 subsystems are nearing completion, arguably the most important 2 subsystems (embedded stuff and software), so we think that we are on good pace to close out the project.

Jason Huang

Accomplishments:

• Obtained facial data via online (http://www.face-rec.org/databases/)
• Updated code that takes one photo and sees if it is registered; will be using this to demo next week
• Wrote an additional function that takes every single photo in a certain registered user’s photo directory, and average out the facial features to make the facial recognition more robust.

Upcoming Work

I want to integrate my code with Emily’s so that when a picture is taken, I can compare the photo with the database of registered faces and see if anything matches. I hope to also extend my code so that it detects depth, since right now I can take a printed-out picture of myself and still unlock the door, so I am trying to prevent that.

Schedule

I traveled last weekend so was unable to get as much work done. Still working on integrating code with Emily, but hope that will happen soon (not sure if it can happen before demo, but I have something ready for the demo at least).

 

Emily Wong

This week we finally got our original camera so we could finally do some interfacing with the Intel realsense sdk and the raspberry pi. All of my time spent this week was working on figuring out how to install all the libraries required to use the SDK, and figuring out how it works to get pointcloud and rgb data. So far, I’ve been able to get pointcloud data to be visualized through streaming it realtime to the pi. I was also able to get the RBG camera feed to sync with the pointcloud. However, I’ve been having trouble with the SDK getting the full pointcloud+rbg instead of it auto filtering out the background. I also need to figure out how to capture pictures instead of streaming video, and save those in formats that are useful to Jason. I think that I’m a bit behind since figuring out the SDK has been a pain, since there’s very little & bad documentation. Hopefully I should be able to figure it out on Sunday to get it working in time for the demo.

Joe Zhao

This week I got a lot of the parts for the inductive power. I spent the week assembling the coils and the rectifier. So far, it seems like there is a decent amount of voltage being matched on the receiving end. The only issue is that the amperage is kind of low. I am not 100% sure if this is due to the coil itself, or the fact that the function generating I’m using has a maximum current based on the voltage. I wanted to make the switching circuit, but it seems like I made a mistake when ordering the PMOS. The actual IC is way too small to be mounted on a breadboard, and also too small for me to try to solder wires to. I’m currently working on seeing if there are other options that I can do to connect to the NMOS, but it seems like I may need to order different PMOS. For next week, I hope to get all the MOSFETS so I can get the switching circuit, and then also the voltage regulator to get the correct amount of voltage to the RPI.

Jason Huang

Accomplishments:

• Obtained facial data via online (http://www.face-rec.org/databases/)
• Successfully implemented basic 2D facial recognition (no depth yet), based on dlib
• Wrote additional code to store registered users in a directory, and have the user’s face be compared with all the registered faces with the tested face

Upcoming Work

I will continue testing my new code with random facial data from online as well before the camera arrives. I hope that by the next few weeks, the code will also be able to detect 3D, so that someone can’t unlock a door by holding a picture of a face. This is to ensure that the actual human is in front of the camera to unlock the door.

Schedule

Made substantial progress this week, with the help of some online libraries and more free time. I am almost back on track, and hope to include depth in consideration of our facial recognition algorithm after our depth camera arrives.

 

Joe Zhao

This week I was able to talk to Professor Kelly, and work out the correct type of wire to use. Litz Wire would have too high of an impedance, so the best type of wire to use would be one with a higher gauge. We will compensate for the skin effect by reducing the frequency that we will be oscillating the coil at. The magnet, along with some other components for the front have been ordered, so I am now waiting for those. In the mean time, I helped Emily debug some issue with the bluetooth circuit, as she was also doing a lot with figuring out the camera issues.

For next week, I hope to have the parts so at least the coil will be completed, and I can make sure that it is able to handle the correct oscillation frequency and amperage. Then, I will also construct the class D switching circuit, and test that before adding it with the coil. Two coils will be made so I can measure the output resistance. I am slightly behind due to the coil wire problem, but I am looking to catch up this next week.

Emily Wong

This week we finally got our original camera (Intel SR305), and I can pick it up on Monday, so I’m excited to get some real camera data. The circuit is basically all set up, so next week will consist of getting the camera to interface with the RPi, which has already been flashed with Linux from last week, and figuring out how to make sure that none of the picture data gets lost in transit to the processing part. This week in particular was a bit slow because a lot of the groundwork for the front and back were already built, so it was just a bit of tweaking and making sure the correct data was being sent around when in different states of the control flow. I also spent a good amount of time designing an outer enclosure for both the back and the front. Haven’t had time to print anything out yet, but I will be consulting some mechE friends to see if my design makes sense before doing anything concrete. Joe and I also spec’ed out some motors and a door, so hopefully we can turn a deadbolt in the upcoming weeks!

 

Team

Things seem to be looking up, since we finally got our camera issues sorted out (hopefully once and for all!). Next week will consist mainly of Emily and Jason working together to merge code so picture data flows well between the two systems that were built so far. Since in-class demos are coming up, we can hopefully have at least the facial recognition working with the front/back circuits.

Jason Huang

Accomplishments:

• Obtained facial data via online (http://www.face-rec.org/databases/)
• Continued implementing Similarity Matching Schema, based on paper An Introduction to Biometric Recognition by Anil K. Jain, Arun Ross
• Started integrating facial data with my existing code

Upcoming Work

I want to continue playing around with the facial data that I’ve found online since we’re having more camera issues. Since most of our components have arrived, we can now progress forward at a quicker rate than before.

Schedule

This week also proved to be a busy week, with a few essays and a presentation due next week, but I was able to make a little more progress. With a second three-day weekend coming up, I hope to write more code to fine-tune my existing codebase.

 

Emily

This week was quite a busy one with midterms and papers, so I made a bit less progress than I had hoped. At the end of last week, we finally got what we thought were all the components we needed to finally interface with the depth camera that we have (Intel SR300), but yet another setback has occurred! Even though the Intel website said that the cable for USB communications was a USB 3.0 to Micro B, that was wrong! Instead it uses a special I-PEX to USB 3.0 connector that is REALLY hard to find online, so this poses quite a big problem… I’ve emailed our TA about our problem, so hopefully we can figure it out next week 🙁 In other progress, there has been more progress on the circuitry for communications between the front and the back. Button presses send the correct data/state to the other microcontroller, and the LED’s flash correctly! I’ve also made some very crude designs on how I want the outer enclosure to look. Preliminary talks with my other team members have ruled that we probably want to use some type of acrylic as material. We’ve also decided that we are probably going to laser cut it, I’m going to be taking the 1-unit mini to get certified to use the laser cutters in the TechSpark space 🙂

In terms of timeline, I’m S U P E R behind on getting camera data to Jason because the camera we have/Intel makes me 🙁 Other parts of the project are going smoothly, so while I’m making progress on the other parts that are in the future timeline, I’m really behind on that one part.

 

Joe

This week, I put in the order for the circuit parts I’ll need for the back part of the circuits, as well as starting to look into different batteries I’ll need to actually achieve the amount of power for 24 hours. On the actual construction of the inductors,  I ran into an issue that required more research. One was the ampacity of wires, the effective frequency that the wires can support, and the resistance of the wire. Since we were originally going to drive the inductor at 1mhz, we needed wire that can support that frequency, due to the skin effect. However, the gauge and strand count of Litz Wire to support that frequency is stated to be around 7.8k ohms/meter. On the other hand, using solid wire would require a much larger gauge, and also may be incompatible with higher frequency. I need to do more research on how the high resistance may impact the circuit, or reduce the clock speed.

My goal for next week is to get the magnet wire ordered, construct the rest of the circuit and see if I’m getting the proper output at the ends where the inductor will be attached.

 

Whole Team:

This week several of us were busy and we also found a couple of unexpected issues. This puts a bit behind, but we had some slack left in our Gannt chart to account for this. After clearing out the issues with Intel weirdness and inductor wire choice, we should be able to continue as planned.

Jason Huang

Accomplishments:

• Camera arrived, can start gathering facial data from it now
• Continued implementing Similarity Matching Schema, based on paper An Introduction to Biometric Recognition by Anil K. Jain, Arun Ross

Upcoming Work

Next week, I would like to finish implementing the similarity matching schema based on the algorithm provided by the paper. Since our camera has now arrived, I’m able to more easily test the existing code I have.

Schedule

I had a midterm this week, and was generally busier than the previous weeks, so I fell behind a little bit. I did not get as much work done as I wanted to, so hopefully next week I can shift my focus on 18-500 and catch up again.

Emily

This week I created the preliminary circuit to do Bluetooth transmission between the back and front including button input to start the recognition/registering processes. The cable for the camera came in on Friday, but I was gone for the weekend, so on Monday we should be able to get some real camera data! Things are on schedule, and I’m excited to try sending data to the pi for some processing next week 🙂

Joe

This week I worked on determining the values for the capacitors that would be necessary to provide sufficient peak power, and recharge in time for repeated use. From there, I also took a look at the appropriate voltage regulation and voltage to operate the coil at to maximize power utilization. This is a little behind schedule because of an onsite and midterm prep this week, but I hope to gain back ground next week. I want to get parts ordered this next week.

Whole Team

We are working to finish the design review paper and we are (mostly) on track. After doing the design review, we actually have a much better idea of what we want to accomplish and how to implement the rest. We will meet with the tech spark consultant tomorrow to see if we can leverage the tools there to help our progress.

Jason Huang

Accomplishments:

• Found additional sources for creating Face Curvature Maps and implementing Similarity Matching Schema
• Completed implementation of evaluating facial curvature maps
• Started on implementing Similarity Matching Schema, based on paper An Introduction to Biometric Recognition by Anil K. Jain, Arun Ross
• Created slides for Design Review presentation

Upcoming Work

Next week, I would like to finish implementing the similarity matching schema based on the algorithm provided by the paper. With our cameras arriving soon, I would also like to get started working on 3D facial pre-processing, so I can use the data points to create a facial curvature map and test different functions that I have written so far.

Schedule

I am currently on schedule. As stated from last week, there are four parts to the algorithm and I plan on tackling each section on a weekly basis. Instead of implementing Viola-Jones, I plan on using 3D data from the RGB camera instead, which eases the workload a bit.

 

Joe Zhao

This week I talked to Professor Kelly about the preliminary inductive power circuit. He advised to tune the circuit in series instead of parallel, in order to deliver greater current through the inductor. I spent the next few days tuning that circuit, but then we decided to move the raspberry pi to the front, increasing the amount of power needed. During the period where we were debating this issue, I designed the front power circuitry, as well as the backup battery circuit for the back. Parts will need to be spec’d again.

Last week my hope was to begin implementing the power system with parts ordered, but since there was a change in the location of our primary computation, I will need to redo the math in order to get the proper power delivery. In lieu of this, I decided to design the front power system, thereby still completing a weeks worth of work, just moving things around as we came to a decision. Now that we have decided on the location of the RPI, I hope to deliver a working power delivery system that can send 4W. This will be sufficient to power the rpi during idle and the camera. Then, during active power consumption, capacitors in parallel will be able to supply peak power during the 5secs of computation.

 

Emily Wong

This week I spent a good amount of time trying to figure out how to get our depth camera to talk with our microcontrollers so that we would be able to use Intel’s SDK to get RGB and depth pictures. There was a lot of conflicting information on whether the camera’s USB 3.0 was backwards compatible with USB 2.0, but I wasn’t able to test since the cable didn’t come with the camera.. 🙁 Other than that, I’ve started wiring things together (LED’s and buttons to microcontroller) to get the base of the front started.

Next week will be primarily spent on getting camera data so that Jason can finally base his algorithm on concrete data (which is what this week was for but there were a lot of problems [see above]). Once I get the cable, I’ll be able to test out getting data from the camera through the microcontroller, and since I’ve already been working on the base for the front, it shouldn’t be that bad to get back on track, since according to schedule I should be working on having the bluetooth modules send data (permit/deny) back and forth. 

 

Team

We worked on the design review slides this week, and had some discussions mostly based around the camera problems we were having. We found that the Raspberry Pi 4 has USB 3.0 ports, so we changed our design so that our computation for facial recognition was in the front rather than the back, meaning that the bluetooth should only be used for sending state/decisions for locking/unlocking. Other than that, everything seems to be mostly on track, and everyone has a good understanding of what everyone else is doing, so help is abundant 🙂