- This week, I spent a lot of time finalizing the final project presentation slides, as well as help my teammate Doreen prepare for her presentation.
- In the previous week, I finished the essential hardware components of the project so I moved over to the software side to help Surafel implement some aspects of the software. The team found a new Python library called facial_recognition that seems to have a much higher accuracy than our current code. Surafel worked on initial testing of the library and I worked on adding code to check users in and saving their face if they were not in the system. I also implemented checkout and the deletion of their face after checkout.
- Afterwards, I worked on the CSS of the web application, cleaning it up for a smoother looking final interface.
- I am slightly ahead of schedule for my own personal work of working on the hardware components. But, I am now pivoting to help with software. I would like to start completely wrapping the entire project up within a day or two, but I am not completely sure if its possible because we might need 3-4 days for finalizing software. The base functionality of the project is finished and I think most of the stuff we are doing now is extra (such as a possible buzzer on the hardware side, or user check-in/check-out logs with manual checkout on the software side). At this point a lot of work will have to be put in to make sure it gets done on time!
- Next week I hope to clean up the entire project, finish the final deliverables like the video, and poster, and be done with the semester (and my time at CMU!)
Team Status Report for 4/20/24
- There are a couple of significant risks that could jeopardize the success of the project. One risk is the stability of our hardware components. Through our long testing, we discovered that our NRF24L01 wireless transceivers were not reliable 100% of the time. A majority of the time transmission from the web app side of the system to the hardware side of the system worked, and other times the signal don’t go through, which definitely is not ideal when the system hinges on the transceivers to be the bridge between our 2 major subsystems.
- As a contingency plan, we purchased extra NRF24L01 transceivers that can be swapped out if something is wrong with the system. As a last resort, our web app keeps track of item stand transactions, which allows the customer to easily find user items (which is just a normal coat checking system).
- The second risk is facial recognition. This is also another very important component to our system. Without very high accuracy facial recognition, the system could prove to be frustrating to users who may not be able to retrieve their items easily. It is also a very hard thing to perfect and there is not much time left.
- As a contingency plan for the facial recognition system, we can allow the user to enter a name or identifier in place of scanning their face. Any other kind of identification method could work.
- Through our end to end testing, we discovered that some of our design requirements were not realistic. For example, we initially required that once a user’s face was matched, the stand should display the user’s position on the rack with an LED within 1 second. This did not account for the time it would take for the motor to rotate from it’s current position to the user’s position after they had checked in. Therefore, we changed our requirement to 7 seconds to allow the motor to rotate the the target position.
- Overall, there is not much change to the schedule.
Ryan’s Status Report for 4/20/24
- The past 2 weeks, I have been finalizing the hardware for the item stand and working on the final presentation. I have also been conducting some end to end testing.
- One main challenge I faced was the breaking of our system upon the addition of a new buzzer. We decided to add a new feature to the system by adding a buzzer and writing code that would allow the buzzer and our red LED to flash when an unauthorized change of weight was detected. Upon adding this functionality though, our wireless transceivers stopped working, essentially halting the functionality and progress of our project. Doreen and I spent several long days attempting to fix this problem and eventually managed to restore the item stand back to a previous state. Unfortunately we do not believe we have the time to implement this new buzzer feature without risking our system breaking again.
- This week, I worked on the final presentation and running some testing on the hardware. I conducted some item stand integrity tests, which consisted of adding weights to each of the load cells, adding weights on the item stand to make it imbalanced, and adding many weights to test the max weight bearing capacity of the item stand.
- Currently, I believe I am right on schedule. There were some hiccups in the past 2 weeks but I believe the preallocated slack time made up for that. I plan to continue testing, but start to focus my work on helping improve the software portion of the project.
- Next week, I hope to finish testing and wrapping up the project, as well as start on the final report.
WHAT I HAVE LEARNED THIS SEMESTER:
- This semester, I learned a lot about every aspect of the project I am doing, especially on the hardware end. Even in the construction of the item stand, I learned how to use a jigsaw, hammer and chisel, a sander, various drills, and other wood shop tools. It was also my first time assembling a large wood item and spray painting. I learned most of these things through online searching and lessons provided by some wood shop employees.
- On the electronics side, I learned a lot about the various components that went into the system. I learned about load cells, their calibrations and how they measure, I learned about how to use an Arduino microcontroller, I learned how to use wireless transceivers, I learned how to choose the right stepper motor for my project and which components are enough to power it, and more. Most of the knowledge I got to use these components came from the many tutorials I found online.
- Lastly, I dabbled in a bit of software. Before this class, the last time I touched Arduino code was in 18-100, which I took in my first semester at CMU (talk about full circle). I also spent a lot of time with various Arduino libraries, which I learned about through their documentation and other online resources. I learned a bit about how facial recognition and what algorithms go into it from talking to Surafel.
Team Status Report for 4/6/2024
RISKS AND MITIGATION
After testing most of our components and the entire system this week, we determined 2 big risks that could jeopardize the success of the project. One risk is wireless communication. In our system, we leverage NRF24L01 wireless transceivers to communicate wirelessly between our facial recognition system and item stand. Sometimes the communication is inconsistent. Data could be send from one transceiver but not received by another. This may be because we are toggling both transceivers between a receive and transmit state. We are planning on testing this more next week. Currently, we put a big delay between toggling the receive and transmit state, so the component has enough time to adapt.
The second risk is the accuracy of the facial recognition system. The facial recognition system is not as accurate as we would like at this moment in time. This is a big deal in our system because inaccurate face detection could result in the wrong user receiving their items. We plan to continue testing and refining the algorithm, especially since the team is wrapping up on the hardware component of the project.
DESIGN CHANGES
From the stand integrity part of our design requirements, we stated that each load cell should be able to hold 25 pounds. However, we have realized that the load cell we have purchased can only detect up to 22 pounds in weight. Additionally, when doing end to end testing this past week with items of various weights, the heaviest item that was placed on the rack was only 10-15 pounds. As a result, we are decreasing the maximum weight for each load cell from 25 pounds to 20 pounds. This would mean the the maximum total weight that can ever be on the item stand at once is 120 pounds. This modification does not incur any costs. It is simply more realistic with the weights for the types of items we expect users to place on the rack. For the other design requirements (detect an item has been added or removed within 1 second, 95% accuracy for facial recognition, etc), we will do further testing and make necessary modifications to attempt to satisfy them.
SCHEDULE
The schedule remains the same. However, we are utilizing some time in our slack time to fix any unforeseen issues and ensure our use case and design requirements are meant. Not considering these minor modifications, all members have completed their required tasks, and what remains is further testing.
VERIFICATION/VALIDATION
As a team, we plan to combine all the components into one system before running our tests. For facial recognition, we want to mostly implement the tests we described during the design proposal. We will test with users standing at various distances from the camera, with the ideal result being that users can only interact with the system within 0.5 meters. We also want to maintain accuracy, so we are planning to test our facial recognition accuracy with at least 20 different faces, with an ideal result of at least 95% accuracy. To test the integration between our hardware and software components, we plan to run tests that model real life scenarios, in addition to the specific tests we listed in our design proposal. For example, we can have several people check in and out of the system in various orders in order to determine if the system is able to accurately keep track of user belongings. For the item stand, we will be using various weights and seeing if the item stand is damaged or can rotate. We will also be introducing weight imbalance- putting a very heavy weight one side and rotating the item stand. Lastly, we want to time how long the process of checking-in our checking-out takes. We want to ensure that the motor can rotate the the correct position and that updates between the facial recognition system and item stand do not take long. This will mean making modification to our code to lessen certain delays or increasing the acceleration/speed of the motor.
Ryan’s Status Report for 4/6/2024
THIS WEEK
- This week, the NEMA 34 stepper motor arrived, along with a very larger motor driver and a AC to DC 6A power adapter. The team and I tested these components to our satisfaction and determined it provided enough torque to handle our applications.
- The item stand did not have enough space to accommodate a large motor, so I worked on cutting a large hole in the structure so that the motor can sit in it. I also modified a gear so that it could attach to the larger drive shaft of the motor. We combined all the electronics together for some integration testing, which used the facial recognition software to directly control the item stand. After that was done, Doreen and I worked on code to turn the motor to a specific position, accounting for the gear ratio between the motor shaft and the item stand.
- Next, we installed 2 LEDs to display information to the user- a blinking yellow LED to show the user which position to place their item, and a red LED to tell the user that their either checked in or checked out their item too late and would have to retry again.
NEXT WEEK
- Just like last week, I am still slightly behind schedule, though I have caught up more since last week. As a team, we should be approaching the end of end-to-end testing, but we are just getting our system cleaned up and finished for the start of testing. Again, the slack time we allocated perfectly covers this case. There is not much work left to be done on the project.
- Next week, I plan to install all the components permanently and help Surafel with facial recognition speed and accuracy. After that, we can perform testing and wrap up the project.
SUBSYSTEM VERIFICATION
- The main subsystem I have been working on this semester is the hardware item stand, which in itself contains several smaller components. For example, the load cell system polls on weight change, the item stand was built to support high load, and the motor was chosen to be able to move heavy load. The use case requirement that relates directly and solely to the item stand is its hardware integrity (how much weight it can handle) and how fast the item stand can detect user interaction (placing vs withdrawing an item).
- I have already somewhat tested putting weights on the item stand, but not anywhere near our original 150 pounds. I also have not tested increasing the weights or introducing load imbalance yet. After the system is done, I plan to get objects of varying weights and test item stand integrity.
- The second system that relates to a design/use case requirement is the detection of user interaction. I have already tested this subsystem a while ago, by removing and adding random objects to the load cells and detecting when the load cells see the change. It takes less than 1 second for the load cells to detected weight changes.
- Overall, I plan to implement the tests described in the design proposal related to the hardware item stand and see if they hit the benchmarks we prescribed.
Ryan’s Status Report for 3/30/24
THIS WEEK
This week, I completed the testing of the hardware components. Unfortunately, our previously single stepper motor did not seem to provide enough torque for our applications. I laser cut a new gear for use on the second backup stepper motor we had. Both motors together still did not provide enough torque. Going back to the drawing board, I calculated the torque ratings we needed and got the necessary components list we needed for a new stepper motor, the NEMA 34. We put that order in an are awaiting its arrival.
Next, Doreen and I implemented back and forth transmission between the two Arduinos and completed all of the code necessary for the item stand/rack to function. For example, upon the receipt of a check-in message, the Arduino Mega on the item stand will run a rack balance algorithm, determine the best location to place the item, rotate to the location, and poll on the weight sensor until it has detected a user placing their item, then sending back a success or fail message to the other Arduino.
Afterwards, we integrated the facial recognition code with our Arduino code by reading and writing to serial from the Python and Arduino code. Recently, we tried scanning our faces and it successfully sends check out/in instructions to an Arduino plugged in to the computer from the computer. The Arduino connected to the computer then transmits wirelessly to the Arduino on the item stand and the instruction is processed.
NEXT WEEK
Though I have done a lot of work this week, I am still slightly behind schedule. At this point in the schedule, I am supposed to be running end to end testing. This is because of some issues such as the motor still not being strong enough. Right now is a waiting game, for the new motor to come in and for facial recognition to be completely fleshed out. Next week, I plan to dedicate extra hours to wrap up the project, testing and installing the new (and much more powerful) motor, and helping on the web app side to make sure everything is integrated together well. I allocated slack time when I built the schedule so it is expected that the end of the semester might not fit the schedule completely.
Team Status Report for 3/23/24
We are entering the phase of the project requiring testing of all individual components and full integration into a unified system. Currently, we are in the process of merging all hardware components and developing Arduino code to manage their operation. Additionally, during motor testing this week, we found that a single NEMA 17 motor may not deliver sufficient torque for our intended application. Consequently, we have decided to utilize two of them. In the event of further difficulties, we may need to procure a larger stepper motor. Our contingency plans also include using different motor drivers to provide more current to the motor, helping to create more torque.
There has been a slight alteration in our design requirements as we conduct motor tests. Considering safety concerns and feasibility, we are contemplating a reduction in the maximum weight permitted on the rack hooks. Previously, we established the maximum weight for each hook at 25 lbs, but we now believe that reducing it to 15 lbs would be more feasible. This adjustment would still enable users to place large items while also improving the rotation of the rack. In addition to this, it may be necessary for us to add an additional motor to the system to provide more torque, enabling smoother and sturdier rotation. If this additional theory of adding an additional motor works, then we may again consider if the previous maximum weight of 25 lbs is actually feasible.
Our schedule has stayed the same, as it currently states that we should be doing integration testing between the hardware and software component, which have have begun this week.
We have accomplished various tasks this week. Firstly, we have finally been able to control the motor, setting the angle and speed at which it moves. We utilized the DRV8825 to drive the motor initially, but realized that the L298n motor driver would be able to provide more current, and since have switched to using this method method instead. Secondly, we have been able to test wireless transmission by passing a message from one Arduino to another Arduino’s serial monitor. Lastly, we calibrated our 6 load cells and are now able to read any weight placed on one of the hooks in pounds.
On the software side of things, the main improvement this week was tuning the training model to produce better results. After doing more research and looking over the codebase, we realized that the hyperplane that was trying to be found (the “kernal” being used) was linear. Since the data is not going to be linearly separable, we changed the kernal being used to a non-linear one, specifically the Gaussian kernal. Now that there are more parameters to worry about with the Gaussian kernal, we now also do a randomized search over a large range for each parameter to find the optimal parameters each time we train the model.
Ryan’s Status Report for 3/23/24
- This week, I worked on testing more hardware components. The new motor driver was delivered, but broke, prompting us to wait for a replacement for a couple of days. After the replacement came in, we tested it with the stepper motor and rotation works (we have some concerns about the strength of the motor but further testing is required).
- I also tested the NRF24L01 wireless transceivers, and was able to get the transmission of strings to work, which will form the basis of our wireless commands.
- After the rest of the components were tested, I attached all the load cells permanently to the stand. Next, I started on the final Arduino code that would be needed to parse wireless commands and control components on the item stand.
- According to the gantt charge, I am slightly behind schedule, due to my underestimation of the complexity of getting some components to work. The entire next week was originally dedicated to end to end testing, but at least half of that time will be working on the final Arduino code and testing motor strength with weights. I have been putting about 15+ hours of work into the project each week since the start so I believe I will catch up by next week (the gantt chart was a bit optimistic for the stage of “putting everything together”).
- Next week, I plan to test out motor strength and rotation, and finish writing the final code for the Arduino before we screw the last few bolts in.
Ryan’s Status Report for 3/16/24
THIS WEEK
- This week on the project, Doreen and I finalized a big part of the hardware components and conducted some continued testing on some electronic components.
- First of all, I cleaned up the wood for the item stand with wood putty and sanding. Then we spray painted and sealed the wood. Next, we assembled the bottom of the item stand.
- Next, I tested the electrical components, spending several hours troubleshooting the stepper motor (it kept vibrating instead of rotating), and eventually accidentally lighting a motor driver a friend gave me on fire. I ordered a different stepper motor driver that was recommended online.
- I also tested connecting all 6 load cells concurrently, and the test was a success. I then taped up the electrical connections and connected wires to the slip ring, which is inserted into the 6 sided top rack.
- Currently, I am still on schedule, as this week and next I am supposed to attach electronics to the item stand.
- The progress made can be seen in the team status report!
NEXT WEEK
- Next week, I hope to fix the issues with the stepper motor with the new motor driver that will be delivered. Then, I want to test out the wireless transceiver and begin writing item stand controller code so that the hardware component of this project can wrap up.
Team Status Report for 3/16/24
- The most significant risk we face in regards to the hardware component of our project involves the rotation mechanism for the hardware. We have faced some challenges regarding faulty components in the past week that has delayed the testing for the motor. We found out from testing our motor that the motor driver may not provide enough voltage to drive the motor, causing the motor to only vibrate, rather than fully rotate. As a result, we have ordered a new motor driver and asked a peer to use theirs so that in the following week we can get back on track with testing the component and fully assembling the rack. In addition to this, we need to complete testing our wireless transceiver.
- The system requirements have stayed the same. Similarly, our schedule has is the same as previous weeks.
- For the software, there is now a way to register faces! While the system is running, it is keeping track of the most recent frames it has read. When a new face is detected, it will save those frames in the system and then retrain the recognition model with those new frames included. Works ok for now, and it will hopefully work better after tuning the parameters of the recognition model to get better results.
- This week, we painted and finished the bottom portion of the stand. As for the six-pronged component at the top, we attached a slip ring and load cells and further tested the wiring for the load cells to ensure that they could capture weight changes. Here are images depicting our current progress with the hardware component.
