Category: Team Status Reports

Our team made a lot of progress this week. We are getting very close to finishing up the integration process with the mirror UI, the Jetson, the app, and the outfit recommendation algorithm. Some risks that are being dealt with at the moment are incompatibility issues between the mirror UI code and the Jetson. The main reason for this is because of the OS differences. Packages and code that compiled and ran on the macOS are suddenly not working on the Jetson’s Linux OS. Our plan B in case this doesn’t work out is possibly to set up a virtual machine that would run the mirror UI on a macOS. This is our last resort because running on VMs may result in slower response times and potentially more bugs. Another risk that we face at the moment is being able to run the openpose shell script through a click of a button in our app. Considering the various roadblocks we have faced because of the Jetson, it may be very likely (hopefully not though) that the Jetson will have trouble opening and running the shell script when the app sends a request. Our contingency plan for this is to have to run the script manually, but we will do our best to avoid this at all costs because it significantly hinders the convenience of our smart mirror.

Some changes were made to the outfit recommendation schema. Originally, we were planning to recommend one outfit, but we changed the output to 5 outfits. We also added the option for users to input whether they want a formal outfit or an informal outfit because it enhances the purpose of our smart mirror and is more tailored towards a personal preference this way. This did not induce any costs but may potentially pose a challenge of having to send data about the formality to the Jetson.

Photos and videos of our progress will be reflected on the final presentation slides as well as our individual status reports.

As we are coming close to the final demo, our schedule has become a lot busier. Integration is a lot more difficult than expected, but most of us are staying in lab till night to get things sorted as fast as possible. As a result, we are doing testing and integration at the same time instead of separately. Other than that, I feel like our team has made a ton of progress this week and it will stay that way.

The most significant risk/task we are facing is the integration between app and Jetson, and Jetson and mirror UI. We are expecting that both integration would require creating server-client socket connections, and there is a no easy way out. However, Jeremy and Yun have some experience in this area, so we are hoping to get this task done if we push through.

In terms of the system design, we initially planned to connect the app with mirror UI as well; however, we decided to remove that interaction and use Jetson as the intermediate for the communication as we need to build connections of Jetson-mirror UI and Jetson-app anyway.

We do not have any change in the schedule from the last week. We set a small goal last week to finish the modular tasks by this Thursday and work on integration from Saturday, and we are on track for this schedule.

To explain in a bit more detail of what we did this week, we worked on completing all remaining modular tasks. The remaining tasks were 1) automating torso and top detection, 2) user wardrobe input form from the app, 3) color + weather recommendation, and 4) mirror UI.

• Wonho was able to write the automation script for Jetson which covers from spinning off Jetson to taking a screenshot using Arducam, running OpenPose, and to cropping the screenshot.
• Jeremy finished making the user wardrobe input form that asks apparel type, length, season. After the user submit the form, the data gets saved into google spreadsheet, which will be the user wardrobe database.
• Ramzi finished the color and weather based recommendation with Python. The color recommendation based on the survey he created on which colors go well with which, and the weather recommendation is based on the real-time feels-like temperature.
• Yun finished creating the mirror UI using Electron. It has main page, recommendation loading page, recommendation result page, and thank you/feedback page.

More details can be found on each of our individual status report. The above four tasks were all of the modular tasks remaining, which means all we actually need to do is integrating these parts.

 

The most significant risks that pose a threat to the overall completion of our project is the issues that we have been facing with OpenPose. There are a bunch of Github threads that have been made by others who have had problems with setting up OpenPose on their NVIDIA Jetsons yet the documentation is either outdated or incorrect depending on each person who had the issue. Our team faced the same challenges and therefore created a contingency plan where we will be using trtpose now. We plan to wipe the SD card that was in our NVIDIA Xavier and start from scratch by installing trtpose instead of OpenPose.

In terms of the mirror frame that we built, we made a few changes to our initial blueprint of what the frame would look like. Our most recent diagram shows the exact dimensions of each side of our frame and this is what we referred to when constructing the frame in the woodshop downstairs.

None of the scrap wood in the woodshop matched the dimensions of our mirror frame so we had to go to Home Depot to buy wood. This incurred an extra cost of around $50, but it is still well under our budget limit. Another extra component to our mirror design that we thought of after constructing the frame was how we would hold the acrylic two-way mirror in place. The mirror itself is quite bendy and malleable for some reason, so we decided that we would attach triangular pieces to each corner of the frame which the mirror could stick to without falling forwards. A picture is shown below of our finalized frame.

During the construction of the mirror, we got assistance from an instructor in Techspark who was generous enough to give us a few tips about woodworking as well. One obstacle was the fact that Home Depot claimed they sold this wood plank with dimensions of 2in x 10in x 8ft, but the height of the wood was actually shorter than 2 in which resulted in an unforeseen consequence of having to cut the wood again to match the design we wanted to in our original blueprint.

We moved the frame back to our bench, so now that the frame is done, all we need to do now is get the software working and running. We are on schedule in terms of what we have built so far, but we are behind schedule with testing out the software because it hasn’t been set up properly yet. That will be our group’s main focus for the coming week.

Our team is planning to get the basic structure of the product working by the end of next week. Thus, this week’s short term goal was to complete modular tasks for constructing the mirror frame, get Arducam to work with OpenPose and operate video, develop basic structures of the App, constructing color coordinating database, and get ColorThief working in local machines. Everything except for the mirror frame construction has been completed. The delay in mirror frame construction was caused due to slow communication with the TechSpark faculties, and we solved it by reaching out to students working at TechSpark. In the end of this week, we have prepared woods that we can use for the frame and have communicated with the students at TechSpark. Thus, we are very optimistic about completing the mirror frame by the end of next week along with other components combined together.

There is no change into the design of the system. However, through meetings with the faculties, we are planning to add special features that will make our smart mirror to be distinguished from others such as scoring the user outfit and helping the user with packing for trips. These will take some time; however, modular tasks required for such additions are already being handled in our basic structures; thus, it will only be a matter of linking them slightly differently.

Some pictures of our progresses are shown in individual reports. We have a picture of Arducam working(Wonho’s), screenshots of the basics of App(Jeremy’s), and the RGB values result shown from ColorThief ran on sample images stored in local computer(Yun’s).

The most significant risks that we came across over the past few weeks were figuring out the focus of our project and how exactly we would be able to recommend outfits to the user in a suitable fashion. We have managed both risks by centralizing our project on outfit recommendations rather than our original plan to include heart rate monitors and daily schedule reminders on our mirror. We have also decided to push through with the idea of having users take pictures of their own clothes and input it into a database which would then be used for outfit recommendations based on a color wheel. This makes the overall project much more straightforward and simple.

We have made some changes to our use-case requirements and system spec after our project proposal presentation. Using the students’ feedback, we decided to lower the time it takes for and outfit recommendation from 10s to 4s. This change was necessary and viable because we were successfully able to acquire the Jetson Xavier and because we realized that 10s may be too long to maintain the user’s attention. We have also decided to switch from using OpenCV with Python to C++. This change was necessary because our team realized that the documentation available is similar for both languages and C++ is much faster in terms of runtime which will directly help our use case requirement of fast outfit recommendations.

We have also started to think about how to connect our mobile app to the mirror for turning on/off the mirror and we realized that the Jetson Xavier that we received from the 18-500 inventory already has a built-in WIFI+BT chip that we can use which is very useful and saves us a lot of money and time.