Posted on

Yun’s Status Report for 3/19

This week, I worked on constructing database and modifying ColorThief API appropriate for our project. In terms of database, I found a sample color coordinating chart(Figure 1) and converted into CV format with Microsoft Excel. As you can see from Figure 2, colors are denoted with their RGB values in an array in each cell. A color has 5-6 colors that match. In terms of the database, I am expecting that there should be an alternative data structure to store the color data more efficiently for the future computation and search, and am planning to figure it out in the following week.

Figure 1: Sample color coordinating chart
Figure 2: color database in CV

Another task I was able to complete was modifying ColorThief API more suitable for our project. The original ColorThief API is based on the web service which is unnecessarily complicated for ours as the project operates in local level. Thus, I modified some pieces of codes within the original API so that I can get the dominant color’s RGB value of an image file in a local machine all operating in local machine. In addition, the original API is in node.js and displays the results in stdout. However, we want to use the results in C++; thus, I added some features such as logging, and storing the results in a text file for each run. On top of that, the original API had an issue of unable to detect white as a dominant color. Thus, I fixed the error in different files within the API so that it is able to detect white as well. The following Figure 3 is a small code snippet for bring a local image and getting RGB color data, although there were additional modifications to operate this made in other files as well.

Figure 3: short code snippet

I think we are on track in terms of the schedule we proposed and also in terms of our short term goal which is to get basics working till the end of next week as we finished modular tasks working except for constructing the mirror frame. For next week, I am planning to connect Arducam OpenPose result with ColorThief so that we can get the torso and top color detection working. On top of that, it would be ideal if I could also get a very simple color-based outfit recommendations working by coming up with brief codes of finding an appropriate color based on the color coordinating chart database.

Posted on

Team Status Report for 3/19

Our team is planning to get the basic structure of the product working by the end of next week. Thus, this week’s short term goal was to complete modular tasks for constructing the mirror frame, get Arducam to work with OpenPose and operate video, develop basic structures of the App, constructing color coordinating database, and get ColorThief working in local machines. Everything except for the mirror frame construction has been completed. The delay in mirror frame construction was caused due to slow communication with the TechSpark faculties, and we solved it by reaching out to students working at TechSpark. In the end of this week, we have prepared woods that we can use for the frame and have communicated with the students at TechSpark. Thus, we are very optimistic about completing the mirror frame by the end of next week along with other components combined together.

There is no change into the design of the system. However, through meetings with the faculties, we are planning to add special features that will make our smart mirror to be distinguished from others such as scoring the user outfit and helping the user with packing for trips. These will take some time; however, modular tasks required for such additions are already being handled in our basic structures; thus, it will only be a matter of linking them slightly differently.

Some pictures of our progresses are shown in individual reports. We have a picture of Arducam working(Wonho’s), screenshots of the basics of App(Jeremy’s), and the RGB values result shown from ColorThief ran on sample images stored in local computer(Yun’s).

Posted on

Wonho’s Status Report for 3/19

Prior to spring break, I received the micro SD card from our order and configured it to run Ubuntu for our Jetson Xavier NX. I took the SD card and wrote the proper image on it, which was downloaded from the Nvidia website to help set up for the Jetson Xavier.  Once we got the camera set up and connected to the Xavier, I spent about a week on it to get live camera feed working. At first I could only get it to take pictures but them realized the display connection cable was the problem and after switching it to a HDMI cable we were able to get live video feed showing.

Once we got live video feed working, the next step was to get Open Pose working. We downloaded the correct files and tried to follow the instructions for setting up Open Pose. I spent time after class this week both on Monday and Wednesday to set it up but it seemed like the micro SD card we bought did not have enough storage. So I brought an 64GB (what we had originally was 16GB) micro SD card and configured it to work on our Xavier but this still didn’t seem to fix the issue of compiling the Open Pose file. Jeremy and I spent 3 hours on Thursday tries to install correct prerequisites and drivers but somehow the system cannot find/recognize the correct library. We hope to try it again on Monday but incase Open Pose does not work we have a secondary plan in place to try out. We found a program similar to OpenPose called trt-pose which is capable of detecting limbs and gestures real time just like OpenPose but just for one person. This should be sufficient enough for what we need since only one person will be standing in front of the mirror.

Jeremy and I also were able to find scrap wood that we can use for our project in the TechSpark woodshop. The wood pieces were the appropriate pieces for the frame we will be building for the mirror.

In terms of progress, I would say we are on track to have our project going. The next couple of week will really be the time where we need to grind out both the software and hardware components for our project but I’m confident we can get it done.

Live video feed from Arducam

Posted on

Jeremy’s Status Report for 3/19

Post-spring break, I dove straight into implementing our mobile app that the user would use to interact with our smart mirror as well as input his/her wardrobe into the database. I am building the app with React Native and testing it with Expo Go. I have attached a few photos below which show my progress with the app. As of now, I have a very bland and simple home screen for our app and a ‘+’ button in the top-right corner which navigates the user to the page where he/she can add clothes to the database. I’m still working on having the app be able to handle image and file uploads. After that is done, then I plan to make the app a little more pretty and then connect it with the database implementation that Yun is working on. Besides working on the app, I also helped Wonho set up OpenPose with NVIDIA Xavier. We spent 3 hours on Thursday night trying to install all the correct dependencies and builds, but there seems to be an issue with either caffe or cmake not being able to recognize one of the libraries needed to run OpenPose. Wonho and I will try again on Monday, but if it doesn’t work then he and I discussed a plan B which is to use a different software tool, capable of real-time gesture recognition, called trtpose which was developed by NVIDIA. I also went to the wood shop with Wonho and picked out some scrap wood planks we will use to build the frame of our smart mirror. Next week, I will finish the app and build the frame of our smart mirror. I would say that our team is slightly behind schedule because there have been so many issues with setting up OpenPose on the Xavier, but if that is resolved then we may even be ahead of schedule.

Running the server
Home page
Add clothing
Posted on

Yun’s Status Report for 2/26

This week, I mainly worked on revising design presentation design and details. We have finalized most details of our design plan reasonably. I have also started to look into open source APIs such as OpenPose and ColorThief in more details in order to brainstorm an integration of open sources and my team’s software.

We are a bit behind the schedule, but as most details for design are all set now, I am hoping that my team to finish most modular tasks such as recommendation algorithms and assembling mirror before spring break.

For the upcoming week, we will write a draft of our design document and finalize it. On top of that, we will be starting implementation for both software and hardware, but with a slightly more focus on software.

Posted on

Wonho’s Status Report for 2/26

This week I mainly worked on the slides for the design presentation as well as ordering everything we need for our project. As of now, everything is ordered and we just need to wait for the parts to arrive to start assembling the mirror. I also requested for space in TechSpark to assemble and build our smart mirror. The majority of our budget has gone to ordering the display for our mirror and the two way mirror itself but the remaining parts are cheap enough that budget should not be an issue. Continuing from last week, I have configured the Jetson Xavier to be ready to be setup as soon as our micro SD card arrives so we will able to extract information from the camera feed. This should happen in the next upcoming week as well as finishing my part of the design report. Next week, I hope to finish this part before spring break comes as well as start assembling parts of our smart mirror so we can test it out.

Posted on

Jeremy’s Status Report for 2/26

This week I was mainly working on brushing up on the details of the design review presentation. I also wrote down some responses to frequently asked questions that might have come up in our presentation. In doing so, our group now has pretty much solidified all implementation details and system specifications. I have also started to come up with specific algorithm implementations for our recommendation scheme. It has to include the weight of the clothing, the color, and the weather. I’ve been working on an equation that could use all three of these parameters and provide a correct/satisfying recommendation output.

We are currently slightly behind schedule, but we hope to nail down a lot of things right before spring break so that we can come back from break and get working at a relatively fast speed.

In the coming week, we plan to finish our design report and get it reviewed by a TA and then start working on building the mirror frame when the mirror arrives. We plant to do this at the woodshop or makerspace. I also hope to get a beta version of the app working.

Posted on

Wonho’s Status Report for 2/19

This week I mainly worked on figuring out getting the camera module and the Jetson Xavier board setup. I looked online to look at the SDK required to set up the SD card for the Xavier board as well as figuring out how to install the necessary drivers for the camera module. This part looks like there needs to be more research done on how to make it work so I’ll continue to do that as we get our micro-SD card delivered next week.

As for doing work with the team, I also worked on filling out the design review presentation slides and the bill of materials. I ended up putting in the order for major part such as the mirror and the monitor as well as some other little parts we need for the project.

Next week, I’ll hopefully get the camera up and running so we can start figuring out how to collect data from the camera input.

Posted on

Yun’s Status Report for 2/19

This week, we focused on finalizing project design details and putting them together to present it for Design review presentation. During the process, I was in charge of clarifying and rearranging user case of how each component and users will interact with one another. On top of that I did more research on which App development tool to utilize and analyzed pros and cons for app development tools.

As we continued to work on finalizing design details this week, we are behind of the schedule that we proposed during the proposal presentation. However, considering the fact that design review is scheduled to be next week, I do not think we are behind too much relative to the course schedule and time we have left. Thus, it seems necessary to adjust the proposed schedule so that we can work accordingly.

In the following week, I am planning to finish collecting basic color related database and develop structures of color recommendation software.

Posted on

Jeremy’s Status Report for 2/19

This week I mainly worked on the design review presentation slides and reviewing our system specification. I added more details to our block diagram that shows how each component of the project interacts with one another.  I also modified some of our use case requirements after considering the expectations of our users. We are currently on schedule but hopefully in the next week, we actually try and test out some of the hardware and software that we decided on.  Along with that, I hope to receive helpful feedback from my presentation that will help steer us in the right direction.