Category: Yun’s Status Reports

This week I worked on implementing the mirror UI, and its integration with the rest of the project. Last week, the mirror UI did not work on Jetson. Thus, I decided to create a virtual machine with the same exact environment as Jetson in my own laptop, and work on it there. This way, I was able to build a distributable form of mirror UI Electron app for Linux. However, although the app ran completely well on the virtual machine, it did not run on Jetson due to the fact that some requirements for running the app did not align with what are required for the rest of the project. Considering this, and also considering the fact that the mirror UI should be able to communicate with the mobile app, I decided to create a web app for the mirror UI. The reason we originally decided to make a desktop app instead of a web app was because OpenPose and color recognition already created too much loads on Jetson, which slows down the connectivity. However, as we tested with other web browser, OpenPose and color recognition we have now seemed compatible with a web app. This way, it became easier to connect the UI with our app as well. I have made a basic structure of web app that can be compatible with the mobile app, and will shortly test it with Jeremy’s mobile app as well.

Although we are behind the original schedule, I think we are approaching very close to the end of the project as we are wrapping up. I think we are in a good pace considering the deadlines for poster and final demo.

For the upcoming week, I will work on the integration briefly and final demo.

This week I worked on 1) running bash script with a click button in electron desktop app, 2) deploying the electron app in Jetson, and 3) deploying colorthief in Jetson.

I started working on 1) running bash script with a click button from my macOS environment; however, it did not work even after multiple trials. Thus, I decided to deploy electron app in Jetson and see how it goes. In terms of 2) deploying the app in Jetson, I was unable to install Electron in Jetson. Thus, I decided to build a distributable electron desktop app from my macOS laptop using electron-builder, and run the app from Jetson. Although I was able to successfully build and run a distributable desktop app for macOS and Windows, I was not able to run the app for Linux. I have not figured out this part yet. In terms of 3) deploying colorthief in Jetson, Wonho and I encountered “cannot find module” errors; thus, Wonho fixed it by using OpenCV for colorthief instead of node.js.

Overall, this week has been lots of trials and errors. We are behind the schedule as we have not been able to successfully integrate the mirrorUI desktop app with Jetson. As it is an important part of our project, if it does not work for another day or so, I am planning to develop mirrorUI using different platforms other than Electron.

For the upcoming week, I am planning to finish the integration of mirrorUI with Jetson and improve the mirrorUI so that it has better connections with the overall system.

This week, I touched a bit more upon the weather api as the one that I worked on last week was based on http request. It may cause unnecessary delay or complication in the future, so I worked on running everything completely locally, and was able to complete the task. On top of that I did more research on some basic recommendation algorithm, and also acquired some dataset regarding which outfits are appropriate for which weather.

I think we are bit behind of schedule, especially due to the unexpected delay in the integration of camera, jetson, and openpose. Two people are working on resolving that issue which delays the rest of the project. However, I am very optimistic about the rest of modular tasks as well, because I am almost done with the recommendation which means I can hop on the mobile app development and general integration as well.

Next week, I will work on interim demo presentation, and will work on integrating all modular tasks that my team has worked on so that we can have a demo. If I have some time left, then will work further on the basic recommendation system I have and improve it to be more sophisticated.

Since I worked on modular tasks related to color detection last week, I moved on and started to work on weather API. Using weatherapi.com, I was able to successfully run weather API locally using node js which is the same programming language used for color detection. On top of that, I also visited TechSpark, got some advice from Ryan, who is in charge of woodshop, and also arranged our wood work schedules with him. As a team, we all worked together at wood shop, building the frame holding mirror and monitor.

I think we are on schedule as the wood frame is done, and the rest of modular tasks are almost done as well. Next week, I am hoping to connect weather API and color detection API together to build a small system that recommends an outfit based on those information.

This week, I worked on constructing database and modifying ColorThief API appropriate for our project. In terms of database, I found a sample color coordinating chart(Figure 1) and converted into CV format with Microsoft Excel. As you can see from Figure 2, colors are denoted with their RGB values in an array in each cell. A color has 5-6 colors that match. In terms of the database, I am expecting that there should be an alternative data structure to store the color data more efficiently for the future computation and search, and am planning to figure it out in the following week.

Another task I was able to complete was modifying ColorThief API more suitable for our project. The original ColorThief API is based on the web service which is unnecessarily complicated for ours as the project operates in local level. Thus, I modified some pieces of codes within the original API so that I can get the dominant color’s RGB value of an image file in a local machine all operating in local machine. In addition, the original API is in node.js and displays the results in stdout. However, we want to use the results in C++; thus, I added some features such as logging, and storing the results in a text file for each run. On top of that, the original API had an issue of unable to detect white as a dominant color. Thus, I fixed the error in different files within the API so that it is able to detect white as well. The following Figure 3 is a small code snippet for bring a local image and getting RGB color data, although there were additional modifications to operate this made in other files as well.

I think we are on track in terms of the schedule we proposed and also in terms of our short term goal which is to get basics working till the end of next week as we finished modular tasks working except for constructing the mirror frame. For next week, I am planning to connect Arducam OpenPose result with ColorThief so that we can get the torso and top color detection working. On top of that, it would be ideal if I could also get a very simple color-based outfit recommendations working by coming up with brief codes of finding an appropriate color based on the color coordinating chart database.

This week, I mainly worked on revising design presentation design and details. We have finalized most details of our design plan reasonably. I have also started to look into open source APIs such as OpenPose and ColorThief in more details in order to brainstorm an integration of open sources and my team’s software.

We are a bit behind the schedule, but as most details for design are all set now, I am hoping that my team to finish most modular tasks such as recommendation algorithms and assembling mirror before spring break.

For the upcoming week, we will write a draft of our design document and finalize it. On top of that, we will be starting implementation for both software and hardware, but with a slightly more focus on software.

This week, we focused on finalizing project design details and putting them together to present it for Design review presentation. During the process, I was in charge of clarifying and rearranging user case of how each component and users will interact with one another. On top of that I did more research on which App development tool to utilize and analyzed pros and cons for app development tools.

As we continued to work on finalizing design details this week, we are behind of the schedule that we proposed during the proposal presentation. However, considering the fact that design review is scheduled to be next week, I do not think we are behind too much relative to the course schedule and time we have left. Thus, it seems necessary to adjust the proposed schedule so that we can work accordingly.

In the following week, I am planning to finish collecting basic color related database and develop structures of color recommendation software.

I personally worked on improving the proposal presentation slides, revising the schedule (Gantt chart), and doing more research on which software languages and tools to use for the implementations. We realized that each modular task was in larger pieces than we wanted; thus, I worked on making tasks more modular so that each task can be completed in about a week.

My team has been working on specifying details for design choices by considering which languages to use to meet latency requirements and how the user experience would be. I believe that the team is on track in accordance with the planned schedule now. However, it is early in the project, so I am planning to work ahead if time allows, so that we can have some buffer time in the future when unexpected situations that are not planned occur.

In the following week, I am planning to do some research on finding resources for outfit color matching, and ideate what kind of format we would want to store such data. In addition, if time allows, I will also be studying app development briefly.