Carnegie Mellon ECE Capstone, Spring 2020: Ashika K, Abha A, Jade T
This week, I worked on collecting data for story cohesion and user satisfaction. I had my family members fill in templated stories and then gave them to Ashika, who used them to determine story cohesion relative to random stories and the original Aesop’s fables. I …
This week, I mostly worked on assembling the robot. I also helped Jade collect data for speech recognition accuracy. First, I assembled the head and body of the robot. I put all of the electronics inside them and made sure everything still worked. One minor …
On Monday, I set up the face display and got code working on it to draw shapes and lines. This was a precursor so that I could debug issues with drawing shapes on a canvas before creating the actual face.
On Wednesday, I worked with Jade and Ashika to integrate Jade’s socket code with the Pi and hardware on my side, and debug issues we had with integrating it with the hardware. The socket code is used to transmit information between my laptop and the Pi.
On Friday, I worked with Ashika to integrate her code to bold text that are generated by KATbot (and not the template), and debug hardware issues with it, such as markup issues and testing to determine which sizes of canvas and text are appropriate for our displays.
On Saturday, I returned to the face display and worked on drawing the face. This is time consuming because there is a lot of experimentation involved with figuring out the desired size and which pixels to use for them.
In the upcoming week, I plan to finish the face display and assemble everything into the physical robot to finish the MVP. If I have time after that, I will work on the robot showing sentiment, but since that is outside of the MVP, it is not a focus right now.
On Sunday, I cut out the cardboard pieces for the head, and one of the arms.I also found a mini servo motor in my home, so I attached that to the arm. I soldered the motor shield that came in last week, and attached it …
On Sunday, I tried to debug the issues with showing an image on the eyes display through an SD card. Initially, my plan was to put an image of a cat eye on the SD card, and the eye display would read it. Most of …
What we did:
Abha worked on the face display on the hardware side, updated the CAD files to reflect the design changes for the head, and wrote code to move motors via a Raspi. Ashika wrote code to display the sentence output on a graphic screen (which will later be shown on the text display) and created templates. Jade finished getting the audio input and output working on the pi, and also started working on getting the ML algorithm running on the raspberry pi.
Major design changes:
When Abha worked on displaying eyes through the text displays, we realized that an important part of being able to show expressions is the way that the mouth is shaped (smile, frown, neutral, etc). With just the eyes changing shape, it is really hard to discern sentiments.
Since the goal of the eyes display was to be able to change the eye shape to show sentiments, we needed to change the scope of this aspect so that we could accomplish our goal of showing sentiments. Now, we will be having a full face display, that displays eyes, nose, and mouth. We will be able to update the eyes and mouth to better display sentiments.
Risk management:
The risks haven’t changed since last week.
On Monday, I fixed a couple issues with the CAD of the body frame. I’ve also started the CAD for the head frame and figuring out how to redo the arms now that I don’t have access to a 3d printer. Initially, my plans involved …
The first part of the week was spent on figuring out how to deal with the coronavirus changes, since that majorly affected my part of the project. Since things were changing every day (wrt CMU, Techspark, where I would be, etc), it took me a …
We have no changes to the existing design, and are currently on schedule.
First, we worked together to finish the design report. Then, we split off to work on our individual tasks. Ashika started the user interface program, which integrates user error detection, synonym generation, and FitBERT. Jade wrote a program a simple storytelling interface that uses our templates and prompts for user input. Abha worked on setting up a Raspberry Pi and getting the text display to work with it. Abha also had a meeting with a TechSpark representative, who gave useful feedback on design iteration for rapid prototyping.
Currently the most significant risks remain the same as what they were last week. One potential risk is that self-written pitch shifting might take too long to run and increase our latency. If that is the case, we’ll probably use built-in functions.
The first part of my week was spent working on the design report. After that, since the SD cards had come in, I set up a Raspberry Pi with Jade’s help. Then I worked on setting up the display for the text. Although I didn’t …