Carnegie Mellon ECE Capstone, Spring 2020: Ashika K, Abha A, Jade T
We have no significant changes to our schedule and the risks of our project mostly have remained the same. This week, we worked on the design presentation and report, as well as our individual tasks. For the design submissions, we created a new system architecture …
This week, I focused on two aspects of the project. First, I finished up the storytelling portion of the design report. I already finished ironing out the details and conducting research on the metrics and validation prior to the design presentation, but I edited the …
The first thing I did this week was work with Jade to determine the communication protocols between all of the hardware in this project. After we determined them, I created the system architecture diagram for our design report/presentation and for future reference.
This week, my goal previously was to work on the displays and Raspberry Pis. However, since all of our parts hadn’t come in yet and we were having issues with setting up the Raspis, I pivoted my focus to designing the overall robot and robot arm. After I finished designing them, I started CADing the robot arm. Since I’m a bit rusty on mechanical design and CAD, this process has taken a bit more time than I anticipated.
I also worked on the design presentation and design report. I finished working on the design presentation and made significant progress on the report, which should reduce the amount of time I spend on it next week and let me focus on the actual project.
So far, I am on schedule. My goals for next week are to finish the design report, finish CADing the robot arm, and work on the displays/Raspi system. I would like to pivot back to the displays because they are more essential to the project than the robot arm.
This week I investigated many different text to speech packages. I evaluated TTS packages on ease of install and use, whether the TTS required internet connection, and the quality of the voice. A lot of the text to speech packages had robotic or choppy voices, …
We have no changes to the existing design, and are currently on schedule. We have been deep diving into figuring out how specifically we will be implementing our requirements as well as working on our design report and presentation. We finalized the communication protocols we …
This week, I started writing the code for story creation. I wrote two python programs: the first checks that the part of speech of a word matches the expected part of speech given a sentence for context. This will be used for error detection on the user side. The second algorithm takes in a story and a templated story and tags each blank with the corresponding part of speech and where the input is coming from (user or FitBERT). To do so, I also had to design a system for encoding the templates when choosing which words to remove:
Besides the code, I worked on the design presentation and the design report. I did more research into the metrics and testing methods we will be using, particularly for story cohesion. I am almost done with my part of the design report, so I will have a lot of time next week to work on synonym generation for FitBERT inputs. I also hope to work on template making as I go, so I can start testing these components. As of now, I am right on schedule.
This week, I first worked on finalizing the Bill of Materials. I specc’d out the exact parts for the Text Display, Face Display, Motors, and the Motor Driver. The Text Display will be a 5in x 3in LCD screen, which connects to the Raspi through HDMI …
This week I worked on finding the components I need for the audio input, as well as choosing a speech recognition package and investigating text to speech packages. For audio input hardware I decided to go with a simple TRS to USB soundcard input into …
This week, I accomplished three main tasks. First, I found a story dataset for the templates: a collection of 177 of Aesop’s Fables. All these stories are about 5-7 sentences long, so they fit our desired template length and will be the only story dataset we will use throughout the project. Second, I downloaded the the NLTK package for part of speech tagging and synonym detection/recall as well as FitBERT (fill in the blank BERT), and I researched how to use both tools and their capabilities and limitations.
Once I had all these resources, I designed the machine learning model (shown above). I also created an example template to fit this model:
| Original | Template |
| A Nightingale sitting on the top of an oak, singing her evening song, was spied by a hungry Hawk, who swooped down and seized her. The frightened Nightingale prayed the Hawk to let her go.
“If you are hungry,” said she, “why not catch some large bird? I am not big enough for even a luncheon.” “Do you happen to see many large birds flying about?” said the Hawk. “You are the only bird I have seen to-day, and I should be foolish indeed to let you go for the sake of larger birds that are not in sight. A morsel is better than nothing.” |
A Nightingale sitting on the top of an **USER-NOUN** , singing her **fitBERT-Random** song, was spied by a Hawk that was **USER-ADJ**, who swooped down and seized her.
“If you are hungry,” said the Nightingale, “why not catch some **USER** bird? I am not **fitBERT-syn** enough for even a luncheon.” “Do you happen to see many **fitBERT-Syn** birds **USER-VERB** about?” said the Hawk. “You are the only bird I have seen today, and if I let you go for the sake of **fitBERT-syn** birds that are not in sight, I would be **USER-ADJ**. A **fitBERT-ant** bird is better than nothing.” |
I have added the details of this model to our design report (not yet finished).
I am on par with the planned schedule. Now that I have all the resources, I can start working on part of speech tagging for the user input and for template generation. Next week, I hope to have two or three example templates complete with parts of speech identified for any user or fitBERT inputs. I also hope to finish the machine learning part of the design presentation.
The dimensions of KATbot are as shown above. Risks and Risk Management: The most significant risk is a poor fill in the blank algorithm for the machine learning based stories. Without this feature, the stories will not be cohesive or grammatically correct, defeating the purpose …