Carnegie Mellon ECE Capstone, Spring 2020: Ashika K, Abha A, Jade T
This week, I first worked on finalizing the Bill of Materials. I specc’d out the exact parts for the Text Display, Face Display, Motors, and the Motor Driver. The Text Display will be a 5in x 3in LCD screen, which connects to the Raspi through HDMI …
This week I worked on finding the components I need for the audio input, as well as choosing a speech recognition package and investigating text to speech packages. For audio input hardware I decided to go with a simple TRS to USB soundcard input into …
This week, I accomplished three main tasks. First, I found a story dataset for the templates: a collection of 177 of Aesop’s Fables. All these stories are about 5-7 sentences long, so they fit our desired template length and will be the only story dataset we will use throughout the project. Second, I downloaded the the NLTK package for part of speech tagging and synonym detection/recall as well as FitBERT (fill in the blank BERT), and I researched how to use both tools and their capabilities and limitations.
Once I had all these resources, I designed the machine learning model (shown above). I also created an example template to fit this model:
| Original | Template |
| A Nightingale sitting on the top of an oak, singing her evening song, was spied by a hungry Hawk, who swooped down and seized her. The frightened Nightingale prayed the Hawk to let her go.
“If you are hungry,” said she, “why not catch some large bird? I am not big enough for even a luncheon.” “Do you happen to see many large birds flying about?” said the Hawk. “You are the only bird I have seen to-day, and I should be foolish indeed to let you go for the sake of larger birds that are not in sight. A morsel is better than nothing.” |
A Nightingale sitting on the top of an **USER-NOUN** , singing her **fitBERT-Random** song, was spied by a Hawk that was **USER-ADJ**, who swooped down and seized her.
“If you are hungry,” said the Nightingale, “why not catch some **USER** bird? I am not **fitBERT-syn** enough for even a luncheon.” “Do you happen to see many **fitBERT-Syn** birds **USER-VERB** about?” said the Hawk. “You are the only bird I have seen today, and if I let you go for the sake of **fitBERT-syn** birds that are not in sight, I would be **USER-ADJ**. A **fitBERT-ant** bird is better than nothing.” |
I have added the details of this model to our design report (not yet finished).
I am on par with the planned schedule. Now that I have all the resources, I can start working on part of speech tagging for the user input and for template generation. Next week, I hope to have two or three example templates complete with parts of speech identified for any user or fitBERT inputs. I also hope to finish the machine learning part of the design presentation.
The dimensions of KATbot are as shown above. Risks and Risk Management: The most significant risk is a poor fill in the blank algorithm for the machine learning based stories. Without this feature, the stories will not be cohesive or grammatically correct, defeating the purpose …