Posted on

Mahlet’s Status Report for 11/30/2024

As we approach the final presentation of our project, my main focus has been preparing for the presentation, as I will be presenting the coming week. 

In addition to this, I have assembled the robot’s body, and made necessary modifications to the body to make sure every component is placed correctly. Below are a few pictures of the changes so far. 

I have modified the robot’s face so that it can encase the display screen. Previously, the head was a solid box. The servo to head mount is now properly assembled. The head is well balanced using the stand I used to mount the motor to. This way there is space to place the Arduino, speaker and RaspberryPi accordingly. I have also mounted the microphones to the corners as desired. 

Before picture: 

After picture: 

Mounted microphones on to the robot’s body

Assembled Body of the robot

Assembled body of the robot including the display screen

 

I have been able to detect a clap cue using the microphone, by identifying the threshold of a loud enough clap detectable by the microphone. I do this processing in the raspberry pi, and once the RPi detects the clap, it runs the signal through the direction estimate function, which spits out the angle. This angle is then sent to the Arduino to modify the motor to turn the robot’s head. Due to the late arrival of our motor parts, I haven’t been able to test the integration of the motor with the audio input.  This put me a little behind, but using the slack time we allocated, I plan to finalize this portion of the project within the coming week.

Another thing I worked on is implementing the software aspect of the RPS game, and once the keypad inputs are appropriately detected, I will meet with Jeffrey to integrate these two functionalities. 

I briefly worked with Shannon to make sure the audio output for the TTS through the speaker attached to the RPi works properly. 

 

Next week: 

  1. Finalize the integration and testing of audio detection + motor rotation
  2. Finalize the RPS game with keypad inputs by meeting with the team. 
  3. Finalize the overall integration of our system with the team. 

Some new things I learned during this capstone project is how to use serial communication between Arduino and a raspberry pi. I used some online Arduino resources that clearly teach how to do this. I also learned how to perform signal analysis on audio inputs to localize the source of a sound within a range. I learned how to use the concept of time difference of arrival to get my system working. I used some online resources about signal processing, and by discussed with my professors to clarify any misunderstandings I had towards my approach. I also learned from online resources, Shannon and Jeffrey how a WebSocket works. Even though my focus was not really on the web app to RPi communication, it was good learning how their systems work.

Posted on

Mahlet’s Status Report for 11/02/2024

This week, I worked on the robot base structure building. Based on the CAD drawing we did earlier in the semester, I generated parts for the robot base and head that have finger edge joints. This allows for easy assembly. This way we can disassemble the box to modify the parts on the inside, and easily reassemble it back. The box looks as follows: 

During this process, I used the 1/8th inch hardwood boards we purchased and cut out every part of the body. The head and the body are separate, as they will be connected with a rod to allow for easy rotation and translational motion. This rod will be mounted to the servo motor.  As a reminder, the CAD drawing looks as follows.  

I laser cut the boxes and assembled each part separately. Inside of the box, we will be placing the motors, RPi, and speakers. The wiring of the buttons will also be placed in the body of the robot. The  results are as follows. The “feet” of the robot will be key inputs, which haven’t been delivered yet. The result so far look as follows: 

       

 

In addition to these, I worked on the TTS functionality with Shannon. I did some tests and found that the Pyttsx3 library works when running text input iterations outside of the webapp. The functionality we are testing is integrating the text input directly into the text to speech engine. This kept causing the loop error. When I tested the pyttsx3 in a separate file where I pass in various texts back to back by only initializing the engine once, it works as expected. 

We also worked on the gTTS library. The way this works is, it generates an MP3 file for the text file input and then reads that out once it’s done. This file generation causes a very high latency. For a thousand words, it takes over 30 seconds to generate the file. Despite this, we came up with plans to break up the file into multiple chunks and create the MP3 files in parallel, lowering the latency. This would get us to a faster TTS time, without having any issues similar to the pyttsx3 library. This is a better and fully functional alternative from our options, with a reasonable tradeoff of having slightly longer latency for longer texts for a reliable TTS machine.

In the coming week, I will be working mainly on finalizing the audio triangulation along with some testing, and begin integrating systems the servo system with the audio response with Jeffrey.