Posted on

Mahlet’s Status Report for 10/12/2024

This week, I focused mainly on the design report aspect. After my team and I had a meeting, we split up the different sections of the report fairly, and proceeded to work on the deliverables. 

I worked mainly on the audio triangulation, robot neck motion(components included) and the robot base design. In the design report, I worked on the use-case requirements of the audio response and the robot base. I made the final block diagram for our project. After this, I worked on the design requirements for the robot’s dimensions, and the audio cue response mechanism. After identifying these, I worked on the essential tradeoffs for choosing to use some of our components such as the Raspberry Pi, the servo response, choice of material for our robot’s body, and microphone for audio input. After this, I worked on the system implementation for the audio cue response and unit and integration testing of all these components. 

Following our discussion, I finalized the bill of materials, and provided risk mitigation plans for our systems and components. 

In addition to this, I was able to spend some time discussing the audio response methodology and approach with Professor Bain. After implementing the forward audio detection system (i.e. knowing the location of the audio source and the location of the microphone receivers), my goal was to work backwards, without knowing the location of the audio source. From this meeting, and further research, I concluded my approach as follows, and will be working on it in the coming week. More detail on this implementation can be found on the design report.

The system detects a double clap by continuously recording and analyzing audio from each microphone in 1.5-second segments. It focuses on a 1-second window, dividing it into two halves and performing a correlation to identify two distinct claps with similar intensities. A bandpass filter (2.2 kHz to 2.8 kHz) is applied to eliminate background noise, and audio is processed in 100ms intervals.

Once a double clap is detected, the system calculates the time difference of arrival (TDOA) between microphones using cross-correlation. With four microphones, it computes six time differences to triangulate the sound direction. The detection range is limited to 3 feet, ensuring the robot only responds to nearby sounds. The microphones are synchronized through a shared clock, enabling accurate TDOA calculations, allowing the robot to turn its head toward the detected clap.

I am a little behind on schedule as parts are not here yet. I will be working on the Robot base building with Jeffrey once that is complete, and do testing on the audio triangulation with microphones and RPi after performing the necessary preparations within the coming week. 

Posted on

Mahlet’s Status Report for 10/05/2024

This week, my primary focus was gathering data for the design report and completing tasks related to audio localization.

For the audio localization, I used MATLAB to simulate and pinpoint an audio source on a randomly generated 10×10 grid. I arranged the microphones in a square (2×2) configuration and randomized the location of the audio source. By calculating the distance between each microphone and the audio source, and considering the speed of sound (approximately 343 m/s), I determined the time delays relative to each microphone.

I applied the Time Difference of Arrival (TDOA) method. For each pair of microphones, the difference in the time it takes for sound to reach each microphone forms a hyperboloid. I repeated this process for every microphone pair, and the intersection of these hyperboloids provided a reasonable estimate of the audio source’s location. In MATLAB, I looped over the grid and computed the integer intersections of various locations. Using the Euclidean approach, I predicted the distance and calculated the corresponding TDOA using the speed of sound. By comparing the predicted TDOA with the actual time delays, I tried to estimate the error in the localization process.

The following figure illustrates the results, where ‘X’ represents the audio source, and ‘O’ marks the microphone positions. Additionally, I will include the relevant equations that informed this approach.

 

Currently, I am facing an issue with pinpointing the exact location of the source. To address this, I plan to refine the grid resolution by using smaller iterations, which should allow for greater accuracy. I will also calculate and display the approximate error in the final results. So far, I have a general idea of the audio source’s location, as indicated by a dark blue line, and I will continue working to pinpoint the exact position. Once I achieve this, I will conduct further simulations and eventually test the system using physical microphones, which will introduce additional challenges.

I am slightly behind on the project schedule. By next week I aim to finalize the audio localization section of the design report, along with the remaining parts of the report, in collaboration with my team.  I had a goal to setup the robot neck rotation servos by this week. This hasn’t been done as well. We will be finalizing the bill of materials by this week. I will be working on this as early as the components get in. To make up for this, I will be spending some time over fall break working on this.

According to the Gantt chart, Jeffrey and I had planned on building the robot, by the end of this week. This hasn’t been completed yet, but the CAD design is already completed. This week we will meet and discuss more about space constraints and make decisions accordingly.

Posted on

Mahlet’s Status Report for 9/28/2024

This week, I worked on the CAD design of our Studybuddy robot using Solidworks. After discussing with my team about the space constraints and objects we will need to integrate in the robot, we decided on general dimensions that give optimal dimensions. The base box is 8in x 7in x 6in. The head is 6in x 6in x 5in. The DCI display screen will be attached to the head in the designated extrusion as shown in the CAD drawing. The legs will contain buttons to power on the robot, pause and continue timers if necessary and buttons to interactively play rock paper scissors. Output of the buttons will be displayed on the DCI display from both players.

Considering the fact that the directional microphones are not reliable to pinpoint the exact direction of sound, I would have to start by creating simulations to see how the audio is delivered at different corners of the robot. I am still planning to use a combination of the MEMS array of microphones along with the directional microphones. Following the previous week’s feedback and finalized the microphones and will start creating simulation models to conceptualize the system behavior. 

 I am on track with the progress for this week. I have identified the microphones and servo motors we will be using for the robot. In addition, I have borrowed (free) a photoresistor, an ultrasonic sensor, and a temperature and humidity sensor for testing purposes. 

By next week, I would like to get more research regarding the audio triangulation mechanism and mathematical derivation. I will be setting up the text-to-speech libraries on a computer and figure out the integration with RaspberryPi, and setting up speakers from the RPi by week 7 with Shannon. I will also meet with Jeffrey to analyze motor specifications including the voltage, power and torque using datasheets. 

Posted on

Team’s Status Report for 9/28/2024

The most significant risk for our team right now would be the integration of the Robot with the WebApp. We are thinking of using WebSockets because of its low-latency nature with full-duplex communication which would allow better real-time communication between the robot and the WebApp.  However, a key challenge is that none of us have prior experience with using WebSockets in this specific context, creating uncertainty around implementation and potential delays. To manage this risk, we plan on scheduling dedicated time for learning WebSocket integration and seeking advice from mentors or who have used WebSockets.  As for our contingency plans, we plan to possibly switch to a standard HTTP-based communication using REST APIs over WiFi, (though this might introduce higher latency), or using a physical Ethernet connection to reduce the risk of network disruptions, (though this would reduce flexibility in robot placement and mobility).

Another possible challenge is integrating the DCI display with the Raspberrypi. Ensuring a reasonable frame per second value along with a smooth facial transition, such as blinking and smiling,  to ensure human-like interaction with the bot. To implement this, we will use certain python graphics libraries like Pygame for simple 2D rendering, or Kivy for a more advanced interface.  To maximize the lifespan of the screen, we will be using screensaver and sleep mode  during idle moments or periodically change the content displayed on the screen by making slight changes. This can be done during timer countdowns and is generally not a concern if the user is not using the bot. 

We also got together as a group to update our system specification diagram, which we included in our design proposal.

We decided to allocate specific time to the WebSockets integration of the robot next week.

Part A was written by Mahlet Mesfin

Our StudyBuddyBot is a study companion meant to motivate students to study, track their study habits and provide relaxation when they take breaks in between their study sessions. This allows students to have a good experience while being productive inducing psychological satisfaction. The bot can guide students to follow optimal study schedules(such as the Pomodoro technique), which ensures a well balanced approach to work and rest. This will help prevent overworking, overall leading to a better mental and emotional health.  In addition, we will be incorporating reminders for the user to take breaks in reasonable intervals reducing fatigue and eye strain. 

The game feature of this StudyBuddyBot allows for a short but fun experience during these study sessions, timed well so that they don’t cause prolonged distraction. This will also help in fostering the sense of companionship and reducing the feeling of isolation for those who can’t focus well in the presence of other individuals. This can boost the well-being of someone through emotional support. 

Part B was written by Shannon Yang

The StudyBuddyBot will improve productivity and the well-being of students in academic environments. It will serve as a structured study companion that can help students. In situations where students have limited access to in-person interaction due to cultural factors, the robot is able to simulate a studying environment with a friend. The features for interaction that the robot has can also help to bridge gaps in the social and emotional support systems that students may lack from their surroundings. Some of the robot’s features could also be used to cater to specific cultural or social preferences (for example, setting prayer time reminders for those who observe religious practices). By incorporating both study assistance and social engagement, the robot aligns with the growing trend of technology being used to support mental health and productivity, acknowledging the cultural and social importance of companionship in learning and promoting greater work-life balance. 

Part C was written by Jeffrey Jehng

With the StudyBuddyBot, we want to use cost-effective components to balance affordability with quality. By implementing a modular design, we can have a scalable distribution in the future and ensure durability of the final product. 

An example of our final product use-case could be in a school setting, where administration/students may have a limited budget for these educational tools. By designing the StudyBuddyBot with affordable components and integration with a companion web-app to decrease the need for high-performance hardware, we can focus on developing key functionalities such as robot interaction and features to motivate student studying. The emphasis on affordable components under our $600 budget can make our design a cost-effective solution to assist schools in integrating advanced technology into the classroom.   

Posted on

Mahlet’s Status Report for 9/21/2024

This week, I worked on some of the feedback we got from the proposal presentation regarding the audio triangulation, the robot motion, choices of microphones, and justification for using 3 microphones. According to my research, I can justify the use of three microphones by indicating that it allows for 2D audio recognition, when paired with a directional microphone. The three microphones are going to be MEMS microphones. The triangulation technique requires very accurate time measurements and using MEMS might introduce some timing delays affecting the precision of audio. However, since a directional microphone can give us a sense of the general origin of an audio, using the signals coming through both of these and aligning and processing them will help us get a more precise audio output with an aim of 5 degree of margin of error.

In light of the proposal update, I have made slight modifications to my goals within the next few weeks on the Gantt chart. I will be working on identifying specific components for the purposes mentioned above, and I swapped the deadline for the robot neck logic and the triangulation math, based on priority. I am on track based on the Gantt chart.

During next week, I will be working on identifying good directional microphones to integrate with the MEMS microphone to have good results, and identifying the motion motors for the neck of the robot. I will also do more research on allowing audio triggers within a certain length radius from the robot. Once I identify the servos I will be using, I will work on the audio triangulation method. I will be working on the bill of materials(BOM) with my team to finalize the parts list.