Category: Akintayo’s Status Reports

WORK ACCOMPLISHED:

In the past 2 weeks, I have been collaborating with members of the team in order to integrate the navigation and audio components with the entire system as a whole. Additionally, we were able to begin some testing of the audio input with GPS and navigation on a test route from Porter Hall to Phipps Conservatory. I have also worked on enabling the system to relay the navigation instructions to the user via a speaker. 

PROGRESS:

I slightly behind in regards to testing the subsystem and building out the audio feedback components. I will be working on catching up with this work this week.

NEXT WEEK’S DELIVERABLES:

For the upcoming week, we will be testing the GPS and navigation using sample GPS coordinates and real-time GPS coordinates while riding the bike. Additionally, there will be extensive testing of the audio aspect of the system for recognizing the user’s destination from their voice command. 

For the upcoming week, I will be primarily working on the audio feedback portion where the user can receive the navigation instruction via audio on a speaker and potentially a bluetooth headset. Additionally, validation and testing will be done to ensure that the navigation and audio feedback systems work as expected and provide the navigation instructions accurately and in a timely manner in accordance with the user design requirements.

TESTING:
1. For recognizing destination from voice commands: testing with 5 different voices (from different people) for 20 different destinations within the Pittsburgh area and checking the accuracy from the output of the speech-to-text system
2. For accurate navigation instructions: test multiple GPS coordinates on 10 different routes and check that the generated navigation instructions are accurate by comparing with the actual turns on a map. Additionally, I will be testing that the audio for the instructions works and returns the audio command with low latency

WORK ACCOMPLISHED:

This week, I completed integrating the navigation logic to the Raspberry Pi 4 system and began work to join the GPS and navigation subsystems together. Also, began testing different locations for the speech recognition part of the project. 

PROGRESS:

I am slightly behind with tasks; I would have liked to begin integration of the navigation and GPS functionalities by now.

NEXT WEEK’S DELIVERABLES:

For next week, I will be working on my individual subsystem in preparation for my interim demo and do more testing for my subsystem before integration. 

 

WORK ACCOMPLISHED:

This week, I primarily worked on the navigation generation aspect of the project. Essentially, I worked on the code for suggesting the next direction instruction based on the user’s current location. Since we will begin working on integration of two distinct subsystems, one potential risk is how the subsystems are talking to each other. For now, the tentative solution is that the GPS subsystem will be periodically writing the user’s GPS location to a text file, and then the navigation subsystem will be reading that GPS location from that file. One issue that may arise from this is the timing between the two processes and how outdated the data may become based on the timing at which the navigation subsystem reads that data. Additionally, the accuracy of the GPS data will affect the functionality of the navigation system 

PROGRESS:

I am slightly behind with tasks; I would have liked to begin integration of the navigation and GPS functionalities by now.

NEXT WEEK’S DELIVERABLES:

For next week, I will be collaborating with other members of the team in integration of the navigation subsystem and the GPS subsystems in order to have a fully functional system that tracks the user’s GPS location in relation to the route for their journey.

 

 WORK ACCOMPLISHED:

This week, a lot of time was spent testing the capabilities of different models for Google Speech-to-Text AI for extracting the destination for a journey from the user’s voice commands. After testing the different models, the decision was made to use the Chirp 2 model with model adaptation. The use of model adaptation is very important as it improves the accuracy of the recognition system. When testing, it was noticed that the system struggles with words that sound very similar such as “weather” and “whether”. As a result, with model adaptation, I can set a “boost value” for a phrase such as “weather” so that the system is optimized for identifying specific phrases.

Additionally, the logic for navigation suggestions was developed a bit and we have worked on some code that uses R-tree algorithms for identifying the appropriate navigation instruction based on the user’s real-time GPS location.

Snippet of Navigation code using R-tree algorithm:

Sample output:

PROGRESS:

I am currently on progress with my work.

NEXT WEEK DELIVERABLES:

For next week, we will work on building out the navigation system and handling the case when the user is completely off path. Also, we will be working on fabrication and 3D printing for the bike mount. Also, we will start looking at how to convert text of navigation to audio.

WORK ACCOMPLISHED

The past 2 weeks, I was able to accomplish a number of important tasks as well as some deliverables that were overdue.

Firstly, I was able to complete the integration of the USB Microphone with the Raspberry Pi, so I was able to successfully record sample audio of a user giving the location for their journey.

Following having obtained a sample audio, I began to test the Google speech-to-text AI framework and did some testing for which models work best for extracting the required text.

Log of different tests with different models

Sample transcript from audio (actual audio is myself saying: “Ride take me to Tepper”)

I also worked on some of the logic for translating the destination text from speech-to-text endpoint to the longitudinal and latitudinal representation using the Google Maps Geocoding API.

Code Snippet:

Output for Destination of “Tepper School of Business”:

PROGRESS:

I am on schedule as I made some good progress this week. I probably need to speed up a bit to ensure I have ample time to join the different components of my feature together.

NEXT WEEK DELIVERABLES:

I will primarily continue testing for the appropriate model for speech-to-text translation. I will also start working on the logic for navigation suggestion using a R-tree algorithm.

 

WORK ACCOMPLISHED:

This week, I tried to work on setting up the Raspberry Pi 4, but I realized I would require a micro SD card reader; hence, I was unable to move forward as I was missing the device. I also worked more on the Google Maps API.

Additionally, I decided to modify the design of the system by removing the web server and localizing the navigation and audio system to the Raspberry Pi instead. This drastically reduces the latency required for our system.

PROGRESS:

Due to some issues I faced, I’m currently behind schedule as I had expected to finish up with how to record audio files from the Raspberry Pi and also begin to work on integrating the Google Speech-to-Text AI.

NEXT WEEK’S DELIVERABLES:

I am mostly will try and catch up on last week’s deliverables. So, I will working on how to record audio files from the Raspberry Pi and sending it to the Navigation endpoint. I will also begin to work on integrating the Google Speech-to-Text AI.

WORK ACCOMPLISHED:

This week, I primarily worked on designing the workflow for using the user’s voice commands to extract the destination for the trip and also began thinking about the relevant data that will be required for the Google Maps API call.

Google Maps API url

(Cleaned) API response  with locations and navigation instructions

Additionally, I decided to change the type of microphone being used for the system from a MEMS Omnidirectional Microphones to a standard USB microphone. The main reasoning behind this was that the USB microphone is easier to configure and has better sound quality compared to the initial microphone.

PROGRESS:

I am in progress for the moment.

NEXT WEEK DELIVERABLES:

For the upcoming week, I will be working on how to record audio files from the Raspberry Pi and sending it to the Navigation endpoint. I will also begin to work on integrating the Google Speech-to-Text AI.

For this week, I was working on research for the speech and navigation aspects of our system. Specifically, I was identifying the different software components that would be required in translating the user’s speech, containing their desired destination, into inputs that can be used by the Google Maps API platform for retrieving the intended route. This is the tentative workflow for this process:

1. Microphone receives location from user’s speech
2. Google Speech to Text AI is used to extract destination from user’s speech
3. Google Maps Geocoding API to translate location to Longitude and Latitude (Numerical Representation)
4. Use the location information from JSON response with user’s GPS location to get route from Google Maps Direction API
5. Each leg of journey is stored in a database so, each “leg” in the journey maps to an instruction at that point e.g. turn left on xxx road, turn right at yyy intersection
6. The system will use the real-time GPS location to locate the closest leg of the journey and use that leg’s instruction for audio output to the user. The algorithm for figuring out closest leg to the current location is the R-tree algorithm.

For next week, I will be working on programming working demos that utilize the Google Speech APIs and Google Maps APIs. Additionally, I will be working on how to use the R-tree algorithm for the navigation system.For next week, I will be working on programming working demos that utilize the Google Speech APIs and Google Maps APIs. Additionally, I will be working on how to use the R-tree algorithm for the navigation system.