Author: cne

This week I completed the server/client code to send and received audio data between the raspberry pi and the server.  This server code here AWSServer

receives data from 2 separate audio devices and places the input into a queue to be sent out to the other audio device.  So for example device one sends a packet to the server, this packet it put in to the queue and is sent to device 2.

The client code is here RPiClient

This code will package the audio information into a list which then is turned into a byte string, using the python library Pickle, so that it can be sent to the server.  Pickle is used on the server side as well to unpackage the data.   Input audio data is also placed on a queue to be played on the speaker for the users.

This week I also worked on the design report document, I have finished a draft of my sections, on Sunday I will revise this writing for a final draft so that we can discuss the document on Monday.

I am on schedule this week.  Next week I will make the “Top” file for the raspberry pi to connect the client code and the audio I/O functions for the ReSpeaker, I will continue testing the code as I write it.

This week I started finished the design for the network connection between the raspberry pi and the AWS server.  We are still planning on using a UDP connection here to transmit packets, the packets will be a struct of the micID, audio bytes and direction of arrival information for the audio.   The micID is assign to a single rapsberry pi audio device from the server, when the connection is first made.  The AWS Server will receive information that the audio device wants to connect and will return the mic ID, this will be resent until the raspberry pi audio device receives the micID or the connection times out.  On the server both listen and send will need to run on separate threads to make sure that the server is always listening.  There will also be a separate thread for each audio device connection to the server.

I set up a simple server code to test the connection between the aws server and the raspberry pi.  Since the AWS server is remote it is in a different wifi network so I had to set up port forwarding, so that the raspberry pi can access the server code on the AWS Server.  AWS has its own way for users to setup port forwarding, using AWS Systems manager, I have been following this tutorial https://aws.amazon.com/blogs/mt/amazon-ec2-instance-port-forwarding-with-aws-systems-manager/.

We also received the AWS credits so we upgraded our ec2 instance to m5a.large, we are tracking our usage of the server.

I am currently on schedule, In the next week I am going to complete the code to handle the networking from raspberry pi to AWS Server, this code will include the initial raspberry pi audio device handshake with the AWS server, and the transmission of our packets.

 

 

This week  I gave the project proposal presentation on Monday, and I ordered a respeaker to start configuring the hardware.   I already have a raspberry pi 3 B+ so I spent some time downloading the libraries we will need to the raspberry pi, to test downloading and running code on the raspberry pi.  I also started to work on the design for the code that will run Raspberry Pi.  The flow of this code will need to handle listening for audio on the microphone, sending this audio in real time to be processed, then  transmitting the audio to the AWS server.  For this, I found we need to listen for audio on the microphone on its own thread separate from audio processing, we also need to run the server listen and accept on its own thread.  I found 2 ways we can do this in python, 1 by using the library thread or 2 by using the library asyncio.  I am leaning toward using the asyncio library since it is more helpful in creating io routines and also for networking.  One main challenge here is that we will need to tell the respeaker to output audio potentially during the time that it is listening to audio.  I will test this functionality on the respeaker when it arrives here.   I also researched how we can send the input audio to be processed, we can use a python numpy array or a .wav file, so this format depends on how Mitchell will filter this audio, and what the format needs to be for his script.  For the server code, I am planning on using the python socket library and a udp connection. The packets will be compressed audio and the metadata containing the direction of arrival (DOA) of the audio.

My progress is on schedule this week.  For Monday I will create a system diagram of the software on the raspberry pi, and continue working on the design presentation slides.   Next week I will write the server code that goes on the AWS server and start to test the raspberry pi connection to the server.

 

 

This week we worked on creating an outline document with our research.  I added my research about hardware, why we are using a raspberry pi, which respeaker we are using and which AWS server we should use.  I also added research about how to transmit audio data on the application layer to the AWS server and how to compress the audio for creating packets to send.

From this research document we created the slides and since I am doing the proposal presentation I have been reviewing what I will say.  I also talked about each slide with my group.

The most significant risks we could have at this point would be incorrectly laying out our work on the gantt, and not getting the timing right since we are just estimating how long each task will take us.  To mitigate this we will be updating the gantt chart as we figure out more about how long each piece of the project will take.

We have worked on designing the system for the most part this week so we don’t have any changes to report.  We also have just created the schedule this week which is linked in our team report.