Author: eseeser

Ellen’s Status Report for May 1

We did a lot of testing this week. I was responsible for the transcription related tests so I spent time calculating word error rates, speaker ID error rates, and the like. The transcript latency test was fairly time consuming to evaluate as I couldn’t come up with a way to automate it but in the end there were a good number of samples and positive results.

I helped to prepare the final presentation slides this week. I also whipped up a script for the final video, and we got a lot of filming done today.

There’s not much else to report! I’m glad we pushed very hard to do a nice integration before the interim demo, so now we aren’t rushing to finish our MVP. Next week we’ll keep preparing final materials.

Team Status Report for May 1

This week we finished development and both started and ended our testing phase. Testing was conducted on Friday and Monday. This involved developing testing strategies, doing the tests while we were gathered in the same place, and then evaluating the results. We were happy to find that all tests passed based on our requirements!

We also finished preparing our final presentation, filling in the final test results and additionally including links to sample test data. Mitchell practiced giving the presentation to the group.

We started planning our final video this week as well. We outlined what we wanted to include in the video and drafted a script for the voiceover and the footage we want to capture.

At this point, since we’re focused on our final materials, our risks and scheduling constraints are passing into the rear view mirror. We’re on schedule with our work. Our main risk is that the second covid vaccine dose incapacitates two of our members who are getting it next week.

Ellen’s Status Report for April 24

We’ve made a lot of progress the last two weeks. Our demo last Wednesday went well, and afterwards because of the feedback we received we decided to revamp the speaker id setup process to make it more user-understandable. I worked on doing that all weekend, and then we tested it a bit on Monday while testing the AWS deployment.

Then Tuesday and Thursday I worked on outlining and fleshing out our final presentation slides. I also came up with a potential method for transcript latency testing.

Since we’ve already moved on to validating requirements (with a high confidence that they’ll pass)  I’d say we’re on schedule according to our Gantt chart. This week we’ll finish gathering and evaluating test data. I’ll probably be calculating a lot of transcript related error rates. We’ll also prepare our final presentation and work on our video.

Ellen’s Status Report for April 10

This week has been a busy one for our team! Over the weekend I wrote audio-accumulation for the UDP side of the webserver and accumulated raw audio instead of files. Then from Monday forward our focus was on integration. After every time we met I had a bunch of todo items to debug or just improve the server processing and transcript generator. We met Mon., Wed., Fri., and Sat. for multiple hours each time. My work has included small (but important) tweaks like an improved filenaming system, improved audio accumulation, improved error reporting, changing what does and does not count as a “speaker change” in the transcript and in the backend, and others I can’t recall at this point. Larger changes I made would be changing the way speech to text results are accepted from Google (only accepting “final” and not interim results) and adding more branches and states to the state machine of microphone setup.

From now on my schedule is the same as the group’s schedule as a whole. We think we have a demo prepared at this point but when we meet tomorrow we’ll do more testing on the demo we’re preparing as well as testing how two mics in the same meeting work together. By the weekend we’ll be done integrating and debugging our two-mic setup and we’ll have some idea of how well the transcript is matching our requirements/targets.

Ellen’s Status Report for April 3

This week I worked intensively on the speaker identification subsystem. If you’ve already read the team status report you’ll know that we decided to ditch the speaker diarization packages we had previously identified as possible solutions – crucially, none of the packages provided audio streaming abstractions – and proceed with google cloud speaker diarization, which could be added to the current google speech to text request by simply tweaking a configuration variable. This created the opportunity to integrate the speech to text and speaker ID modules, so I had a lot of code to write and rewrite to both get the speaker ID module integrated and to add in the processing the way we desired it (including the DOA augmentation). Initial tests (using random vocal samples I found online) suggest to me that the system is going to work well and that DOA augmentation is actually going to be quite valuable – but I conducted this testing just yesterday, so the jury is still out, I suppose.

At this point we’re sort of reaching the point of schedule ambiguity. I would consider speaker identification to be finished as of yesterday, which is a good half-a-week early. I don’t have any more individually-assigned coding tasks. Now I’ll  be supporting integration through debugging and revision. As for what I know I want to accomplish this upcoming week: I want to rewrite the network-to-transcription queueing interface to accumulate multiple packets (if available) before sending the audio off to transcription. I also want to help Mitchell get transcript streaming fully working.

Team Status Report for April 3

This week we started to wrap up our individual modular code-writing and looked towards integration and demoing. As we tested and composed some of the most complex sections of the solution, we had to do some problem solving that led to design changes and improvements.

There are two notable design changes. The first is with regards to audio i/o on the raspberry pi. Our initial design involved passing audio through multiple queue data structures as processing was performed on it and it was prepared for networking. However, we found that this caused latency issues that resulted in sporadic and choppy audio. With the queueing being too slow, the audio stream now doesn’t go through any intermediary data structures before being networked, and audio processing is isolated on the webserver, as transcript preprocessing.

The other change pertains to the speech to text and speaker identification ML modules inside the transcript generator. We decided to utilize google cloud speaker identification because this would allow us to do all of the ML processing in a single integrated transaction. And this further allowed us to integrate the two modules into a single module. This also decreases the amount of multithreading.

Now we’re looking towards full integration of all components with each other. We’re doing local testing before deploying to AWS, which creates the risk that what works on our own computers doesn’t work once we deploy properly.

Ellen’s Status Report for March 27

This week I continued moving forward on my tasks in the order I assigned them to myself. First, I finished the ethics assignment over the weekend. Also over the weekend, I finished adding functionality to the buttons and forms on the website.

Then, over the week, I worked on the mic setup process. When a mic first joins a meeting, it’s in setup mode – audio is used to assign names to speakers. In this process the system compiles names, initial locations, and vocal samples for all speakers. Then when the web user says the process is done, audio starts getting routed to the actual transcript file. I finished coding up this process in a way that meshes with the current speaker id system (non-ML) but will be easy to use in the ML system as well.

I’m a few days ahead of schedule. I was supposed to start speaker ID ML on Monday, but I’ll start it on Saturday. Speaker ID ML is my last solo task.

Speaker ID ML doesn’t have to be finished by the end of next week, but it should be most of the way there; I should have at least one option working. The other thing I’m going to do is rework my data storage so that everything lives inside the database. Hopefully that’ll be done by later today.

Ellen’s Status Report for March 20

I worked on a couple different things this week. Each day up to Wednesday we were making tweaks and additions to the design report paper, and then submitting it on Wed.

I finished the (initial) meeting management module. After a fistfight with python import statements I was able to interface it properly with the Django models that represent microphone objects.

I also wrote a script intended to be run after the django server starts, that will start threads for Cambrea’s udp server and my transcript generator. The two will communicate with a queue object. New threads will break off to “serve” every item the udp server puts in the queue.

Then for the rest of the week I worked on setting up the web pages for getting around the website. I configured the urls, wrote the html, and made the Django “views” (controlling what http response gets sent and parsing POSTed forms and that sort of thing). I did that for a bunch of pages but there are still a couple of pages that need to be added.

I’m on-track in terms of scheduling. I was supposed to start the web page stuff today (Friday) but I started on Wednesday.

None of my current tasks have due dates before the next status report. Nevertheless, I hope to have all my web pages set up with buttons/forms that perform the intended effect on the backend as well. I will also start the mic initialization backend work, which is a task I took over from Mitchell since it’s so heavily connected to what I’ve already done. And I’ll also be doing the ethics assignment this upcoming week.