Posted on

Ryan’s Status Report for Saturday, Mar. 28

This week I’ve worked on synthesizing a recording of an air leak for use in our performance testing and validation.

First, I downloaded an air leak sound effect and imported it in Audacity:

I then amplified it and kept only the middle portion where the leak noise was consistent:

I did an FFT to get an idea of the main frequency content:

We have peaks in the following locations:

2563Hz: -24dB

8667Hz: -18dB

13597Hz: -17dB

I then created an EQ profile to apply to a white noise source:

After applying the EQ to white noise, we get an FFT:

This looks very close to our original leak. We can use this synthesized sound to test the system after we verify it with a single tone sine wave. The synthesized source can act as an intermediary between the single tone and a real recording of a leak because this is more complex than a single tone, but less complex than the sound of a real leak.

Further work:

Deeper analysis into what sounds the system detects well and which sounds the system does not perform well with, and why.

Compare and contrast real world air leak recordings and synthesized versions.

Posted on

Jonathan’s Status Report for Saturday, Mar. 28

This week I mainly worked on the microphone boards and FPGA drivers.  The microphone boards were finished early this week, and ordered on Wednesday.  They were fabricated and shipped yesterday, and expected to arrive by next Friday (4/3).

As mentioned previously, this design uses a small number of large boards with 16 microphones each, connected directly to the FPGA board with ribbon cables.  This should significantly reduce the amount of work required to fabricate the boards and assemble the final device, though at the expense of some configurability, if we have to change some parameter of the array later.

As the schematic shows, most of the parts on the board will not be populated, but were included as mitigations to possible issues.  The microphone footprints were included in case we need to change microphones, and there are several different options for improving clock and data signal integrity (such as differential signaling and termination), if needed.  Most parts, particularly the regulator, are relatively generic, and so can be acquired from multiple vendors, in case there is a problem with our digikey order (which was also placed this week).

While working on the FPGA ethernet driver, one problem that came up was with the UDP checksum.  Unlike the ethernet frame CRC, which is in the footer of the packet, the UDP checksum is held in the packet header:

This means that the header depends on all of the data to be sent, which, means that the entire packet must be held in memory, then the checksum computed, then either the checksum modified in memory before transmission, or, during transmission the “source” of data has to be changed from memory to the register holding the checksum.  I didn’t particularly like either of these solutions, and so, came up with another.  I made the checksum an arbitrary constant, and added two bytes to the end of the UDP payload.  Those two bytes, which I termed the “cross”, are computed based on all of the data, and the header, so that the checksum works out to that constant.  The equation below isn’t exactly right, but gives the basic idea:

In this way, the packet can be sent out without knowing it’s entire contents ahead of time.  In fact, if configured to do so, could actually take in new microphone data in the middle of the transmission of a packet, and include that in the packet.  This greatly simplifies the rest of the ethernet interface controller, at the expense of a small amount of data overhead in every packet.  Given the size of the packets though, this tradeoff is easily worth it.

This coming week, I mainly plan to work on getting the information flow, all the way from the FPGA PDM inputs to a logfile on a computer working.  This was expected to be completed earlier, but the complexity of ethernet frames ended up being significantly greater than expected, and took several long days to get working.  At this point all the components of the flow are working to some degree, but do not work together yet.

Posted on

Team Status Update for Saturday, Mar. 28

This week largely saw a return to a more normal work schedule, and beginning to act on the updated plan.

Our new timeline to account for the recent changes is shown below:

Most components are on track or completed, though there are several components which have fallen slightly behind.  Mainly the computer-side driver to receive and decode data from the FPGA, and the array processing software.  To help get back on track, and since much of the audio calibration/testing has been removed from the scope of the project, Ryan has become more involved with the software/processing side.  John also plans to finish the network driver this weekend.

On the hardware side, the architecture of the physical architecture was slightly changed to simplify the design.  Specifically, from having a single carrier board with connectors for each microphone element, to having a small number of boards with many microphones each, that connect directly to the FPGA board using ribbon cables.

Our risk management has largely been reduced the last few weeks, as most of the main risks we foresaw were eliminated over the course of the project so far, or, did not end up being issues.  Our PCB and digikey orders have been fulfilled already, so unless there are problems with shipping, we should have the necessary hardware.  If there are any problems, board design includes several generic alternatives for each critical part, so as long as at least one supplier remains open, we should be able to finish the hardware (see John’s update for more details).  On the software side, risks and mitigations have not changed.

Posted on

Sarah’s Status Report for Saturday, Mar. 28

This week, I mainly worked on writing up time delay beamforming processing.

For a linear array of microphones that are located perpendicular to the sound source, the signals captured by the microphones have similar wave forms but varying delays. In order to account for these delays, I calculated it from speed of sound, distance between the microphones and the sound sources. Also, signals must be shifted for runtime difference. Finally, all the signals can be summed up. The summed signal is then normalized by the number of microphone channels.

For next week, I plan to run simulations with sample data and fix any problems that come up.

Posted on

Jonathan’s Status Report for Saturday, Mar. 21

All of the changes to daily life this week didn’t leave much time for work on capstone outside of class, so the additional work completed has been minimal.

Most of what I was able to work on this week was design of the microphone array boards, which was originally scheduled to be finished over the last couple days of Spring Break.

I was able to complete a simple breakout for the microphones, as we’d originally planned to have an array where each element is identical, and on its own small carrier board, with the boards connected to the FPGA with FFC cables.

However, with the changes to the project, I decided this was probably too work intensive to build for a single person to complete, and, difficult to build without the tools we expected to have available.

Instead, the design was changed to a set of 6 boards, each with 16 elements, and a single connector to the FPGA.  This board is still in progress.

This cuts down the total number of components needed significantly, and reduces the number of connectors from about 200 to just six.  The drawback is configurability, if we later find that the 1″x1″ array spacing does not work well, there is not much path to changing it, some workaround would have to be found.  All of the analysis so far has indicated that 1×1 is close to optimal for the frequencies we’ve chosen, but, practical testing will undoubtedly find problems that simulation did not account for.

I have done some analysis of possible workarounds to changing array geometry, mostly pertaining to moving the sub-arrays slightly off-grid relative to each other (so, for example, two array are placed with their nearest elements just 0.5″ apart instead of 1″), which does allow for aliasing and other artifacts to be traded off with directivity, though the tradeoff is often steep in whatever is being traded against (i.e. to gain a little directivity, you have to tolerate significant artifacts, and vis versa).  Hopefully we will not have to do any of this, and all the analysis so far says we won’t, but it is good to know roughly what we’re facing if we do have to.

Photo shows some of the strange artifacts that emerge with unevenly spacing sub-arrays.  The two sources are real sources, so the array is not aliasing, but the sources are no longer round, they are shaped strangely, and, not exactly the same.  The output is still usable, but this illustrates a significant tradeoff of this method of constructing the array.

Posted on

Team Status Update for Saturday, Mar. 21

This week brought some significant changes to the project, the new scope is summarized in our updated SOW:

Team E0 Statement of Work_3_21_20

Short summary is that the only major changes are to the details of the design, to account for losing access to campus resources, and several tests/characterizations/calibrations of the array will no longer be possible, and will therefore not be performed.

Work will be roughly divided by area, John will construct the physical array, Ryan will write the software to do audio-level processing, and Sarah will write the software to do array-level processing.

With the shutdown of roboclub and most other campus resources, we’ve had to check out most of the tools that we planned to use to construct the project, and set them up an at-home lab.  That along with the mass of changes to all of our daily lives this week has taken up most of our time, so not much more has been completed.

A few things have been done though, some amount of board design for the microphone carriers was done (see John’s update for details), though still in progress, and, the reel of microphones arrived from TDK/InvenSense.

Posted on

Sarah’s Status Update for Saturday, Mar. 21

This week, I had a meeting with my team members and Professor Sullivan to talk about modifications in our project. Since my team will have to work on the project remotely for the rest of the semester, we have to break up the parts so that the workload is evenly divided to each person. We are thinking to have Ryan collect audio date from the microphone array and pass it to John for processing and then finally passed to me for localization. There need to be more discussions on the plan for the upcoming week. In addition, I worked on coding PDM to PCM in Python. I also researched about localization of audio sources in Python as well.

Our schedule will need to be modified due to the remote working environment. Yet, I am still on track with the major components.

Next week, I will have more discussions with my team members and work on debugging python code.

 

Posted on

Ryan’s Status Report for Saturday, Mar. 21

After talking to Patrick from TDK Invense, we were able go get 110 of the ICS-41351 from them. We’re very thankful for this because this would have been the most expensive part of the project, as shown below in the bill of materials from the design report:

If we were to order the microphones from Mouser, it would have costed $124.30. Now, we’re able to allocate that for monetary slack, as we may encounter situations during the build process that we may need to order new parts.

 

The candidates for the microphones were ICS-41351, INMP521, and INMP522. The latter two had slight better frequency response at the frequency of interest (5kHz and above), but we ultimately decided on the ICS-41351 because it had a footprint that was much nicer to work with.

It has relatively large pads on the bottom:

Because it has the microphone port on the top, the bottom solder pads are not constrained.

The INMP521 and the INMP522 however, have the microphone ports on the bottom, so it becomes  much more challenging to solder properly, because the solder ring around the port has to be completely soldered for best acoustic performance, and it’s not easy to verify proper soldering when we do not have access to professional reflow tools.

The main task moving forward is to get the PCBs that John designed made and reflow them. I think it’ll be a fun challenge.

Posted on

Sarah’s Status Update for Saturday, Mar. 14

Before heading off for Spring break, there were some updates.

I worked on finalizing PDM to PCM code with help from Ryan. The FIR low pass filter I used has a large order which has a narrow transition band. The image below shows our filter.

The image below shows the filtered PDM signal using the FIR filter.

It shows that there is high-frequency content, critical for identifying our sound source. I plan to test more pdm data for similar results.

I also worked on the design report with my members. We received feedback from Professor Sullivan and will further discuss the details through our meeting.

We are on track and there are a few things to do this coming week as planned.

To do next:

  • Discuss with my team members about re-scoping the project and make design decisions due to a change in our locations.
  • Discuss design report feedback with members.
  • Work on pdm to pcm in Python.

 

 

Posted on

Jonathan’s Status Report for Saturday, Mar. 14

Unsurprisingly there were some significant changes to the progression of the project this week.  Before the switch to online classes was announced, I completed two things for this project, pertaining to the aspect which was slightly behind schedule, the gigabit ethernet.

The Numato PHY was ordered last week, but had not come in by Wednesday, so I decided to freehand solder the ADIN1300 sample that we already had.

The ultimate purpose of this was to debug the RGMII implementation, which should be a lot easier, since we have a full datasheet for the ADIN1300.  To this end, I added a microcontroller to communicate with it over the MDIO interface.

By looking at the errors that came up when attempting to use it, I eventually found that some of the bits were backward (forward for RGMII, but ethernet is lsb first, so the preamble and CRC were backward, though the payload was correct).  Also the CRC computation was slightly off.  Both of these issues were fairly easy to fix, and the end result was that I was able to send UDP packets to my computer.

Applying these changes to the other board (the RV901), got it working as well.  Given the interruption to everything from Corona and that this is working as is, we’ll probably end up using this board for the final device to save time.

The day after I got this working was when all the Corona stuff really started kicking off, so I haven’t gotten much further than this.  I did start on the microphone carrier boards, but the design of that may have to change to deal with suppliers (possibly) not being open.