Posted on

Team Status Report for 04/25

We spent the week as a team working on perfecting our individual subsystems and working on perfecting the system interface and integration. Moreover, we conducted some user testing with mechanical engineers in TechSpark, as further detailed below. Everything is very much in progress, and we are well on track to be done with our project on Wednesday and have everything set up for demo day. Our schedule did not change, and there are no significant risks to project progress now that our user interface has been completed and is easy to use.

These are the set of tests we conducted for subsystems and the overall system:

Latency:

There were two main types of latency tests conducted, which were MIDI End-to-end latency tests and overall system latency tests. As mentioned in our final presentation, the following involved using an oscilloscope and cursor measurements to get the difference in time between the first MIDI bit being sent and the last signal value being sent out to the audio CODEC (and the GPIO pin connected to the FPGA for MIDI end-to-end latency tests).

We tried out all three types of MIDI commands supported in our system (noteOn, noteOff, controlChange) and averaged the latency out through multiple attempts (10 for each command type). This would be in conjunction with different types of signals and effects applied on them. We got the following values for such.

MIDI End-to-end latency: 1.324 ms

Overall System latency: 2.384 ms

This was all within our required <5 ms latency requirement, so no aspect of the system needed to be refactored.

Audio Accuracy:

While we use “audio accuracy” as the broad term for this, we really mean how accurate notes are with regard to cents. We recorded audio for all the supported notes in the Monoprice keyboard (with all four types of signals), and used a spectrum analyzer to extract the frequency value for the specific notes. We then averaged out the difference in cents to the expected value, and got the following:

Average Frequency Difference (in cents): 0.33 cents

Highest Difference in Frequency (in cents): 1.42 cents

This was all within the difference we were trying to be under, which was 5 cents, so no aspect of the system was changed.

Audio Quality:

For audio quality, we tested our different signals for the following metrics:

Total Harmonic Distortion (THD)

Signal to Quantization Noise Ratio (SQNR)

Spurious Free Dynamic Range (SFDR)

We recorded the output for each signal on a SystemVerilog testbench, capturing such on a .txt file. Afterward, we used a Matlab script and corresponding builtin functions to extract the metrics for each signal.

It yielded the following:

Sine

THD: -55.29 dB

SQNR: +44.99 dB

SFDR: +59.32 dB

Square

THD: -6.31 dB

SQNR: +30.10 dB

SFDR: +9.54 dB

Sawtooth

THD: -1.90 dB

SQNR: +25.33 dB

SFDR: +6.02 dB

Triangle

THD: -18.33 dB

SQNR: +108.37 dB

SFDR: +19.08 dB

These results are mostly expected, as sawtooth waves have strong harmonics and square have odd harmonics. The other numbers, particularly for sine waves, are not necessarily ideal.  While triangle waves indicate barely any quantization, the rest show some semblance of such. At the moment these were measured, we were using 1024 samples for our waves. We had assumed that these were of high enough resolution to avoid signal quality issues, but going with a higher sample number could possibly mitigate some of these issues. While our sound was “good enough” at that point, we decided to attempt increasing our resolution to 4096 samples on Sunday, with the results to be recalculated for our final report.

ADSR:

For ensuring ADSR was working properly, we implemented a SystemVerilog testbench to capture and compare different ADSR values and their subsequent effect on the output. Afterwards, we compared this output and inspected the result to determine whether the envelope shape matched what we expected. This, combined with on-device testing, helped show that ADSR was implemented correctly.

Polyphony:

To ensure polyphony was working properly, we designed a SystemVerilog testbench. Because we were effectively creating a round-robin arbiter, understanding the logic of the module was not too difficult. However, on the actual FPGA there were still issues with how the input was being processed. To determine the source of these bugs, we implemented live debugging tools (specifically using the seven segment display and Arduino LEDs).

VGA:

For testing VGA, we knew that once we had some kind of image on screen it would be easy to continue development (and would most likely require no further testing). Thus, we deemed it most time-efficient to examine different VGA implementations online and go straight to synthesis rather than simulation. This is because we believed that the setup time for a debugging setup outweighed the time it would take to just set it up on the actual monitor. There was a slight bug with outputting the VGA SYNC clock, but this was an issue that came after synthesis anyway.

Effects:

To evaluate system behavior, we first implemented a SystemVerilog testbench to test specific components by inserting them into the wavetable/phase accumulator path and observing the resulting output. In parallel, we developed a fixed-point model in Python and compared it against a floating-point reference to quantify the error introduced by quantization. If this error fell within acceptable bounds, we then validated the fixed-point Python model against the SystemVerilog testbench results. To provide an example, during testing, both in simulation and on hardware, we observed that FM modulation introduced significant sideband activity, which ultimately led us to exclude it from the design.

UI:

For the user interface, we focused on minimizing setup time, targeting under five minutes for a first time user. To evaluate this, we recruited volunteers individually from TechSpark, provided them with a basic set of instructions, and timed how long it took them to complete the setup on their our computers. The results were encouraging: all participants finished in under five minutes, with the longest time recorded at approximately four minutes and thirty seconds, indicating that the system met our usability goal.

On average, we found that
Setup Time: 3 minutes and 16 seconds

We tested the measurements of the chassis (to make sure that it’s portable enough) with a tape measurer, and found
Chassis Measurement: 22.1 x 11.2 x 5.5 inches

Moreover, we weighed the overall system itself quite recently (chassis, FPGA, breakout board, Arduino) with a scale, and found

Chassis Weight: 6.72 lbs

Which was a little over our expected weight (5 lbs), but no changes were made to the system as it was within a reasonable bound due to the fact that we had not accounted for all of the interface additions (keys, LCD, plaque) in our initial weight metric. When we asked users, they said the system was portable enough to carry around, which also influenced our decision to make no changes to the chassis.

MIDI Interface:

To test the MIDI interface for correctness, I set up the FPGA’s hex display to show both the key note values and velocity values being transmitted to it. Moreover, the knobs were tested through the same manner, with the knob value and channel being displayed in the same location as the note values/velocity. I thoroughly tested for each of these, which verified that both the MIDIFsm and UART modules functioned properly. Apart from this physical testing, I had previously set up a SystemVerilog testbench to assert correctness with my UART modules, since adhering to timing rules was one of the most pivotal components of such. 

Audio Interface/Communication Protocols:

Interfacing with the FPGA’s built in audio CODEC required the utilization of two communication protocols: I2S and I2C. For those two, I’d developed SystemVerilog testbenches to test correctness and accuracy with regard to timing, since it’d be hardest to debug in the physical board with an extremely fast clock.  Things like checking for correct timing on the acknowledge bits and start/stop conditions were all covered. After this, testing if the audio interface worked went hand in hand with testing the different types of signals. I would connect audio output onto an oscilloscope and play different signals, checking if it visually matched whichever had been chosen. Apart from that, we’d check if the audio of the waves would correspond with what we had expected to hear.

Posted on

Jose’s Status Report for 04/25

I spent the first part of the week conducting basic user testing with Mihail and generally receiving feedback on the design of our chassis. We checked how much it took (on average) to load all of the code onto the FPGA itself, and had some users try to set it up based on a simple instruction sheet we had set up. While the numerical results looked good (and are discussed in depth in the team status report), it was clear that the UI itself was not fully integrated. As previously mentioned, I received good feedback from testers on things that would add onto the user interface, with one of them being buttons for the drum-track. Most of the people mentioned that something akin to keyboard switches would be good, so I ordered such switches in combination with spark fun cherry MX breakout boards:
Cherry MX Switch Breakout

Once I received all of the components, I soldered one of them with a switch and set it up on a breadboard to check that it worked before modifying the box. This included creating Arduino code that would account for debouncing, which was something I had not initially considered for keyboard buttons. It all worked without many issues (since I feel that my soldering skills have improved after dealing with the SparkFun MIDI breakout board), so I moved on to placing everything on the chassis.

This second portion of work for the week began with three reveals: 1. I had measured the knobs incorrectly such that they do not fit on the circular hole in the chassis, 2. I did not account for buttons or LCDs in my original design, 3. I did not consider that adding new components would require laser cutting the components from the beginning (as everything glued together would not be compatible with the machine at TechSpark). Because of that, I spent a significant amount of time redesigning the chassis, taking measurements for the knobs, keys, and LCD to make everything fit properly.

This, furthermore, required soldering all of the other cherry MX breakout boards and pins, which took up a significant amount of time (particularly since I messed up the initial orientation to solder in which required desoldering).  Once the box was laser cut again, I put all of the components together, which resulted in the following interface being finally set up:

Now that everything is hooked up properly, testing and general setup should be much quicker in the final days of implementation. Everything on my end is going well, and there should not be much left for me apart from conducting more user testing/correctness testing for our report and setting our project up for demo day.

Posted on

Team Status Report for 04/18

This week, the team spent most of our time adding onto our implementations with new interfacing components. While the signal processing components of the system stayed the same, the overall system now accounts for VGA output and a physical box with all the general inputs that the user could need for our system. This not only makes setup much easier, but also the user experience in general is much friendlier. Moreover, we all worked on setting up testing for our system in conjunction to creating and finalizing the final presentation. With that in mind, our biggest risk is simply not being able to integrate the entire system together, as all the individual components are on track to working properly. While it’s not in our plans, there is the possibility that one thing doesn’t mesh well with the other, which would involve us putting in extra time to getting everything together working (or segmenting pieces of the system). Overall, though, we’re on track to get everything working by demo day, and our schedule remains unchanged.

Posted on

Jose’s Status Report for 04/18

Most of my work during the week of carnival involved setting up knob input from the MIDI keyboard. This required debugging and adding onto the UART and MIDI receive FSMs to account for more than one byte of data being processed (since knob inputs require both channel data and value data). Doing this required utilizing the FPGA’s hex output to display what values were being received, which allowed me to notice the bugs with the system (which were mainly timing/fsm structure related). Since most of it was building on top of already existing components, though, the work was not as painful as setting up note on/off command receiving. The rest of the time was modularizing the code so that Mihail and Kaloyan could easily assign values to the knobs without having to interface with any components they may not necessarily understand.

I spent the week after carnival primarily working on the external interface of the system, with the rest of the time being focused on getting testing data for the final presentation and the creation of such.

Most of my time when creating the interface was getting accustomed to the laser cutting interface in TechSpark. Everything was smoother than expected, though, thanks to the help of both the people working there and their video resources. I used a hinged box model extracted from the website Makercase.com, as pictured below:

I downloaded its corresponding SVG file, and modified it as necessary in CorelDraw. 

Once that was done, I used the laser cutting machine on three slabs of plywood (which was around $9 dollars, much cheaper than the 3d printing costs). It resulted in the following box (which I engraved with some designs, on top of that):

After that, I spent my time configuring the knobs that would be present in the box (the Modulino knobs that we’d ordered beforehand). I configured the code in the Arduino itself so that it could be considered the same as the knobs in the MIDI keyboard, and the data would be transferred through the same GPIO port that that keyboard data gets sent through. Thanks to the Modulino library (and the knobs’ support for QWIIC connections), setting up the I2C communication was relatively painless.

Finally, I set up the testing interface for extracting latency (both overall system and MIDI end-to-end). This required the use of the Agilent oscilloscopes present in some of the ECE labs. I set it up so that one channel would be connected to the keyboard’s MIDI output, while the other end would be connected to the FPGA’s GPIO port. Then, in the oscilloscope, I set a trigger for the falling edge of the MIDI output (when UART began sending the message). Once that was triggered, I could then use the measure feature (coupled with cursors) to measure the time difference between the keyboard sending a message and the FPGA receiving it.

Overall, though, everything is on track as it should be. Most of the work for this week was building up on previous work or doing physical work (with laser cutting), so no real hiccups were expected. My plan for next week is to add more components to the physical box setup (keys, LCDs, etc.). Moreover, conducting user testing of some sort is in our plans.

While I felt that my pre existing knowledge on digital design and FPGAs was sufficient for many of the tasks I worked on, there were quite a few that involved learning new things. Laser cutting is the first one that came to mind, as I had to learn how to use the machines in TechSpark and also the drawing software for the schematics (CorelDraw). I used a mixture of YouTube tutorials on those pieces of software (generally titled “How to laser cut a hinge box”) and help from TechSpark staff. Apart from that, learning about and implementing the I2S protocol involved reading quite a few things. In particular, I adopted my usual strategy of reading forum posts on the matter, as they tended to clearly outline the requirements for the communication protocol in a clear manner. This was the same for both UART and I2C as well. In the same vein, I used forum posts and online videos on the MIDI protocol as a basis for my FSM implementation and general debugging of the keyboard interface through the SparkFun breakout board. Finally, the tools I used for debugging (oscilloscope, matlab, systemverilog test benches) all required looking at thorough documentation, forums, and YouTube tutorials for proper use, as I had not resorted to using these in a long time. Overall, I tended to stick to informal methods of learning for these new pieces of information (tutorials, forum posts, etc.) as they tend to be the most easily digestible for me.

Posted on

Team Status Report for 04/04

So far, everything is going according to plan and no major risks are present with regard to our project’s progress. As Jose’s interfacing baseline is setup, the major risk of having no button/key/tactile feedback with our system has been resolved (the video showing this is shown in Jose’s report for the week). As talked about in the interim report, the schedule has been updated to represent the work we’ll be doing on the actual overall physical interface. Since we want to make a box storing all the components and the like, time will need to be invested in designing and printing/laser cutting. Here is the updated schedule for such:

Our plan for testing the whole system initially involves playing different audio wave types/drums in combination with filters, checking that the audio corresponds to the intended filters/waves chosen. Note frequencies will be tested by using a spectrum analyzer app, making sure that the MIDI key played is accurate to the one outputted. End-to-end latency (where we’re targeting < 10ms) will be tested through an Audacity two channel recording. The first channel will be connected through the MIDI keyboard (where we can have it connected to a USB apart from the MIDI DIN) while the second is connected to  the output port of the FPGA. Signal quality/aliasing will be tested, as previously discussed, through Matlab. We’re going to record the output coming out from the FPGA through the 3.5mm cable and then import it into Matlab to compare with an ideal signal.

Posted on

Jose’s Status Report for 04/04

I spent the week further working on getting the MIDI board to work with the FPGA. After arduous testing and debugging with no results (and conversation with faculty), I decided to pivot to interfacing through Arduino. Instead of having MIDI -> Board -> FPGA, I’d be doing MIDI -> Arduino -> Board -> FPGA. Because the board itself was made as an Arduino shield, interfacing is much more simplified (and also good for confirming if the issue is with the board or the RTL code). I worked on setting up the interfacing using the FortySevenEffects MIDI library for Arduino, where the Arduino would receive MIDI data through UART at a 31,250 baud rate in the board’s RX/TX ports.

The board has two separate configurations for data transfer, which are software and hardware.
Serial Selection Jumpers

By using the software port, data gets transferred through the Arduino’s 8/9 digital pins and SoftwareSerial library, leaving the 0/1 pins for serial debugging.  This required soldering the tiny rectangles pictured above, but doing so was difficult and brought about a multitude of debugging challenges. While it ended up working for a little bit of time, the board’s decoupler IC got damaged. I ended up pivoting to the backup board, and resoldered it for the standard hardware UART configuration. While I cannot do serial debugging, I can still send out the received Arduino data through the 8/9 SoftwareSerial pins to the FPGA.

Once this was set up (and confirmed that data was being received through the Arduino’s LEDs), I connected the software TX/RX to the FPGA’s GPIO pins and integrated it with my existing UART code. Moreover, I set up the hex displays to show the note values of whichever note I’m pressing. I configured it with sound output, and was left with a functional interface for key presses. Here are the two videos showing the following results.

Key Pressing:

https://youtube.com/shorts/B49BmfY5yy4?feature=share

Hex Display Setup:

https://youtube.com/shorts/aDOGcXA_wZU?feature=share

My plan for next week is to configure the knob interfacing with the keyboard and begin drafting/setting up the physical interface for the entire product (utilizing CAD for laser cutting or 3d printing depending on whichever approach we choose). Everything is on track now that the interfacing works for keys, as the further work builds off of this.

My plan with regard to testing this system (and further interface additions) involves utilizing the hex displays that I’d previously set up. As shown already, I’m using it to make sure that the note number and velocity values when pressed are correct, but my plan will be to use it for the knobs and sliders as well. Key latency will be tested utilizing an oscilloscope, measuring when the FPGA’s GPIO pin receives MIDI data relative to when it was pressed. Audio validation, for now, has been purely auditory and will further be tested when we begin sampling the signals for audio quality data. While accuracy needs to be tested through these means, overall correctness is tested through the previously mentioned technique. Apart from auditory tests, though, the planned tests have not been conducted.

Posted on

Jose’s Status Report for 03/28

I spent the week working on integrating MIDI keyboard input with the SparkFun shield. The first step of doing so was soldering all of the components that it came with, which was the most arduous task of them all. Once done, I began investigating GPIO inputs for the DE10-Standard FPGA. The only inputs necessary for getting MIDI IN transmitted are a 5V power source, ground, and RX output. This resulted in the following setup as pictured below:

The rest of the week up until today has been spent debugging the implementation. I’ve tried multiple approaches, including Quartus’ SignalTap tool and manual multimeter testing. The former did not work properly with my laptop’s USB ports, while the latter did work for the purposes of making sure that the 5V output supplied by the FPGA was correct. As of right now, my code is set up to blink a light when UART is transmitting a message. While it is blinking, there is no way to make sure if what is being outputted is garbage or real messages. I’ve also tweaked the UART code to make sure that it is not an issue on the RTL side, making sure that I’m respecting UART timing rules and the like. My plan for next week is to use the Arduino we’re ordering for the purposes of making sure that the shield is functioning properly (as mentioned before, SparkFun has their built in Arduino interface for printing MIDI data). Frankly, this issue is taking me longer than I thought it would, which means that I will have to put in more time the following week to make sure we are caught up with everything. 

Posted on

Jose’s Status Report for 03/21

I spent the week working on integrating and modifying the audio CODEC output with the wavetable sampling/phase accumulator portions developed by Kaloyan and Mihail.  The I2S module that I originally developed would output 16-bit values, which was noticeable in the output quality of the device once these sampled waveforms would be outputted. With that in mind, I modified the protocol code to work with 24-bit values, allowing for much higher quality audio resolution. This was tested utilizing Kaloyan’s wave frequency oscillator, such that the FPGA would output (from the line out port) a frequency increasing sine wave.  The volume output was quite low, which required changing the I2C setup module to send a write command for volume (setting it to its maximum for now).

Once this was done, I moved onto creating boilerplate code for the actual MIDI keyboard input (as the MIDI shield is supposed to arrive by Monday). I had the following setup with the FPGA itself, where each switch represented a MIDI key:

I set the code up so that flipping a switch would send out the corresponding MIDI signal as generated through the breakout board (generally, the switches should also be representing the different piano keys). My idea with this in mind would be to just have to change the outer receiver logic, but have the inner processing done beforehand. With that in mind, the FPGA is now able to output different frequencies through the CODEC with different key presses in mind, as shown in the video below!

https://youtube.com/shorts/-aKYDpD8pd4?feature=share

Right now, everything is going well with regard to our schedule as no major hiccups were faced. My plan for next week would be to set up the actual MIDI input receiver once the breakout board arrives, such that the system can be tested with a physical keyboard.

Posted on

Team Status Report for 03/14

The main design change made was covered in Jose’s status report, which is the asynchronous FIFO (rather than a normal FIFO) for managing data between the general FPGA and audio CODEC system. This was done after a talk with teammates where we realized that there was a need for clock domain crossing in our system. While necessary for our system to function and send out data properly, it does not incur any external costs aside from the time spend implementing/debugging (which has only been around three hours so far). Overall, though, our schedule has not changed and we look to keep progressing with our project as planned (and begin doing more thorough integration with our FPGA before the interim demo). Our biggest risk so far is simply getting audio output integration with signals working, since it serves as the backbone for our system. Hopefully, it’s up and running by the following week. In the case that progress staggers, we plan on spending more time than allotted to the course next week to get it working (in addition to the time spent on the other portions of the system.

Posted on

Jose’s Status Report for 03/14

After working on the individual ethics assignment, I spent the week further debugging and working on planning out the audio codec’s integration with the overall signal generation pipeline. I investigated the possibility of using external IP for generating a PLL (Phase Locked Loop) for the I2S portion of the interface, since further research showed that there was a need for having a stable clock. The protocol needs to operate at a 12.288 MHz clock, yet the input signal I am currently using is the DE10-Standard’s general 50 MHz clock. Currently, we are using a free version of Quartus for managing our FPGA, so I was worried that it wouldn’t be available to us.

After working on this, I spent some time discussing with Kaloyan on how we would be integrating the signal generation system. We realized pretty quickly that it would involve dealing with clock domain crossing, as our overall system would most definitely be running at a different speed to the 12.288 MHz for communicating with the I2S module. The original plan was to stream each value from the generated signal through a FIFO, so we chose to keep that same idea and use an asynchronous FIFO. The writing side for it will run at 50 MHz (the clock speed for the overall system), while the read side runs at the I2S interface’s 12.288MHz. While I wrote the RTL for the system this week, I plan to work on testing it the following week (and trying to get some integration going on). Everything is still in progress, thanks to the fact that we had moved getting keyboard input to work back.

Overall, the system now has the updated setup: