Team Status Report for 04/25

This week, the team starting tying together the final steps in getting the Waveshaper demo-ready. This included a battery of tests for subsystems and the last steps of integration.

Daniel and Jake worked on MIDI CC commands, cleaning up clocking issues between modules, routing the control systems for the synthesis core (i.e. enabling the user to modify ADSR or filter parameters), keyboard integration, HPS architecture, and performed the I/O latency test.

Shayaan worked on synthesis and a polyphonic demo along with project tooling for more quickly translating system parameters into standard units.

Unit tests:

  • Vocal polyphonic independence.
  • Waveform fidelity.
  • ADSR correctness.
  • Chamberlin variable filter.
  • I/O and FX engine latency.

Objective measurement vs subjective perception was a key takeaway here. The relay method for keyboard control was deemed unacceptably slow. The polyphonic MIDI arbitration procedure was also changed in light of these tests.

Daniel’s Status Report for 04/18

This week, I tried to get the keyboard up and running with the WaveShaper as well as implement MIDI CC commands to dynamically adjust the ADSR envelope and filter parameters While this is now up, it is only for one voice at a time.

The keyboard is currently using the laptop from the interim demo setup as a relay. As Jake mentioned in his status report, the distributions of the Linux kernel provided by Altera were not compiled with ALSA support despite having the corresponding dpkg packages. While I believe this can be fixed, I have decided to focus elsewhere for now.

Finally, I am preparing the Quartus file for the final artifact. The time it takes to synthesize is becoming longer and longer with the HPS. Attempts to use design partitions do not seem to reduce this.

Team Status Report for 04/04

This Sunday, the team finalized a functional interim-demo of the WaveShaper that involved end-to-end I/O functionality, oscillators, mixing, and integration with each feature’s component except for the MIDI keyboard which should be on the way soon.

Shayaan worked on the RTL for SVF filters, also wrestling with  the binary-offset encoding confusion in our IP-blocks. Jake worked on the synthesis of the ADSR envelope, ordering a keyboard, and the beginnings of the RTL for effects. Daniel successfully brought up MIDI parsing and system integration. His work result was a demonstration that could take a MIDI file from a host-machine, and play it back with saw-tooth oscillators, whilst showing the youngest notes requisite MIDI information on the DE-10’s seven-segment display in hex.

With the core elements of the subtraction synthesis pipeline at or near completion, what remains is the synthesis of filters, and adding of effects. This will transform the WaveShaper from a set of disconnected digital signal tools to a real interactive instrument.

There were a few setbacks encountered in unifying the pieces which we’re in the process of resolving. Not all components have been modified to support the binary-offset which the DAC requests in producing the output audio stream.

As the final product takes its shape, now is the time to consider both modular and integration tests. Diagnosing issues in each sub-system is much easier when direct focus is applied, and integration often comes with its own difficulties.

There are generally three types of bugs that we will be looking for in the Waveshaper hardware:

  1. Logical. (A fundamental flaw or oversight in the HDL algorithms and circuit designs which break the desired properties. Example: an error-prone frequency modulation algorithm which drifts non-linearly or jumps suddenly.)
  2. Implementation. (An issue in communication between components or hardware bottlenecks breaking system correctness or decreasing system performance. Example: A propagation delay in a MIDI stream reading system which results in lost bytes.)
  3. Subjective. (Something which breaks user expectations or lacks musicality despite technical correctness. Example: two frequencies within a reasonable margin of error of the target but sounding dissonant together due to a mismatch in their ratio.)

(Logic and Implementation may be ambiguous categories. Generally speaking, logic errors are discrepancies between the underlying mathematical functions which drive sound synthesis and the HDL specification. Implementation errors are hardware engineering oversights or mistakes which cause lack of coordination between the components as synthesized. An out-of-spec driver is considered an Implementation error.)

Logical errors for internal components will be caught and addressed by using HDL testbenches for simulation. The oscillators, ADSR envelopes, filters, and FX can all be implemented in MATLAB to provide a reference series of values in a controlled environment. MATLAB references are faster to implement at the cost of being slower to execute than the synthesized logic on an FPGA.

Logic errors for components may also be tested against pre-existing tools. For instance, the Linux command line utility amidi which comes with many distributions of ALSA (a long-standing backend for Linux audio) can be used to compare the parsing of our custom MIDI receiver with a battle-tested one.

Implementation errors will be diagnosed through measurement. There are oscilloscopes in the engineering labs which can measure the frequency of periodic signals and provide ratios between the peek-to-peek range in a signal with a (non-degenerate) envelope and its maximum. Such tests have already been fruitful in finding hardware artifacts in attempting to drive the speaker with discontinuous waves which can be eliminated with imperceptible interpolating stages.

Finally, subjective errors will be suggested by end users attempting to use the WaveShaper. These will be deliberated on, and minor changes to the specifications may be implemented.

Pitch and interval correctness, rhythmic consistency and responsiveness, lack of artifacts, and  waveform consistency are all technical factors which can be tested using an oscilloscope at the audio output. Ease-of-use can be tested in small groups of both experienced and inexperienced users of the WaveShaper.

The software will be tested with Rust unit tests. Importantly, the core library which communicates with the WaveShaper can be tested as a list of API calls.  The GUI interface software is likely too stateful to exhaustively test. As a result, in-the-field bug hunting will be done merely using the software, and an emphasis on robustness and being able to recover to states which are known to be well are critical.

Daniel’s Status Report for 04/04

This week, I brought up the HPS, HPS-FPGA communication over AXI, and implemented a MIDI state machine in SystemVerilog which can parse a stream of bytes.

These system integration tasks tied together all the work in the synthesizer core that the team has done so far into a platform ready for a polyphonic music demo which could play public domain MIDI tracks that other people have written.

Notably, the MIDI track was from a live input as though it were coming in from a MIDI controller like a keyboard. I was able to stream MIDI from my personal laptop and hear the synthesized output from the speakers. I only used command line tools that came with my system’s default package manager which aligns with the goal of the WaveShaper to be compatible with already existing technologies.

This is a first step in driving the synthesizer modules that we have designed outside of testbench environments. This coming week, the same will be done for the ADSR envelope and the filters in a custom CLI representing the software core.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks?

I had to learn Quartus II’s Platform Manager and U-Boot.

What learning strategies did you use to acquire this new knowledge?

I usually read documentation, but many of the linked documents provided by Quartus 404’d. I had to filter through old university course labs to try and guess how to write up the IP blocks internally in Quartus correctly.

Team Status Report for 03/28

This week, the team has been trying to clean up the SystemVerilog HDL for the WaveShaper core and implement it on the FPGA to demonstrate what it can do.

Shayaan has been working on the filtering and regularizing the oscillators for making all the tones sound more similar. Jake has been adjusting the ADSR envelope, adding additional logic and safeguards to prevent glitching. Daniel has been working on setting up end-to-end I/O on the DE-10 Standard FPGA development board.

This coming week, we hope to have a scaffolding of the WaveShaper to then fill in during the month of April as the period to work on the project comes to a close.

Daniel’s Status Report for 03/28

This week, I got audio output to work. Now, we have an easy-to-use streaming interface and can focus on the rest of the WaveShaper core. While it was mostly just pruning small errors in the protocol, it was fairly time consuming.

I have moved onto serial communications, attempting to get a structure up before the interim demo such that we can stream live music to show off the WaveShaper’s unique sound.

Progress is going slightly slower than I like, it has again been a bit of a struggle with an unfamiliar tool to bridge the HPS ARM-core with the FPGA fabric using AXI. Seeing as the deadline approaches, I am looking into using the Avalon-MM interface instead.

I am aiming to have an end-to-end music playing device by Sunday evening by integrating the SystemVerilog which we have so far been simulating on actual hardware.

This week, I also spent some time on a MIDI parsing state machine which is as of now just on paper.

Daniel’s Status Update for 03/21

This week, I started my implementation of the driving HDL for the WaveShaper’s audio output with Jake. This involved reading the documentation and datasheets for several different IP blocks and most importantly filtering out which ones not to use.

 

By the end of the week, we were able to get a noisy signal coming out of the hardware in a manner that is distinct from both environment noise and the state when the hardware for driving the speaker is not initialized. While this is a far cry from the desired result, I feel confident that my SystemVerilog is directly driving the output signal, so it is merely a matter of detecting whichever bug or oversight is throwing a wrench in the works.

 

This also marked the beginnings of a top-level module which has kick-started the iterative design process of architecting the HDL and interconnecting the modules that we have thus far.

 

I was not able to attend last week’s lab session and aim to better touch base with my teammates in preparation for the interim demo on Monday.

Daniel’s Status Report for 03/14

This week, we received all the necessary cables and adapters to begin synthesizing and testing functionality on the WaveShaper hardware on Friday. I began a Quartus project and have started integrated functionality, drawing from the selection of IP blocks and beginning a scaffolding to support the core modules.

As of right now, the most work has been in synthesizing and creating SystemVerilog “HDL handles.” There was some additional delay in finding out how to use IP blocks in the Quartus software.

I (with the help of Jake) was able to perform a simple end-to-end test of the toolchain, ensuring that we are able to program the hardware.

Otherwise, I considered how the WaveShaper both (1) has been designed such far in a way that presupposes the Western 12-tone scale and (2) may serve as a tool that can be used to create annoying sounds for malicious reasons.

Team Status Report for 03/07

This week, the team pushed to freeze the design of, concretely modularize, and begin the implementation for the WaveShaper in time for the semester’s halfway point. 

Daniel finalized the system architecture for the hardware and external I/O. Jake began writing the hardware description for the ADSR envelope in the sound synthesis pipeline. Shayaan began designing and writing the mixing of oscillators. In addition, the team allocated their time towards writing a design report document, organizing system requirements, specifications, and development schedule.

These steps align with the proposed schedule, and the project is progressing smoothly. On that same theme of project management, the unfurling of the SystemVerilog hardware description of the modules in parallel represent a first for the team in implementing a design upon which we all conferred. Hopefully, this is a good trial for team coordination going into the latter half of the project’s development period and receiving feedback on the design report.

Each team member wrote upon the global, cultural, and environment factors and/or implications of the project in their respective weekly report. Jake Tarin wrote about the global factors (Part A), Shayaan Ghandi wrote about the cultural factors (Part B), and Daniel Abujaber wrote about the environment factors (Part C).

Daniel’s Status Report for 03/07

This week, I focused on fleshing out the final aspects of the communication architecture as it relates to the hardware. However, certain details cropped up which in turn informed the “data layer” and resulted in a change in previously made decisions.

Beforehand, I abstracted the model for USB communication on the DE10-Standard development board in order to focus on how to structure the “software side” of the USB-MIDI payloads. In doing so, I did not account for the fact that the USB hub which is downstream of the on-board USB controller presents a restriction in the topology as the USB layer sees it which clashes with the conventions used in standard MIDI devices.

For at the very least the purposes of debugging, I wished to leave headroom the option for a custom serial channel between the WaveShaper and a driving device. While I thought initially this could be achieved by the means of a USB Composite device driver, further inspection resulted in a satisfiability problem between the available USB ports, their direction, and how easily these could all be meshed with standard drivers.

Ultimately (along with consideration of the ease of development), I thought it would be best to compromise slightly on the original technical goals and introduce external hardware components which can ease this.

The final communication architecture resembles involves three streams of data.

  • Serial I/O:
    • WaveShaper interface software <-> FTDI VCP Driver <-> Host PC USB controller <-> FT232R USB UART IC <-> HPS <-> WaveShaper core on the FPGA (over AXI)
    • (The Host PC will register an FTDI VCP device seen from the converting hardware.)
  • USB-MIDI Host PC to WaveShaper:
    • WaveShaper interface software <-> USB-MIDI driver <-> MIDI-USB to MIDI-DIN converter (external hardware) <-> DE10-Standard GPIO on connected to FPGA (UART)
    • (The Host PC will register a USB-MIDI device seen from the converting hardware.)
  • WaveShaper to MIDI controller: WaveShaper core on the FPGA <-> HPS (over AXI) <-> USB3300 & USB251 stack <-> USB-MIDI controller
    • (The HPS supports Linux which has built-in support for USB-MIDI in the kernel https://github.com/torvalds/linux/blob/master/sound/usb/midi.c)
    • These decisions were made with the goal of maintaining a plug-and-play experience, pushing the now necessary converter to something that can be treated as inside the “black box” of the WaveShaper for an end-user, and all the while keeping an I/O interface which could be later developed and refined using specialized ICs such as the MIDI CH345.

On the topic of environmental factors:

While a musical synthesis tool and toolchain may seem on the surface to be benign, one must make the conscious effort to pay mind to possible ramifications of its use and implementation as a result of this fact. Modern electronic devices are all tied into a massive supply chain and manufacturing pipeline which involves the use of many resources and man-hours. At scale, the chemicals, their emissions, and even the choice of solder may present questions and concerns about their impact on the surrounding environment. While these cannot be quantitatively evaluated without a procedure and process, several of the guiding philosophies of the design of the WaveShaper align with a more sustainable future in electronics manufacturing and to a certain extent in computing in general.

The WaveShaper is a dedicated hardware synthesizer. Were the WaveShaper to be developed as a commercial product at scale, the consequences our choosing hardware which is targeted and strictly necessarily would reflect on the both the literal footprint of the PCB and the figurative environmental footprint. By not resorting to using an array of microcontrollers for I/O, synthesis, etc., we reduce the need to put the corresponding devices for this compute on each board. Likewise, by our efforts to keep the I/O as general as possible, this would prevent an end-user from having to purchase and use several adapter cables. (In the case that they would, the other side of the coin is that these common interfaces would be present in other devices they may choose to use in the future.)

In aggregate, these principles and their resulting design decisions not only target a low-overhead set of requirements for compute and processing power but also in the resulting devices which would need to be soldered, interconnected, tested, shipped, etc.