Author: dabujabe

This week, we received all the necessary cables and adapters to begin synthesizing and testing functionality on the WaveShaper hardware on Friday. I began a Quartus project and have started integrated functionality, drawing from the selection of IP blocks and beginning a scaffolding to support the core modules.

As of right now, the most work has been in synthesizing and creating SystemVerilog “HDL handles.” There was some additional delay in finding out how to use IP blocks in the Quartus software.

I (with the help of Jake) was able to perform a simple end-to-end test of the toolchain, ensuring that we are able to program the hardware.

Otherwise, I considered how the WaveShaper both (1) has been designed such far in a way that presupposes the Western 12-tone scale and (2) may serve as a tool that can be used to create annoying sounds for malicious reasons.

This week, I focused on fleshing out the final aspects of the communication architecture as it relates to the hardware. However, certain details cropped up which in turn informed the “data layer” and resulted in a change in previously made decisions.

Beforehand, I abstracted the model for USB communication on the DE10-Standard development board in order to focus on how to structure the “software side” of the USB-MIDI payloads. In doing so, I did not account for the fact that the USB hub which is downstream of the on-board USB controller presents a restriction in the topology as the USB layer sees it which clashes with the conventions used in standard MIDI devices.

For at the very least the purposes of debugging, I wished to leave headroom the option for a custom serial channel between the WaveShaper and a driving device. While I thought initially this could be achieved by the means of a USB Composite device driver, further inspection resulted in a satisfiability problem between the available USB ports, their direction, and how easily these could all be meshed with standard drivers.

Ultimately (along with consideration of the ease of development), I thought it would be best to compromise slightly on the original technical goals and introduce external hardware components which can ease this.

The final communication architecture resembles involves three streams of data.

• Serial I/O:
  • WaveShaper interface software <-> FTDI VCP Driver <-> Host PC USB controller <-> FT232R USB UART IC <-> HPS <-> WaveShaper core on the FPGA (over AXI)
  • (The Host PC will register an FTDI VCP device seen from the converting hardware.)
• USB-MIDI Host PC to WaveShaper:
  • WaveShaper interface software <-> USB-MIDI driver <-> MIDI-USB to MIDI-DIN converter (external hardware) <-> DE10-Standard GPIO on connected to FPGA (UART)
  • (The Host PC will register a USB-MIDI device seen from the converting hardware.)
• WaveShaper to MIDI controller: WaveShaper core on the FPGA <-> HPS (over AXI) <-> USB3300 & USB251 stack <-> USB-MIDI controller
  • (The HPS supports Linux which has built-in support for USB-MIDI in the kernel https://github.com/torvalds/linux/blob/master/sound/usb/midi.c)
  • These decisions were made with the goal of maintaining a plug-and-play experience, pushing the now necessary converter to something that can be treated as inside the “black box” of the WaveShaper for an end-user, and all the while keeping an I/O interface which could be later developed and refined using specialized ICs such as the MIDI CH345.

On the topic of environmental factors:

While a musical synthesis tool and toolchain may seem on the surface to be benign, one must make the conscious effort to pay mind to possible ramifications of its use and implementation as a result of this fact. Modern electronic devices are all tied into a massive supply chain and manufacturing pipeline which involves the use of many resources and man-hours. At scale, the chemicals, their emissions, and even the choice of solder may present questions and concerns about their impact on the surrounding environment. While these cannot be quantitatively evaluated without a procedure and process, several of the guiding philosophies of the design of the WaveShaper align with a more sustainable future in electronics manufacturing and to a certain extent in computing in general.

The WaveShaper is a dedicated hardware synthesizer. Were the WaveShaper to be developed as a commercial product at scale, the consequences our choosing hardware which is targeted and strictly necessarily would reflect on the both the literal footprint of the PCB and the figurative environmental footprint. By not resorting to using an array of microcontrollers for I/O, synthesis, etc., we reduce the need to put the corresponding devices for this compute on each board. Likewise, by our efforts to keep the I/O as general as possible, this would prevent an end-user from having to purchase and use several adapter cables. (In the case that they would, the other side of the coin is that these common interfaces would be present in other devices they may choose to use in the future.)

In aggregate, these principles and their resulting design decisions not only target a low-overhead set of requirements for compute and processing power but also in the resulting devices which would need to be soldered, interconnected, tested, shipped, etc.

I spent the first half of this week preparing to present the design report on Wednesday.

After that,  I took time to revise the Intel documentation for the IP blocks available in the Quartus software tool chain. By taking advantage of the already existing IP blocks for the audio codec, DAC control, and DSP, it may accelerate the interfacing endpoints of the WaveShaper to get the project to a demo-ready stage. I am not familiar with using IP blocks under the Quartus toolchain, so I needed to familiarize myself with the general process and set up a project file that could be used to program the fabric.

We have all the tools required to attempt to create an end-to-end test and produce a series of notes from Shayaan’s oscillator hardware descriptions. Namely, using the audio “stack” of blocks to ultimately drive the Wolfson audio chip.

Moving forward, I would like to use the HSP chip to create a interface to receive USB payload data and pass the wrapped MIDI commands to the main control module of the WaveShaper.

This week, I continued to explore options for communication between the driving PC, the WaveShaper, and additional MIDI peripherals. Earlier this week, I had considered and sketched out a protocol wherein all auxiliary (i.e., non-MIDI) information would be transferred within the SysEx commands of the MIDI spec. While this is still feasible and should be able to keep up for live, real-time music, the USB specification enables the use of “composite devices” which support multiple, simultaneous interfaces. This could lend itself to more flexibility and warrants more analysis before setting in the serial communication in stone. From an ease of implementation point of view, it would have the advantage of having convenient parallelism between these conceptually disjoint streams without the need for additional hardware or cables.

(This design decision was prompted with the choice for the DEC-10 FPGA in mind, seeing as it supports both USB host and device interfaces.)

We would want to consider support the following serial streams.

• PC to the WaveShaper for MIDI notes and events.
• PC to the WaveShaper for a “shell”-like command interface.
• WaveShaper to PC for probing into the sound synthesis datapath.
• WaveShaper to PC for responses to “shell” commands.
• [Optional] A final USB audio input interface which can enable the PC to be used as a speaker / audio sink.

These could all be interfaced through dedicated, corresponding USB device classes.

While it would be incredibly helpful to offload the multiplexing to the USB hardware and controllers, I would want to revise the hardware we are deciding on more thoroughly before making such a jump. That said, the option remains to merely implement the first MIDI with SysEx interface (which would be in line with the original plan).

This week, with the finalized proposal in hand, each group member focused on completing specific details of the projected system implementation. After a group meeting, we divided responsibility for the next set of design decisions to be made in parallel.

I spent time researching both the MIDI and USB protocols to address the following question:

Suppose we want to interleave standard MIDI note events with specialized, device-specific serial commands over a single USB connection (e.g. requests for auxiliary data, control commands for sound synthesis parameters, and FX tuning). What is the best method to achieve this?

While it would be feasible to draw the protocol directly from prior MIDI instrument designs, our anticipated data throughput is likely to present a more intense requirement. In particular, we expect to transmit multibyte sequences that may need to be information-dense enough to define complex effects.

The MIDI 1.0 specification provides a System Exclusive (SysEx) message format that allows arbitrary, “manufacturer”-defined data. As a result, it appears advantageous to “flatten” the interface by embedding our own messaging protocol and custom commands within these SysEx messages, thereby maximizing compatibility with existing MIDI devices and software. It would be much easier to find conformant MIDI streams than have to test and integrate a custom protocol that wraps around the MIDI layer.

While the project is thus far in a draft stage, I believe the emphasis on design decisions and FPGA architecture are paving a clear road ahead. That said, to put rubber to the road, I would like to ensure that skeleton drivers for what is to be the user-facing desktop software are ready by this time next week or (at least near completed).