Author: Hailang Liou

Progress

This week I worked on finishing up the first versions of effects modules. At this point, the distortion module is essentially completed, while the delay and reverb modules should be finished up this weekend. These first versions will have the necessary functionality, but are not tweaked for sounds, which is something that will be worked on in the upcoming weeks. Unfortunately, I cannot really test the effects in real life until the rest of the synthesis pipeline is in, however, I may consider using Matlab to run some simulations to fine-tune some of the effects parameters.

Realizing that the midterm demo is soon upon us, I have also spent a lot of this week revisiting the MIDI interface and decoder parts of the pipeline, since that will be critical for the midterm demo. The parts I need for the MIDI interface were obtained this week and after some simulation testing of the interface this past week, it should just be a matter of putting the pieces together.

Scheduling

While I was able to work over Spring Break, I did not end up having as much time as I anticipated, thus I am not as far as I would like to be. The goal right now is to be in a presentable state for the midterm demo, which I think will be alright, and I am potentially going to be taking over some parts of the synthesis pipeline from Jens in order to get the core of those modules absolutely nailed down. Next week, we will be spending a lot of time doing integration and integration testing, but hopefully, any issues we iron out next week will be mean that integration later will be trivial, as only the internals of the modules should change and not the interfaces between them. For the effects and such, I will begin functional testing, but tweaking and fine-tuning will likely wait until the week after.

Weekly Progress

The beginning of the week was spent busy polishing up the design review report and integrating everybody’s parts together. We had all written our sections independently and consolidated in the end, since it was difficult to work on it collaboratively with the formatting. I worked on filling in extra details on some sections, and also worked on formatting the final report (formatting in Word can be a pain).

Unfortunately, I think that most of our group got caught in a slew of midterms this week which ate up a lot of our available time. We worked as a group to flesh out the interfaces between a lot of our modules this week, and we each planned out our goals on work over Spring Break.

This week, I worked more on the Verilog implementation side as well, although less than I would have liked. Implementation of both the actual modules and the testing framework is underway now and I don’t anticipate there being any major issues in terms of scheduling.

Goals for Next Week

Next week is Spring Break, and I have very little planned, so I will try to get as much of the Verilog implementation done over the break. I have a small electronics lab setup at home, so I will see if I can run some tests on both the MIDI interface components as well as try to test my personal synthesizer for benchmarking purposes. My main goal will be to finish up the implementation stage for effects and begin the testing and fine-tuning steps.

Weekly Progress

I didn’t get a lot of time to work on the actual meat of the project this week, since a lot of the time I had was focused on the design review presentation as well as on the design review paper. The design review presentation went very well, and I am happy with the feedback we got. Through the course of preparing the presentation and the paper, I have fleshed out the designs of the effects modules and have developed block diagrams for the effects modules, as well as working with the other group members to develop the flow of the whole effects pipeline. Thus while the actual implementation did not make much progress this week, I hope the time spent designing the effects modules will prove to save time in the implementation and debugging stages later on.

This week I also got to spend a little bit of time messing with the MIDI keyboard I received last week, and it turns out that it should work well with the project. I found a MIDI cable that has a detachable head and exposes the wires, which will help a lot when wiring together the support circuitry. Unfortunately, I am currently missing a part that I would need to create the complete circuitry, and I am waiting on the rest of the analog parts list to be fully fleshed out in order to save money on shipping ($10 shipping on a $0.80 part?). However, if the parts list needs more time, I will just place the order early this week anyways and just eat the shipping cost into the budget, as we don’t have a lot of parts we need to buy, so we should have plenty of budget to spare. In other news, I also recently purchased a nice synthesizer for fun, and for the project, we can use it as a comparison for testing purposes. Hopefully, we can get our hands on some other commercial synths in the coming weeks so we have more to test against.

Goals for Next Week

Due to us not really anticipating the amount of work that needed to go into the design review, and it not being properly allocated in the schedule, I am probably a little behind in my implementation goals for the effects modules. However, due to our built-in slack, I will be able to pick up the pace of development of the effects modules. I would like to get at least bare-bones versions of the effects modules finished by this week and begin developing a testing framework for the effects pipeline.

I spent a lot of this week doing experiments with FPGA usage and deciding the best way to design both the wavetable and the delay/reverb modules. In this status report, I will describe the experimentation done, the process this week of designing some of the effects modules, and some miscellaneous items as well.

Wavetable Experiments

I started out the week working on wavetable size experiments, something I wanted to get done as early as possible (as mentioned in last week’s report). Since we wanted to avoid putting the wavetables on a memory external to the actual FPGA chip, such as SDRAM or flash, I wanted to see what the FPGA usage would be if I just put all the wavetables in the FPGA LUTs themselves and create a big combinational lookup table to act as the wavetables. Jens generated a 12-bit, 512 sample sine wave to test this idea with, and the results were a lot better than I thought! It turns out, a single one of these tables uses about 3% of the available LUTs on the smallest Terasic DE0 board (Cyclone III) used by the old iterations of 18-341. I tested with additional tables present, and the number of LUTs used scales linearly. In general, it seemed that the usage was approximately 1 LUT per 12-bit sample. However, I was recommended by Prof. Mai and Ford Seidel to explore using the FPGA’s block ram for wavetable storage instead and it seems like this will be a good solution as well. One big advantage of using the block ram is that the Quartus generated block ram modules are two ported, which will make Jens’s life a lot easier down the line when he makes the wavetable access modules. However, one downside that worries me is that using the block ram will make our design board dependent, since different boards will all have different amounts and types of block ram. While this isn’t a major problem, I had wished the design could be as portable as possible. We don’t anticipate needing to do a radical board change and any board changes will be all in the same Altera Cyclone family.

Reverb and Delay Design

I also started designing the reverb and delay modules this week. The two effects are very closely related at first glance, but are actually quite different in the nitty-gritty. Both involve creating a delay queue where the samples will pop out some number of cycles after the sample is played normally, however the queue for the reverb module will feed back into itself after being attenuated to create an echoey effect and will also be shorter than the queue for delay. Unfortunately, since the sample rate for standard audio is 44.1kHz, we will need to queue up 44,100 16-bit samples in order to create a one second delay. As with the wavetable experiments, I tried to create a quick prototype module in Quartus to see the FPGA usage of a naive implementation of the queue, but it ended up being completely crazy and infeasible (also learning that Quartus will only allow a generate loop to go 5000 iterations before dying). A 16-bit sample queue with a queue depth of only 4096 elements alone will take up 65k logic elements on a board, almost 5 times the amount on the Terasic DE0, and this queue is only big enough for less than a 100ms delay! I was again worried that I would have to deal with the on-board SDRAM and would need to hack up a controller, but luckily, it seems like block ram will come to the rescue again. The Quartus Megafunction wizard will let you synthesize a FIFO structure on block ram, which was awesome to find out, but unfortunately, a 16-bit FIFO of length 2¹⁶, a queue depth that would allow for a maximum delay of a bit less than one second, would take up 64 M9k block rams, whereas the DE0 board only has 56 such block rams. I don’t see an alternative path to implementing delay with any reasonable maximum delay, so this will probably force us to switch from the DE0. Luckily, we already had plans in place to switch to the Terasic DE0-CV, an updated board that is almost identical except for using the Cyclone V FPGA instead of a Cyclone III. The new FPGA has 308 M10k block rams, about 6 times the amount as the old board, while only being a little more expensive as to not increase the total project cost by too much. This has the added benefit of allowing us more block rams to add more wave shapes later on, something that will not take a lot of time or be too difficult, while make the user experience of the synth significantly cooler. In addition to nailing the exact implementation of the queues for the delay and reverb effects, I also got around to designing the other parts of the effect.

Miscellaneous

Finally some other miscellaneous updates. I put in the purchase order for the MIDI keyboard on Monday after lots of agonizing over it being the first purchase of the project and received the keyboard on Thursday. Unfortunately I don’t really have a good place to put it yet so it’s being hidden in the capstone lab. I plan on running some tests with the keyboard, and try interfacing it with the FPGA early next week. A lot of time this week was spent working on the presentation and documentation rather than on implementation, but I’m not too worried since a lot of the planning for future steps was hashed out this week as well. Running the tests helped lock in a lot of important design considerations for the modules that needed to be done, so overall things are running pretty smoothly.

Goals for Next Week

For next week, I aim to finish the complete design of the reverb and delay modules, and hopefully have a Verilog description mostly or completely written. I want to start testing the MIDI interface between the keyboard and the board as well. Since the comprehensive design review report needs to be done next week as well, I will not expect all of these goals to be achieved but I expect them to all be well underway.

This week, we have started diving deep into the design of the project. I worked mostly on the design of the MIDI – FPGA interface, but I also spent a considerable amount of time helping design the wavetables, something not in my schedule.

I researched the MIDI specification and determined the best way to connect the keyboard to the FPGA. While most modern MIDI keyboards send messages over USB, I ultimately concluded that time spent designing a USB controller or fiddling with a USB IP block would not be the most productive use of time and decided that the alternate serial MIDI connection would be better suited for the project. Unfortunately, running with the serial connection has significant downsides. First, the serial connection runs at only 31k baud, which means a full MIDI message will take almost 1ms to send. Second, the MIDI spec requires some analog support circuitry on the receiving end, before the signal can be received by the FPGA. There are examples of such circuitry for use with Arduinos online, but we will need to run tests and tweak the circuitry to work with the FPGA. Finally, not all MIDI keyboards have a serial out and I have spent some time looking for a suitable keyboard online that is both cheap and includes enough knobs to control the effects. My aim is to make the purchase order by Monday. Fortunately, the message will be sent over UART, which will be fairly simple to implement in Verilog, which I have begun doing.

On the wavetable side, we have been discussing within the team what way would be best to store the wavetables. Ideally, the wavetable would have enough samples to minimize interpolation, but having at least 4 wave shapes means the memory usage of the tables can grow to be quite large. We have been discussing and researching several options: putting it on the SDRAM (8MB available), putting it on the flash memory (4MB), or putting it in LUTs on the FPGA itself (15k LUTs total). For the SDRAM and flash memory, we would have to either find a memory controller or write one from scratch, with the SDRAM also having the additional requirement of reloading the tables onto the board every power-on. We would vastly prefer not to write one from scratch, but we are still unsure about whether we will be able to find a suitable memory controller elsewhere. I have been researching options and have been looking for memory controllers given by the FPGA board maker, and have possibly found one that could work. More details about this debate can be found in the team report.

I am a little behind schedule since I had wanted the whole interface to be near completion by the end of the week, but I think after overcoming the design problems that I fleshed out this week, implementation will be straightforward. Next week on the schedule is built in slack for me, so I don’t need to take any special actions to catch up in my opinion.

For next week, I want to have be able to have the interface fully implemented and tested. I am unsure about whether I can get the MIDI keyboard shipped here by the end of the week, so I was planning on using Jens’s keyboard to test the circuit and UART receiver. I also want to run some experiments about wavetable sizes in Quartus and conclusively decide on what way to store the wavetables in the board.