Weekly Status Reports

Team_D2_Cho_Dettmer_Yarlott_final_report

What did you personally accomplish this week on the project? Give files or
photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I’m doing a bunch of integration work at the moment. This involves interfacing with the CV code, which is mostly done since it’s just a matter of passing values and reading them. Once I actually know what the final form of the values will be, I can immediately pass them to my own code, in which case they’ll be passed to the Arduino. As for my Arduino code, it’s generally complete, though there are some serial bugs that I’d like to grind out still. One issue I had was with note deaths. In player mode, if a note was played and then immediately replaced with another note, it would be left on forever because of the nature of the player struct. Only one melody note can exist at a given time, and replacing it without culling the note leads to the note remaining on. I had thought this was being properly handled, but for some reason this was because of my ordering. At a high level, I read in data and then update the player struct and manage note times, etc. What I did when reading new notes was immediately setting the note age to maximum before creating a new note. My data reading is non-blocking, but for some reason the player update function wouldn’t work. So, I added it in the first conditional after receiving data saying that I’m reading a new note. This still didn’t work? Then, with only an inkling of intuition, I moved it from the bottom of the conditional code to the top, and then it worked. Notes were properly culled. Thinking back on it, the second code addition should’ve made it so that no notes could play at all, but this didn’t happen. If I were to look more into the issue, I’m sure I could figure out why exactly this was happening, but it doesn’t matter enough since the code works and I have more important matters to focus on. I’m also in the middle of changing the music generation code to create 8th notes instead of 32nd notes. This was really easy on the Python side, but the Arduino is taking a bit of time. This is definitely a load off of my back in terms of data transfer speed, though. For integration with the solenoids, I’ve confirmed with a simple LED array that notes are pushed to the correct pins at the correct times. I’ve also tested delay (without solenoid activation) and when using the Arduino Due’s native USB port, it’s incredibly low. I can’t say that it’s less than 10ms, but it’s low to the point where I can’t notice it with 16th notes being played at 90bpm in player mode. Player mode doesn’t have any sort of sequencer, so if notes are in time, that means that they’re being sent over at consistently quick speeds as well. There was some issue regarding a solenoid activating three times in quick succession every time the Arduino was either flashed, booted, or connected to via serial (which “sort of” restarts the Arduino). Apparently, on boot, the Arduino briefly sets all of its pins to INPUT, meaning that the connected pins are receiving floating voltages. This can clearly be seen when testing with LEDs, as they’re all dimly lit whenever any of the previously mentioned cases occurs. Although the solenoids don’t all attempt to activate at once, I think that must be related to every pin being set to input. When they’re set to their proper state at the end of startup, only pin 13 (the highest pin connected) would activate its solenoid. I am completely unaware of the circuitry on the Due, but maybe it’s related to some flow from surrounding pins being set to OUTPUT. I’m almost certain that if I hooked a measurement device up to the pins and ground I would see a maximum voltage that’s above the turn on voltage for the MOSFET. I’ve mentioned multiple times that a pull-down resistor between the Arduino and the MOSFET would probably solve the issue, so now I’m just waiting for that to be added to the circuit. Then, I can at the very least confirm or rule out whether or not these floating voltages are the cause. In the first place, if the pins are set to OUTPUT, and then immediately set to LOW during the Arduino’s setup() function, wouldn’t one expect them to not activate anything? Either way, once this issue is solved, complete integration is just a matter of hooking up wires to their respective pins. Oh, I also made the switch from the PWM pins to just standard digital pins. I didn’t bother confirming why but the PWM pins inconsistently powered the LEDs, causing them to flicker. The regular digital pins work fine, so that’s what I’m going with.

Also, just in case we lose any more solenoids, I added code that allows us to play MIDI notes only for specific pins. So, if the solenoid at pin 22 dies, I can reflash the Arduino with code that marks the pin with a -1, which can signal for the Arduino to play notes over MIDI instead.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Due to me changing up the minimum subdivision in my music code as per the recommendation during my presentation last week, I’m slightly behind. Plus, my robotics capstone demo is this Monday, so my priorities are a little shifted at the moment considering that system is far larger than this one, and there’s still a lot of final work to be done. However, I plan on completing everything for this well in advance of demo day, and there’s nothing in here that I haven’t done before, I just need to get it done.

What deliverables do you hope to complete in the next week?

I need to finish my planned code changes and that’s about it. Obviously there’ll be debugging involved with that, but I’m predicting that things will go smoothly. Integration should be easy enough, as the serial pipeline is already completed, meaning the most difficult part has been finished for integration. Now, it’s just a matter of passing values between Python code or sending levels to pins.

What did you personally accomplish this week on the project? Give files or

photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

Apparently there were some “issues” with previous status reports, so here’s an overview of my entire set of software that I’ve written, including things I’ve done these past two weeks. Strap in!

I played around with some new chord progressions and added them to the list of possible sets to choose from. These go into the generative framework and will be used to help guide generated melody notes along their path. Progressions are represented as the interval of their root note in the C major scale, so a C major chord is 1, an F major chord is 4, a G major chord is 5, and so on and so forth. The progressions are stored in a 2D array, being n*4 in size, n being the total number of possible progressions. These progressions were picked by hand for two reasons. I want them to resolve back to the 1 somewhat nicely, and I want to avoid strange repetitions or weird patterns that might sound unpleasant to users.

For reference, I made a slight change to generative mode, which hasn’t been mentioned in a short while thanks to dealing with embedded code. As stated, chord progressions are now picked from a list and the program itself isn’t allowed to generate them. Then, melody notes are generated on top of it. There are two parameters that decide what is generated. One decides the tonal mood while the other decides the rhythmic intensity. It should be noted that tonal mood isn’t necessarily saying that the notes should make you feel one way or the other, it’s just saying that the initial vibe of chord progressions and melody are either somewhat bright or somewhat dark (though dark can be hard to achieve in C major without non-diatonic notes). The tonal moods of chords are decided arbitrarily by myself. Rhythmic intensity defines how quickly the piece should be played. The program steps through each possible count, this ranges from 32nd notes to whole notes depending on the intensity, and chooses what count to play on. This is done mostly randomly, since the main randomness comes from what the input parameter is. However, it is also possible to decide on a rolling basis by taking the parameter to be the mean of a probability distribution and randomly selecting a rhythm at each step. If a parameter is centered at 0.5 in the range [0, 1], then it should mean that while half and quarter notes are going to be more common, there’s still a chance for more interesting rhythms to be created. Similarly, if the input parameter is 0.1, then there’s going to be a large number of 16th and 32nd notes (or vice versa, it’s very easy to flip). This process would be repeated for every step along each 32nd note, and after a note subdivision is chosen, then the program will move to the next available count until none remain. The current algorithm does this well. The actual note will then be decided afterwards. Fast rhythms will tend towards smaller changes while slower rhythms will tend to move more freely, though large jumps can happen.

The serial side of things just keeps growing in size, all in the name of optimization. I’ve said before that I’ve changed structs around, and that’s because it’s exactly what I’ve been doing. Frankly speaking, I don’t think that much more needs to be said about that, but it appears that verbosity is the norm rather than the exception. Luckily, I excel in being verbose the exact way I do in being brief. So, here’s an overview of just a few of the various struct changes that I’ve made in the past few weeks. “Oh, I need to know that a note is done playing so that it can be replaced? Got it.” Then I add a field to the phrase struct that’s an array of 256 booleans that determines whether or not a note has been played. At startup, this is marked as true so that any new incoming notes can be added. The struct that stores this is used for generative mode and stores the entire phrase. Then I realized, I can’t block anything, so I need a global variable that tells me what variable to play and when. So, there’s now a global index into this array. Of course, this still isn’t enough to know if it’s safe to change existing notes. Due to the nature of generative mode, new notes may be added before old ones get the chance to be played. However, we’re also working with musical phrases. There’s a few ways to do this. To preserve the current phrase being played, we can only change subphrases that aren’t active. This is just a struct field of booleans. If it’s safe, change it, if not, don’t. Another way to do it is to absolutely change everything on the fly, measure by measure. This involves a counter that marks what measure (of 8) is being played. Then, we pass that measure to a separate struct that goes into a non-blocking player function. We’re then free to change everything else about the music, from its phrase to its melody. Every measure, we take what we currently have, stop updating if really necessary, then pass that measure onto the player struct which takes everything and runs with it. This is explicitly separate from the player mode structure and instead runs off of a global timer that tracks the exact note location in measures and beats down to 32nd notes. It makes use of Arduino’s millis() function and updates every single time loop() is called. The length of 32nd notes is calculated either at the start of the file, or when the tempo is updated (which is a message that can be sent and will be expanded upon in a moment). The function tracks and says, “Ok, where am I?” and calculates its position in a measure based on the start time of that measure. Naturally, this means that there will be times when some notes aren’t selected exactly on their downbeat, but this is still functional enough, as 32nd notes at 120 bpm last for about 15.6 milliseconds. I don’t think the solenoids can handle that, so the tempo will be much slower. The one risk is in reading data. If reading takes too long, it might bypass some notes, which isn’t ideal. So, how can I combat this? There’s a couple ways. The first is simple, I start blocking. Now, I know what I said earlier about blocking and non-blocking, but hear me out. The note isn’t going to change every 32nd note most of the time, so there will be downtime where the pin can hold itself at HIGH for some longer period of time. I would simply account for this by adding a new structure that marks beats as available for updates. This is called at the start of loop(), and based on its result, new data may or may not be read. This seems to be the most simple, since it only involves adding one new array. In fact, I’m pretty sure it is the most simple. A small concern of mine is whether or not the serial buffer will clear itself by the time I get to it since the serial buffer’s size is only 64 bytes. If I send in new chord information, that’s 6 bytes sent in as fast as I can. If my generative phrases are sent in batches, I’ll be sending 128 notes plus a chord progression at around the same pace. That’s a whopping 519 bytes. So, if I start blocking, I might lose data. I could try to send confirmation back, but that just slows the pipeline down and makes everything worse. So, what to do? This actually relates to the second way I mentioned for reading in data without missing notes. One solution is to fill out a bunch of notes before they’re played, then start playing. This gives me a good bit of leeway to fill in new notes and fill notes in behind as well, if I manage to catch up (“What happens if I do catch up?” was one question I had but answered above with the boolean for notes that have been played). “What if you have so many 32nd notes in a row that you can’t read data?” If I get that many in a row, I’ll eat my shoe. The rhythms are based on a sample of a truncated normal distribution. Sure, it’s not impossible to get a lot of 32nd notes in a row, especially with the rhythm parameter maxed out, but the odds of getting so many in a row that I can’t even take a fraction of a second to read in data should be astronomically low. There is one more solution to the problem that I’m not particularly fond of but am willing to implement if necessary, and that’s limiting the speed of data transmission. This opens up a whole new can of worms. I would need to find a good balance between sending enough data that the Arduino can play something, and then sending it at a slow enough rate that it can read new data. The current solution I’m going for is just waiting for a while before starting. The mood is somewhat arbitrary, so it’s output only needs to generally follow the user’s movements. This is going well so far. I also implemented a MIDI framework that allows the arduino to directly interface with a DAW in case we lose any solenoids during our demo. This works great with player mode and just needs a tad bit of tweaking to with with generative mode.

Is your progress on schedule or behind? If you are behind, what actions will be

taken to catch up to the project schedule?

I’m on track. I just need to finish up generative mode’s serial framework and I’ll be good to go with both full testing and debugging.

What deliverables do you hope to complete in the next week?

The Arduino activates some solenoids on startup and connection and I need to diagnose and fix that. I also need to grind away any delays that might become a problem, either through optimization or changing the speed.