This week we wrapped up the final presentation and our integration with different components of the system. the integration went relatively smoothly for my software component with the raspberry pi part. The entire system seems to able to come together and function properly.
We still need to do more testing with playing the keys on the keyboard and monitoring the final sheet music generated on my end, which we have yet to complete.
After contacting Tom Sullivan, we made some pivot in my part to transition into generate the sheet music at the end instead of real time, as there is no point in trying to generate the score as someone is playing something they already have in mind. I am doing some connections between my original code with the new interface that we decided to use.
I also realized that my original implementation of the algorithm does not take into account of the fact that there could be rests in music. This is quite important as notes are not necessarily continuously out in a piece, and there is inevitable some rests in the music. So I am modifying my algorithm to take that into account.
At the start of the week on Sunday, I worked with Sherry and Olek on the final presentation slides. On Tuesday Olek and I figured out a way to mount the force sensitive resistors on the inside of the keyboard such that the user can play the keyboard normally without worrying about exactly hitting the sensors. I ordered clincher connectors to wire up the sensors, as it is not advised to solder directly to the tabs. These connectors are a bit finnicky, however, so I spent some time improving the connectors’ connections to the sensors. Afterwards I mounted the sensors to the underside of the keys and routed the wires out the back of the keyboard through a hole that Olek and I drilled previously. Lastly, Sherry, Max and I worked to write up the final poster PPT on Thursday.
For the upcoming week, I will work to prepare for the final demo on Monday. I still need to clean up the wiring from the sensors to the PCB as well wire up a start, stop, and pause button on a breadboard which shouldn’t take too much time. I will complete these tasks tomorrow (Sunday). I believe I am on schedule but will start these tasks early afternoon tomorrow in case anything goes amiss. Lastly I will work with my teammates to write the final report due next Friday.
This week Jeannie reflowed reflowed the PCB and validated its functionality. Olek and Jeannie worked to integrate the PCB with the Raspberry Pi and Olek was able to use his code to successfully read packets off the I2C bus from the PCB. Sherry made progress with the API, and it is now able to display notes, dynamics, and more. Over the next week, our group will work on the final presentation, final poster, and work on completing integration. Olek will work to read I2C data from all channels of the ADC’s, Jeannie will validate the metronome feature on the PCB, and Sherry will work to finalize the API.
This past week since coming back from Thanksgiving break, I received all the parts necessary to reflow my sensor data collection PCB. I spent three hours on Tuesday and about an hour on Wednesday reflowing the board as well as soldering on the through hole components. I realized that I had the wrong package for some resistors so I found alternative resistors of the same value and reflowed the board. I also ended up needing to rework the board when I realized all of my diodes were backwards and some parts weren’t making good contact with the metal pads. On Thursday I finished reworking the board and validated that the power conversion circuitry and the rest of the board worked well aside from the ADC I2C communication. I am using an 8 channel 12 bit ADC I2C IC that takes in 8 channels of analog data and outputs them over I2C over the SDA pin. Due to this, I needed to collaborate with Olek to validate the functionality of the chip with the Raspberry Pi that requests data over I2C.
On Friday night I met with Olek for a few hours and the two of us attempted to bring up the I2C communication between the sensors and the Raspberry Pi. We were using the oscilloscope to view the SDA and SCL lines, but did not find it particularly useful or insightful as we were seeing weird behaviors. We met again on Saturday for another couple hours and were able to establish communication over I2C between the PCB and the Raspberry Pi! I think it was because the Raspberry Pi GND and the PCB’s digital ground were not coupled, so the I2C communication wasn’t being established properly. I also validated that the buzzer emits a tone when 3.3V is applied. The last thing I need to do is validate the binary coded decimal rotary switch works correctly, which shouldn’t take long. Currently Olek is working on how to request data from all 8 channels of the ADC and how to generate the PWM signal for the buzzer using the binary coded decimal rotary switch position.
For the upcoming week, I will work on the final presentation slides for Monday as well as work on the final poster due on Thursday. Additionally, I will work to mount the sensors onto the piano keys before Monday in order to take a nice video for the final presentation 🙂 Lastly, I want to validate the rotary switch function and replace some resistors on the PCB such that the indicator LED’s for the key presses are more bright. I believe I am exactly on schedule and can finish my tasks before the final presentation on Monday.
Over thanksgiving, I found out that the mouse click processed based on the location on the screen. While it is possible to hardcode some of the buttons we want with mimicking a click at a specific location, the sheet music rescales as more notes are being added, so it become essentially impossible to hardcode that. After discussing this issue with the TA and my teammates, we found out that MuseScore has built in keyboard shortcuts that I can use.
This week I looked into how these keyboard presses are implemented, and find out they are implemented through the Qt package that I installed to run the application with. I read the official documentation because unfortunately the application does not make the function call to the function I want to be calling in my code, and the package all come in executable format. I was able to find out how to call the function, but is unable to verify if that is correct yet.
I finished implementing the algorithm part in my code, and tested it with some sample test cases. It now outputs a line saying which note, note type (e.g. whole) is being pressed for (lightly, medium, hard) on command line, and this also gets written to an output file in real time so the data can be saved.
I had a meeting with Olek where we integrated our separate parts of the code together, that went relatively smooth and we can now read in inputs from the pi (currently randomly generated) and convert that to outputs in the command line through my code.
I tried to compile my part of the code with the application, but the application compilation is too complicated and I have not made significant progress on how to modify the make file to make everything integrated correctly.
Next week I will be focusing on working on the final presentation (on Sunday) as I am the main presenter for this presentation. I will also keep working on integration with the MuseScore application, if not trying to rescope the project for an alternative solution. I am not as on schedule as I would like as the application turns out to be much different than I expected, but I am relatively happy with what I have so far in terms of having the code I wrote working correctly.
This week I made a lot of progress with the Raspberry Pi communicating with the PCB. Now that we have all the project components, the Pi is able to communicate very well with the PCB, and able to access all the sensor channels very quickly. What I plan on doing this week is incorporating a PWM output signal from the Pi to the buzzer on the PCB to give the user auditory feedback on the metronome tempo. Apart from that, I plan on helping my teammates with their tasks to make sure we finish the final project on time for the demo.
I’m not sure if a report is due for this week. I wanted to post one to make up for the ones I missed while I was away. This week I spent working on the raspberry pi code. I am still waiting on the PCB, so I fully fleshed out the code needed to talk with the local software on the computer. All that we need now is the PCB to correctly converse with my i2c code, and my project portion of the mvp should be met.
This week I focused mostly on trying to find where to inject our code section into the API. This ended up being much harder than I expected as they do not have very well written documentation online, the documentations I was able to find were very random, and either contained no useful information or just outdated and wrong information. It took me quite a while to figure it out but I ended up finding where to add our code to. Next week I am hoping to be able to run the system on some sample inputs to display them on the screen.
This week we made progress around the board towards our MVP. The PCB has been ordered and should arrive sometime early next week. Furthermore, the raspberry pi/pcb interface has been modeled using an Arduino in order to support further development while the pcb is being delivered. This allowed us to find that the i2c libraries available on the raspberry pi won’t meet the standard that we need to communicate with the PCB, so a standalone library is being written. Lastly, we found where we needed to inject code into the Muscore library in order to visualize the notes, progress should be made this week.
This week I made a lot of progress on the raspberry pi I2C sending and recieving. Due to our PCB board being delayed, I used a spare arduino that I had on hand to act as a PCB emulator, which allowed me to put together a model of the PCB, and verify that the raspberry pi i2C sensor polling is working as expected.
Jeannie and I had to put together 2 bi-logic converters in order to support the 3.3v and 5v difference of the arduino/rasperry pi, but things worked as expected.
However, in the process I found out that the existing raspberry pi i2C libraries don’t meet my specifications for communicating with the ADCs that will be on the PCB, so I’m writing my own i2C master interface.