Posted on

Rahul’s Status Report for 4/8

This week I was able to accomplish end to end connection of the accompanyBot with all of my team members. This was very exciting! I also discovered a bug while doing this. If the user opens the insert file menu while Audiveris is processing another file and then hits cancel on the new file menu the program crashes. This has been resolved now. Another issue that I have been aware of but cannot resolve is if Audiveris is busy processing and the user wishes to cancel the process or just decides to close the app preemptively. The Audiveris process dialog will continue running until it fully finishes. Considering that this OMR software is built upon very old toolchains and I already tried many options to get it configured for MacOS at the beginning of the semester, I will not dig into how I can send it a SIGKILL or similar termination signal. The easy work around is just to close the window that appears which terminates the process manually.

Earlier in the week, the interim demo took place. After showing the application, my team’s course advisor Tom suggested that I should add a digital player to demo the parsed music as a midi output before the accompanyBot starts to play. I was able to address this suggestion and implemented the digital player which converts the xml into midi first and then begins playing an endless loop. This loop can be paused and unpaused via the spacebar key.

Here is a demonstration of it playing and pausing (there should be audio playing along with the video)

In terms of completing my side of the serial communication I set up sender code and patched up the receiver thread code to get measure data from the notes scheduler running on the RPi. I also eliminated one piece of data that was initially being sent by the RPi. Previously, the number of measures in the song was getting determined by music21 and sent over the serial connection to the GUI app. I decided to process the number of measures algorithmically from the raw xml structure.

Overall, the project is in a good position. I was slightly worried last week that I would get behind, but after getting things working I believe I am on schedule again. Going into next week, I will have some more time to confirm the workings of the communication between subsystems. In particular I need to set up the ssh command to deliver the xml file from the GUI to the RPi. As mentioned earlier, this is necessary to avoid the one-time heavy load over the serial connection. If this gets done quickly then I can begin stress testing early and identifying problems that the GUI or notes scheduler may need to have fixed.

The tests on the application side will involve many different variations of quality of music inputs, randomization of button presses to break application, manual disconnection of serial cable while app is still running (I think I tried this already, and since I’m using threads for communication, they terminate with an error printed to the console, but the main app thread keeps running fine, so this is not unreasonable behavior I suppose), etc. I anticipate the app to crash when tests fail and will analyze the error logs closely to determine the point(s) of failure. Also, I will try to perform actions that spawn and terminate different threads at a time while checking the task manager to make sure all python processes get cleaned up when they should. Part of this effect on the use case requirements is to maintain the functionality and latency requirement between subsystems. In terms of design, the GUI should be user friendly. So any odd errors that prevent intended usage must be addressed.

 

Posted on

Aden’s Status Report for 4/8

Heading into the final weeks of the semester, and thus this project, I have run one primary test for my part of the project, and that was ensuring the power consumption of the circuitry does not exceed 60 W. This goal has been met since we limited the number of solenoids that can be turned on at once to five, since each solenoid draws about 0.9 A and uses 12 V this requirement has been met and tested. Additionally, if this proves to be troublesome in the coming weeks, the voltage can be lowered, reducing the current since each solenoid doe not need 12 V to operate. I will also be helping measure our maximum tempo since the circuit will be a limiting factor due to its physical characteristics. Hopefully, we will be able to meet our preset quantity, but that is probably unlikely.

In addition to the interim demo, the image above is what I have accomplished this week. I realized that there were a couple of issues with the previous version of the chassis design. One of the major issues was that the black and white keys were not actually aligned properly in my previous design. Another major issue was that we wanted to allow the system to be placed over any octave on the keyboard, but after some more careful measurements, I realized this would not be possible for what we had in mind, so I had to redesign the chassis nearly from scratch. Additionally, the previous design was under the assumption that we would just secure the solenoids to the inside of the chassis, but to plan for problems arising for demo day, we switched to a design that would allow for solenoids to be replaced easily. Overall, I believe this is a much more robust and easy-to-use design, and hopefully, it will be suitable for demo day.

I am still technically on track, but I would’ve liked to 3D print a prototype tomorrow, and I’m not sure if I will be able to since I am currently having difficulties with Solidworks right now. Hopefully, I will be able to figure out my problem with enough time to 3D print tomorrow, but if I don’t, I will try to print something out Monday. I hope I can print it out tomorrow, though.

Next week I hope to have a prototype of the chassis completed, and if it does not function properly, I hope to update the design and print another one before the week ends. The end of the semester is coming soon, so I need to finish this by the end of next week.

Posted on

Team Status Report for 4/1

For our interim demo next week, we will demo the working parts of our subsystems. We currently have a functional sheet music parser and UI on the user input side. We also have the scheduler which can actuate solenoids based on a manually inputted XML file. Our main efforts tomorrow will be focused on integrating these two subsystems, although we cannot guarantee that it will be completed by our demo next Wednesday due to reasons outlined below.

This week, we encountered a roadblock when trying to integrate the Raspberry Pi with the local application. Our team did not foresee the RPi serial communication to the computer requiring extra hardware to convert from USB to UART. Hence, we were not able to make much progress and thus may not have a working integration between the GUI app and the hardware controller come interim demo. We are going to offset the risk of ordering and dealing with new parts by adding an Arduino MKRZero to our system design as a middle controller to pipe information between the computer and RPi. This could pose a risk by increasing the overall latency between app and solenoids. We are accepting this risk with the knowledge that the typical payload of information traveling should be very small, <10 bytes. Additionally, we discussed previously the usage of pySerial library for communication over usb. Python comes with a serial library that has a simpler syntax and seems to be meeting our system requirements. So for now, we are making this minor framework switch.

Next week we will be going into our planned slack time, so no changes will be made to the schedule. All remaining tasks deal with integration so we will be working on integrating pieces of the software to controller to hardware in descending order of urgency.

Posted on

Nora’s Status Report for 4/1

This week, I worked with Rahul on integrating the Raspberry Pi with the computer’s local application. Since the computer will be sending and receiving measure numbers as points of reference for where in the piece to start playing and what part of the piece to display on the screen, I had to restructure the code so that the measure numbers were the keys in the dictionary that holds all the notes. This would make it easier for the RPi to look up which notes to play at specific times. With Rahul’s guidance, I re-implemented the scheduler to be compatible with this data structure change. Thanks to Rahul’s suggestion, I also changed the code so that the RPi loops continuously while checking the system time instead of sleeping unnecessarily as was done in the previous version. I was able to alter the code and get it back to a working implementation.

Another area I worked on was introducing the serial communication to the Raspberry Pi. As mentioned by Rahul, we did not have the required USB to UART converter that would allow us to interface with the RPi’s TX and RX pins. To avoid the extra cost and delay to our schedule that would result from purchasing an adapter, we decided to use Arduino microcontrollers that have built in serial ports. In tandem with Rahul, I experimented with using the Arduino MKRZero which has two serial ports: one between the Arduino and the computer and another to the on-board TX and RX pins. By using these two pins, and having the Arduino read the data from the USB and write to the TX/read from the RX and write back to the USB, I was able to set up the system to send and receive strings between my computer and the RPi. While one risk of this implementation is that there may be added latency, the fact that the serial communication is buffered should allow the Arduino to pass the information through at roughly the same baud rate. Tomorrow I will write a more comprehensive test using the system time to find the round trip latency between sending bytes from the computer and receiving a response from the RPi.

At the moment I am behind schedule since we have not completed the integration. However, we did anticipate that this part of the project would take more time and had incorporated slack time for it accordingly. Thus this is on schedule when considering that slack time.

Next week I will focus on further developing the serial communication. I will be meeting with Rahul to agree upon the command bytes that will be sent for various situations that are required by the application.

Posted on

Aden’s Status Report for 4/1

This week finally got around to playing with Solidworks. I was able to generate a basic rendering of what we had in mind for the chassis design. I am still working out how much wiggle room we have between solenoids, but otherwise, I should be set to 3D print soon.  Unfortunately, I am not finished with the Solidworks rendering and would like to add some additional features to it, so I will focus on that tomorrow. Here is the 3D rendering that I am currently working on:

From my last status update, I am more or less on schedule. I was not able to get a copy of Solidworks until yesterday, and even then, I had to get a new license from the software catalog and sit through an hour of download times. Tomorrow, however, I will be cleaning up the rendering and adding some features to the design to hopefully improve the overall performance of the accompanyBot.

In addition to the work I will be doing tomorrow, I also hope to figure out how to 3D print, how expensive it is, and how to generate the files necessary to print it out. I also hope to help Nora and Rahul with the serial communication if they do not finish it before the week ends.

Posted on

Rahul’s Status Report for 4/1

Early in the week I worked with Nora to devise data structures for our notes scheduler. The primary functionality we determined would be to take a measure of notes at a time and play through them sequentially unless it receives a measure change or pause signal from the app. I then proceeded to try and work out the communication methodology between the RPi and Windows computer. Unfortunately, my team did not have a usb to usb-c cable on hand, so I placed an order request for one. While waiting, I was able to borrow one from Alex, my group’s TA. I was able to connect the RPi to the Windows computer, but this amounted to nothing since the RPi is not a peripheral and does not get recognized immediately. We would need to order a UART to facilitate the data communication. To avoid ordering more parts and having to deal with UART, I decided to just plug in an Arduino into my laptop and test serial communication from python to the Arduino. I had to relearn some of the arduino intricacies but got a reasonable proof of concept working.

The accompanyBot GUI application will have to spawn a thread from the start that communicates with the hardware. This communication thread and the app’s main thread should be allowed to modify specific mutable objects such as the playing state variable or the current measure number. I ran a test to verify this functionality and it was successful .

In the print log from the test, the list [“nothing”, “here”] gets modified to [“winner”, “here”], by the thread doing communication the data “winner” also gets received by the arduino which was configured to only send this data if it received the “hello mr arduino” message. The main thread is printing the list, while the communication thread is printing the accurate sent and received messages.

The experienced engineer might be thinking right now that I will have a lot of debugging to do if do not mention using locks for the shared data. Though I did not try that out in the test (as it was not necessary), I will keep in mind to use mutex locks in the integration.

The next steps will be to implement all of the communication signals that will be necessary. The variety needed could change as we flesh out our app, but for now I can get started on some of the essential ones. This is what I will be working to do, going into next week. And in general for the weeks to come, it will all be integration work. My team and I are facing some design changes that we will outline in the team status report so our Gantt schedule will likely be very volatile in the coming weeks. I feel like I am slightly behind schedule, but I am sure I can complete the necessary work without cause for concern. We have set up the slack time just for this reason and hopefully will have a working product by final demo.