Posted on

Aden’s Status Report for 4/8

Heading into the final weeks of the semester, and thus this project, I have run one primary test for my part of the project, and that was ensuring the power consumption of the circuitry does not exceed 60 W. This goal has been met since we limited the number of solenoids that can be turned on at once to five, since each solenoid draws about 0.9 A and uses 12 V this requirement has been met and tested. Additionally, if this proves to be troublesome in the coming weeks, the voltage can be lowered, reducing the current since each solenoid doe not need 12 V to operate. I will also be helping measure our maximum tempo since the circuit will be a limiting factor due to its physical characteristics. Hopefully, we will be able to meet our preset quantity, but that is probably unlikely.

In addition to the interim demo, the image above is what I have accomplished this week. I realized that there were a couple of issues with the previous version of the chassis design. One of the major issues was that the black and white keys were not actually aligned properly in my previous design. Another major issue was that we wanted to allow the system to be placed over any octave on the keyboard, but after some more careful measurements, I realized this would not be possible for what we had in mind, so I had to redesign the chassis nearly from scratch. Additionally, the previous design was under the assumption that we would just secure the solenoids to the inside of the chassis, but to plan for problems arising for demo day, we switched to a design that would allow for solenoids to be replaced easily. Overall, I believe this is a much more robust and easy-to-use design, and hopefully, it will be suitable for demo day.

I am still technically on track, but I would’ve liked to 3D print a prototype tomorrow, and I’m not sure if I will be able to since I am currently having difficulties with Solidworks right now. Hopefully, I will be able to figure out my problem with enough time to 3D print tomorrow, but if I don’t, I will try to print something out Monday. I hope I can print it out tomorrow, though.

Next week I hope to have a prototype of the chassis completed, and if it does not function properly, I hope to update the design and print another one before the week ends. The end of the semester is coming soon, so I need to finish this by the end of next week.

Posted on

Team Status Report for 4/1

For our interim demo next week, we will demo the working parts of our subsystems. We currently have a functional sheet music parser and UI on the user input side. We also have the scheduler which can actuate solenoids based on a manually inputted XML file. Our main efforts tomorrow will be focused on integrating these two subsystems, although we cannot guarantee that it will be completed by our demo next Wednesday due to reasons outlined below.

This week, we encountered a roadblock when trying to integrate the Raspberry Pi with the local application. Our team did not foresee the RPi serial communication to the computer requiring extra hardware to convert from USB to UART. Hence, we were not able to make much progress and thus may not have a working integration between the GUI app and the hardware controller come interim demo. We are going to offset the risk of ordering and dealing with new parts by adding an Arduino MKRZero to our system design as a middle controller to pipe information between the computer and RPi. This could pose a risk by increasing the overall latency between app and solenoids. We are accepting this risk with the knowledge that the typical payload of information traveling should be very small, <10 bytes. Additionally, we discussed previously the usage of pySerial library for communication over usb. Python comes with a serial library that has a simpler syntax and seems to be meeting our system requirements. So for now, we are making this minor framework switch.

Next week we will be going into our planned slack time, so no changes will be made to the schedule. All remaining tasks deal with integration so we will be working on integrating pieces of the software to controller to hardware in descending order of urgency.

Posted on

Nora’s Status Report for 4/1

This week, I worked with Rahul on integrating the Raspberry Pi with the computer’s local application. Since the computer will be sending and receiving measure numbers as points of reference for where in the piece to start playing and what part of the piece to display on the screen, I had to restructure the code so that the measure numbers were the keys in the dictionary that holds all the notes. This would make it easier for the RPi to look up which notes to play at specific times. With Rahul’s guidance, I re-implemented the scheduler to be compatible with this data structure change. Thanks to Rahul’s suggestion, I also changed the code so that the RPi loops continuously while checking the system time instead of sleeping unnecessarily as was done in the previous version. I was able to alter the code and get it back to a working implementation.

Another area I worked on was introducing the serial communication to the Raspberry Pi. As mentioned by Rahul, we did not have the required USB to UART converter that would allow us to interface with the RPi’s TX and RX pins. To avoid the extra cost and delay to our schedule that would result from purchasing an adapter, we decided to use Arduino microcontrollers that have built in serial ports. In tandem with Rahul, I experimented with using the Arduino MKRZero which has two serial ports: one between the Arduino and the computer and another to the on-board TX and RX pins. By using these two pins, and having the Arduino read the data from the USB and write to the TX/read from the RX and write back to the USB, I was able to set up the system to send and receive strings between my computer and the RPi. While one risk of this implementation is that there may be added latency, the fact that the serial communication is buffered should allow the Arduino to pass the information through at roughly the same baud rate. Tomorrow I will write a more comprehensive test using the system time to find the round trip latency between sending bytes from the computer and receiving a response from the RPi.

At the moment I am behind schedule since we have not completed the integration. However, we did anticipate that this part of the project would take more time and had incorporated slack time for it accordingly. Thus this is on schedule when considering that slack time.

Next week I will focus on further developing the serial communication. I will be meeting with Rahul to agree upon the command bytes that will be sent for various situations that are required by the application.

Posted on

Aden’s Status Report for 4/1

This week finally got around to playing with Solidworks. I was able to generate a basic rendering of what we had in mind for the chassis design. I am still working out how much wiggle room we have between solenoids, but otherwise, I should be set to 3D print soon.  Unfortunately, I am not finished with the Solidworks rendering and would like to add some additional features to it, so I will focus on that tomorrow. Here is the 3D rendering that I am currently working on:

From my last status update, I am more or less on schedule. I was not able to get a copy of Solidworks until yesterday, and even then, I had to get a new license from the software catalog and sit through an hour of download times. Tomorrow, however, I will be cleaning up the rendering and adding some features to the design to hopefully improve the overall performance of the accompanyBot.

In addition to the work I will be doing tomorrow, I also hope to figure out how to 3D print, how expensive it is, and how to generate the files necessary to print it out. I also hope to help Nora and Rahul with the serial communication if they do not finish it before the week ends.

Posted on

Rahul’s Status Report for 4/1

Early in the week I worked with Nora to devise data structures for our notes scheduler. The primary functionality we determined would be to take a measure of notes at a time and play through them sequentially unless it receives a measure change or pause signal from the app. I then proceeded to try and work out the communication methodology between the RPi and Windows computer. Unfortunately, my team did not have a usb to usb-c cable on hand, so I placed an order request for one. While waiting, I was able to borrow one from Alex, my group’s TA. I was able to connect the RPi to the Windows computer, but this amounted to nothing since the RPi is not a peripheral and does not get recognized immediately. We would need to order a UART to facilitate the data communication. To avoid ordering more parts and having to deal with UART, I decided to just plug in an Arduino into my laptop and test serial communication from python to the Arduino. I had to relearn some of the arduino intricacies but got a reasonable proof of concept working.

The accompanyBot GUI application will have to spawn a thread from the start that communicates with the hardware. This communication thread and the app’s main thread should be allowed to modify specific mutable objects such as the playing state variable or the current measure number. I ran a test to verify this functionality and it was successful .

In the print log from the test, the list [“nothing”, “here”] gets modified to [“winner”, “here”], by the thread doing communication the data “winner” also gets received by the arduino which was configured to only send this data if it received the “hello mr arduino” message. The main thread is printing the list, while the communication thread is printing the accurate sent and received messages.

The experienced engineer might be thinking right now that I will have a lot of debugging to do if do not mention using locks for the shared data. Though I did not try that out in the test (as it was not necessary), I will keep in mind to use mutex locks in the integration.

The next steps will be to implement all of the communication signals that will be necessary. The variety needed could change as we flesh out our app, but for now I can get started on some of the essential ones. This is what I will be working to do, going into next week. And in general for the weeks to come, it will all be integration work. My team and I are facing some design changes that we will outline in the team status report so our Gantt schedule will likely be very volatile in the coming weeks. I feel like I am slightly behind schedule, but I am sure I can complete the necessary work without cause for concern. We have set up the slack time just for this reason and hopefully will have a working product by final demo.

 

Posted on

Nora’s Status Report for 3/25

This week I worked on adding code to incorporate chord playing support in the RPi. Since the chords are parsed differently from the notes, I had to figure out how to deal with them separately. Another case I had to consider was flats and sharps, since I had to add mapping that mapped equivalent sharps and flats to the same GPIO pin since the name of the note may change due to key signatures and accidentals, but the physical note stays the same. I also worked on making the overall process for scheduling the piece from XML file format to the scheduled notes more smooth.

I am behind schedule according to the previous Gantt chart, but after discussion with Rahul we decided that he would aid with scheduling and integration after his tasks for the GUI are completed. We will

Next week I hope to be able to have the RPi successfully take in inputs from the application. I will be working alongside Rahul to handle communication between the local application and the RPi to handle uploading the XML files, starting, stopping, and changing the tempo. This will put us in a good spot for the interim demo that is during the following week.

Posted on

Team Status Report for 2/25

This week we met in pods with other teams to discuss potential ethical issues associated with our project while also providing feedback on our peers’ projects. Some points that were brought up in the discussion were considerations for possible areas that could go wrong and groups of people for whom our product is not designed for. In terms of problems that could arise, if the product fails during a performance, this could result in embarrassment for users. Additionally, if the timing is off then users will have difficulties when they go on to practice and play with other people. Finally, issues with the product could potentially break the piano if the force is too high or harm people’s hands if they stick their fingers under the solenoids. These considerations all pose risks to the success of the accompanyBot.

Since we are in the integration phase now, another big risk to the project is that we encounter roadblocks with joining the different subsystems. To address this risk, we are going to discuss clear requirements for the interfaces before implementing. We are also redistributing the attention of team members to focus on areas that are at risk.

There will have to be more changes to the design of the chassis since the dimensions of the solenoids did not match up with the dimensions of the piano keys. There is enough space on the black keys, but the white keys may prove to be difficult to build around. We will likely have to  offset the solenoids inside the chassis so that they are not directly adjacent to one another allowing them to fit along the length of the octave. Hopefully it does not prove impossible to construct, but I believe it is still viable. There was a minor change to our system, specifically the image displaying done in the GUI application. This required the incorporation of poppler and pdf2image modules for python. The model of our system from the design report still functions the same however.

In terms of our schedule, some changes will be made. In particular, Rahul will be moving attention to getting the notes scheduling finalized. Aden is primarily taking up the building of the chassis, and will be working on this solo next week instead. Nora will be focusing on integrating with the computer with the help of Rahul in the coming week.

This is the change being made to the Gantt Chart tasks for week 8:

Posted on

Rahul’s Status Report for 3/25

For this week, I had to build out a lot of the framework for the GUI application. Everything I did was able to be successfully implemented in python (alongside the Audiveris and powershell scripts I had written last week and early on in the semester).

This included, adding a variable tempo metric display, controlling this tempo with key presses or mouse clicks on the bi-directional vertical arrow buttons, displaying the pause button when the play button is clicked and vice versa, adding a variable current measure number, creating a slider that controls the current measure number (with equally spaced stops at the integer measures determined by the total number of measures in a piece/song), allowing the measure number to also be more fine-tuned to decrement from bi-directional horizontal arrow buttons.

I also successfully integrated Audiveris into the application, by making a seperate thread call to the OMR program after the file opening thread returns the string of the selected file. Since users may select invalid files, I implemented some basic error handling here as well. Once the OMR parses the music score into XML, it stores it to a cache folder and then the application renders the png of the score into the application for display above the slider. I discovered that pdfs are slightly more complicated than pngs as they consist of multiple pages. Nonetheless, I did some digging and came across the “pdf2image” module for python. Since the app runs on Windows, I had to install and configure an additional library known as poppler to get pdf2image to work properly. With that done I was successfully able to display the pdf music scores in the event the users input was a pdf, which we deemed as valid input in our proposal and design review. At the moment I am only displaying the first page always. Once the notes scheduling portion of the accompanyBot is integrated, I will look to ensure the proper page is being displayed at all times.

This is a short demonstration of what is possible from the GUI as of March 25, 2023.

I also believe I can attempt to offer a more formal definition of what valid input is. In previous work my group and I have done, we just said no handwritten or low resolution scores. Quantitatively speaking, I would say the scores that should be generated need to be at least 400 DPI, otherwise Audiveris begins to have some problems assigning note heads and rests to the scores.

I ran into a bug with pygame not being able to display some of the lower resolution scores that Audiveris was just barely able to process (they were roughly 240 DPI). I will try to look into this, but I did get these pages off the internet, and to crack down on copyright infringement it seems a lot of free content is being made available only at lower resolutions.

My goal for the week was to get as much of the app done that does not require integrating with the RaspberryPi. I think I was able to accomplish that successfully. My schedule for next week states I should be helping with hardware construction and circuitry. Considering my strengths are in software and that we are behind schedule on the implementation of the notes scheduling portion of the accompanyBot, I will shift my time to be spent helping get that done. Nora is working to finish that up, so if she does have it working early in the week then I will instead work to integrate it with the GUI application that I have spent the last 2 weeks building. The new schedule will be updated accordingly.

Posted on

Aden’s Status Report for 3/25

This week I was able to take measurements of the piano we will be using for the final demo. This allowed me to finalize some measurements for the final chassis design. The spacing of the full octave is about 6 cm, the width of one white key is 2.1 cm, the width of a black key is 1 cm, and the distance between the black keys is a maximum of 2 cm and a minimum of 1.7 cm. This will introduce some design constraints since the width of the solenoid is a few millimeters larger than the width of the white keys. This has caused me to rethink the design of the chassis. We may need to alternate sides of the chassis that the solenoids are screwed in to account for the extra space they need. I was able to get a rough sketch of the new design but was not able to complete it since I did not measure the keys until the end of the week. It should not be too much of a setback, but it does introduce some design constraints that previously were not considered.

I would have liked to get a better idea of what the chassis will look like with these new measurements, but I was pressed for time since I did not get them until the end of the week. In an ideal world, I would have also gotten a Solidworks rendering up by the end of the week, but I did not have enough time to log in to a CMU machine and play around on Solidworks. I am not far behind on my Gantt chart schedule, but I would have liked to be a little farther ahead of me now.

After my two exams this week, I would like to get a Solidworks rendering prepared and looked at by my teammates before sending it to the 3D printer. I would also like to place an order for the 3D printing filament and try and find some resources on how to 3D print since I have not done it since spring 2020.

Posted on

Team Status Report for 3/18

One risk that may jeopardize the project is 3D printing the chassis. Aden has not used 3D printing technology in 3 years, so relearning it may take more time than expected. Similarly, he has not used Solidworks in two years. Rahul is less familiar with such tools and will likely spend more time learning once he is done developing the software for the accompanyBot and begins aiding with hardware construction. Clearly, there will be a learning curve to building the chassis. However, if there are significant difficulties with 3D printing then the alternative plan is to use a plank of wood with the solenoids secured on the plank and placed on top of the keyboard. Another concern is that the solenoids are quite heavy, so the support material must be able to withstand the weight. Therefore this would factor into our design choices as well.

No significant design changes were made to the physical portion system. Details are still being worked out in regard to the chassis, but it will more or less look similar to the existing design from the design report. The solenoid circuit also remains the same from the design report. The application design will remain the same as our specification. As for the integration of the app with the RPi, we may use PySerial but if that does not suit our needs in a reasonable amount of time we will rethink this system communication step. Due to slight fluctuations in the timing of the solenoids found during running the RPi code to control the MOSFET switching. Additional calibration may be needed to make sure the tempo meets our system requirement. However, since the variations have resulted in values slightly off from the targets (1bpm difference), we may consider loosening the 100% accuracy measure to allow some tolerance, since we did see Professor Sullivan’s feedback about the metric being a high bar to achieve.

No significant changes need to be made to the schedule. If there do need to be changes, they will take up the slack time that is in place.