Category: Team Reports

This past week, we presented our Final Presentation. During the presentation, we explained the technical challenges we faced and the design tradeoffs we made throughout the design process.

Currently, the main risk we face is being able to 3D print the complete casing within the time left before demo. To mitigate this risk, we will be printing the finished casing in smaller sub-parts and assembling them together. This allows for each individual part to take less time to print, meaning we can more easily adjust the design and reprint the part if there are issues with the physical dimensions.

We carried out an entire system functional test from application to solenoids. This worked out fine. We also have conducted tests to verify each subsystem’s functionality. Our results from this testing in previous weeks have helped us to avoid making any new design changes at this stage of the project. The majority of experimental findings was done in determining latency timings as well as OMR parsing accuracy.

These were the methods we used to test specific requirements:

• • OMR Accuracy → Reconvert back to pdf and compare visually
  • Tempo accuracy → Measure tempo with a metronome 
  • Computer/RPi Latency → Function records system time when start/stop signals are sent and received 
  • Power Consumption →  Measure DC power supply output voltage and current to calculate power

The table below summarizes the results we found through our testing

These metrics show that we pass on all four requirements.

The main risk we have at the moment is getting the dimensions of the chassis correct. We do not have very accurate measuring equipment, so we had to resort to mostly eyeballing the dimensions of the keys/keyboard using a measuring tape. The measurements appear accurate enough when looking at the Solidworks rendering, but we will not be sure until we print it out. Additionally, placing the chassis on the keys may prove to be a challenge, and we may need to adjust the design of the chassis, but otherwise, the physical portion of the system seems to be coming along smoothly.

One change we have made to our design is the decision to use secure copy through ssh to transfer the XML files to the Raspberry Pi. We decided on this route because we found that the latency for sending one page of an XML file through serial communication was too slow (over 13 seconds) to be acceptable. We tested manually scp-ing files to the Raspberry Pi with success. This may require some extra work in running a powershell script to scp to the Raspberry Pi. We believe this will not incur a heavy cost since one of our team members, Rahul, has gotten some experience writing powershell scripts earlier in the semester. From the user’s standpoint, this may require the user to configure the Raspberry Pi on startup to connect to new networks, but we believe the tradeoff of a one time setup in exchange for much faster file uploading speeds is justified because the user will want to upload many files for all the different pieces they will play during the product’s lifetime.

Another change we have made to the design is to no longer support placing the accompanyBot on any starting key. Originally we wished to have the capability of placing the solenoids over any configuration of white and black keys by adding extra solenoids and remapping the GPIO pins. However, based on the physical measurements and differences between the spacing of the black keys, we found that it is physically impossible to account for that, so instead we will have to fall back on our original constraint of a fixed starting key. However, we will still be able to move the accompanyBot up and down octaves, so there is still the ability to play all the notes on the keyboard.

Lastly, a change was made to the pipeline of how the xml file gets processed. Previously we had intended for it to get immediately shipped off to the notes scheduler. After feedback from our interim demo, we allowed for digital playback of the xml via midi signals before the accompanyBot solenoids receive the information to begin playing. This requires operation of the computer speakers which were previously not factored into our design.

For the interim demo, we created an updated Gantt chart showing the tasks that we will be working on for the last three weeks of the semester. The updated schedule is attached here.

During our interim demo, we were able to successfully show the end-to-end communication for starting the music, pausing the music, and updating the current measure number (both from the computer to the RPi and sending the measure number back so that the GUI could update the state). Here is a video of the current implementation in action.

For our interim demo next week, we will demo the working parts of our subsystems. We currently have a functional sheet music parser and UI on the user input side. We also have the scheduler which can actuate solenoids based on a manually inputted XML file. Our main efforts tomorrow will be focused on integrating these two subsystems, although we cannot guarantee that it will be completed by our demo next Wednesday due to reasons outlined below.

This week, we encountered a roadblock when trying to integrate the Raspberry Pi with the local application. Our team did not foresee the RPi serial communication to the computer requiring extra hardware to convert from USB to UART. Hence, we were not able to make much progress and thus may not have a working integration between the GUI app and the hardware controller come interim demo. We are going to offset the risk of ordering and dealing with new parts by adding an Arduino MKRZero to our system design as a middle controller to pipe information between the computer and RPi. This could pose a risk by increasing the overall latency between app and solenoids. We are accepting this risk with the knowledge that the typical payload of information traveling should be very small, <10 bytes. Additionally, we discussed previously the usage of pySerial library for communication over usb. Python comes with a serial library that has a simpler syntax and seems to be meeting our system requirements. So for now, we are making this minor framework switch.

Next week we will be going into our planned slack time, so no changes will be made to the schedule. All remaining tasks deal with integration so we will be working on integrating pieces of the software to controller to hardware in descending order of urgency.

This week we met in pods with other teams to discuss potential ethical issues associated with our project while also providing feedback on our peers’ projects. Some points that were brought up in the discussion were considerations for possible areas that could go wrong and groups of people for whom our product is not designed for. In terms of problems that could arise, if the product fails during a performance, this could result in embarrassment for users. Additionally, if the timing is off then users will have difficulties when they go on to practice and play with other people. Finally, issues with the product could potentially break the piano if the force is too high or harm people’s hands if they stick their fingers under the solenoids. These considerations all pose risks to the success of the accompanyBot.

Since we are in the integration phase now, another big risk to the project is that we encounter roadblocks with joining the different subsystems. To address this risk, we are going to discuss clear requirements for the interfaces before implementing. We are also redistributing the attention of team members to focus on areas that are at risk.

There will have to be more changes to the design of the chassis since the dimensions of the solenoids did not match up with the dimensions of the piano keys. There is enough space on the black keys, but the white keys may prove to be difficult to build around. We will likely have to  offset the solenoids inside the chassis so that they are not directly adjacent to one another allowing them to fit along the length of the octave. Hopefully it does not prove impossible to construct, but I believe it is still viable. There was a minor change to our system, specifically the image displaying done in the GUI application. This required the incorporation of poppler and pdf2image modules for python. The model of our system from the design report still functions the same however.

In terms of our schedule, some changes will be made. In particular, Rahul will be moving attention to getting the notes scheduling finalized. Aden is primarily taking up the building of the chassis, and will be working on this solo next week instead. Nora will be focusing on integrating with the computer with the help of Rahul in the coming week.

This is the change being made to the Gantt Chart tasks for week 8:

One risk that may jeopardize the project is 3D printing the chassis. Aden has not used 3D printing technology in 3 years, so relearning it may take more time than expected. Similarly, he has not used Solidworks in two years. Rahul is less familiar with such tools and will likely spend more time learning once he is done developing the software for the accompanyBot and begins aiding with hardware construction. Clearly, there will be a learning curve to building the chassis. However, if there are significant difficulties with 3D printing then the alternative plan is to use a plank of wood with the solenoids secured on the plank and placed on top of the keyboard. Another concern is that the solenoids are quite heavy, so the support material must be able to withstand the weight. Therefore this would factor into our design choices as well.

No significant design changes were made to the physical portion system. Details are still being worked out in regard to the chassis, but it will more or less look similar to the existing design from the design report. The solenoid circuit also remains the same from the design report. The application design will remain the same as our specification. As for the integration of the app with the RPi, we may use PySerial but if that does not suit our needs in a reasonable amount of time we will rethink this system communication step. Due to slight fluctuations in the timing of the solenoids found during running the RPi code to control the MOSFET switching. Additional calibration may be needed to make sure the tempo meets our system requirement. However, since the variations have resulted in values slightly off from the targets (1bpm difference), we may consider loosening the 100% accuracy measure to allow some tolerance, since we did see Professor Sullivan’s feedback about the metric being a high bar to achieve.

No significant changes need to be made to the schedule. If there do need to be changes, they will take up the slack time that is in place.

As we noted in the design report, the main risk we face in our project is not being able to achieve exact accuracy in meeting timing and rhythm requirements that we laid out in our design requirements. While we have confidence in our ability to replicate common tunes, we foresee challenges for songs and pieces that are very complex in rhythm with very fast playing of notes. To mitigate this risk, we will not support playing songs that have notes with note values smaller than our system can support. In addition, we will preselect a set of songs for demo that we can ensure are playable by our system.

After establishing our model for the system architecture, each of us has been tasked to focus on different subsystems. Rahul, who is working on the GUI, will have to get more familiar with the syntax of pygame to flesh out the application design. Nora, who is emphasizing efforts in the note scheduling, will have to ensure type safety of the data as it is handled in the music21 data classes. Aden, who is leading the building of the accompanyBot player structure, may have to get familiar with the Solidworks and also learn circuit breaker operation to implement that portion of the circuit.

So far, we have been meeting the milestones on time. Our parts have all arrived as well, so we are not worried about any delays due to shipping. Thus no changes were made to our existing schedule or system.

This week, we made it past the Design Review Presentation and have entered a major portion of the development stage this week. Our team’s GitHub has been set up, and initial contributions have been made with respect to the OMR parsing scripts and Raspberry Pi microcontroller scripts. Additionally, we discussed and moved forward with ordering the remaining 14 solenoids and 6 NMOS transistors to meet our hardware specifications.

Integration of the accompanyBot apparatus with the raspberry pi may have a hidden difficulty. Through Nora’s initial GPIO code, we were able to see that LEDs and smaller components respond more quickly compared to the bulkier inductance based solenoids. This may affect our ability to reach the requirement of 6 key presses per second. To mitigate the risk of not meeting this requirement, we may need to rescope the threshold metric so that the hardware is able to meet it. No further changes were made to our design and requirements.

Teaming

One strategy that we have employed for working together effectively is maintaining constant communication. Apart from our Slack channel, we have a group chat where we discuss issues and different ideas for our respective aspects of the projects. For example, this past week we made sure to get one another’s approval before ordering the remaining parts for our build. We also meet up at least once a week outside of class time to update each other on our individual progress, write our team reports together, and work on any deliverables that we aimed to complete that week. Furthermore, we have made the following adjustments to fill in gaps related to challenges that have come up:

• Rearrangement of scheduled tasks – Due to the introduction of music21 to aid in our scheduling process, we adjusted the work assignments so that more time could be allocated to familiarizing ourselves with the library. Additionally, Rahul joined Nora in setting up the notes scheduler algorithm.
• Waiting for parts to arrive – While the remaining solenoids and transistors are being shipped, Aden has been helping Nora with her microcontroller scheduling tasks. This was anticipated and accounted for before ordering parts.

This past week, we ordered and received solenoids for testing. We also received a Raspberry Pi 4 from the 18500 Inventory. This allowed us to explore the parts to help gather data for metrics for the upcoming Design Review Presentation. During our meeting with Professor Sullivan, we also received feedback on a set of additional requirements that we would need to include during the Design Review

Principles of Engineering, Science, and Mathematics

From the 7 ABET principles of STEM, we believe this week we incorporated principles 3, 5, 6, and 7. 

Our rationale for these choices is as follows:

(3) Our work towards the upcoming design review presentation involves effectively engaging and communicating to our audience how feasible our project is turning out to be after having already put together some of the pieces.

(5) Every week, we make sure to meet up outside of class at least once to regroup our work and try to help debug or discuss integration strategies for future development based on current progress/knowledge. This week was focused on how we may go about scheduling our newly arrived solenoids for key pressing. 

(6) To make sure we are meeting the quantitative targets for our design, we must gather and analyze data that motivates our design decisions. This week specifically, we used laboratory power supplies and testing equipment to measure the voltage and current needed to power our solenoids. The data we collected from this experimentation helped us determine which solenoids out of the initial batch we would go with for our final implementation.

(7) Since we are engaging with a lot of new technology on both the hardware and software sides, it was crucial for us to acquire and apply new knowledge when fleshing out our design. For instance, Rahul needed to learn to write powershell scripts, and Nora and Aden figured out thresholding power for the solenoids.

Risk and Risk Mitigation

The main risk for our project is the issue of safely powering multiple solenoids at once. During our testing of the solenoids, we found that the 25N solenoid required around 0.6 A of current at 10 V which results in 6W to power one solenoid. This was much less power than we initially expected, and if we consider our worst case scenario with five solenoids being powered at once, the total average power will be far less than we expected (about 30 W). However, we were only able to test one solenoid, so having all solenoids drawing current at the same time may cause potential problems if the additional load results in a greater current that does not scale linearly (since our power supply has a max amperage of 5A). To mitigate this risk, we are willing to decrease our max number of solenoids to 3.

Of the initial order of solenoids, we noticed that one arrived broken and thus could not test it well. These mechanical components accelerate rather quickly and thus could be susceptible to damage or may cause harm in cases of malfunction. They also make a lot of noise, which may interfere with the sound of the actual piano. To solve such issues we may need to add padding or modify our circuit to better average out the impact.

Changes Made to Design

The main change we have made to the design is the choice to incorporate the music21 Python library to aid in the parsing of the music. We chose to go with this library instead of writing our own parser because the musicXML file generated by Audiveris is quite bulky and contains a lot of extraneous text whereas the music21 library has a lot of functionality that can aid in scheduling, which, as we have been warned, is a non-trivial task. While this comes at no extra cost budget wise, it does require alterations to the schedule in order to include time learning how to use the library and incorporate it into the project. Our updated Gantt chart is included in the Design Review Presentation slides.

This week, we presented our project proposal to faculty and fellow teams. We received some insightful questions that we should keep in mind for our design. For instance, one question asked how we would interpret key words in music such as “rubato” that are not associated with specific tempo values. 

Our project includes considerations for safety, cultural, and economic. In regards to safety, our project will likely be dealing with a large amount of power due to the high current required by the solenoids. Limiting this power is an important consideration for the safety of us as well as the user. Culturally, we recognize that our sheet music parsing is centered more on Western styles of music which puts a limitation on the style of music we can play. Importantly, our project aims to lower the cost associated with piano accompaniment. Since the cost of hiring a professional piano accompanist is high, we can provide a more inexpensive alternative.

We have also begun the process of looking ahead for parts that will be needed. Since we have uncertainties about power consumption, we will be ordering and testing different solenoids for our key press mechanism. The biggest risk associated with our project so far comes with the issue of powering our worst case number of solenoids at once. To mitigate this risk, we have considered reducing the max number of keys pressed to three keys at once. This would reduce the current draw to below the max current of our existing power supply while still allow us to have a polyphonic system that can cover a three note chord.

Our identified OMR solution is looking pretty good so far. We have decided to go with Audiveris, and here is a sample of the transcription.

Original Music Pdf file:

Corresponding Transcribed XML