Leonardo Da Robot

This week our team made significant headway in the 3 main components of our project; the gantry construction, the gantry control systems, and the painting algorithm. A single team member worked in each of these areas, and no significant issues have appeared in our design from what we have been able to achieve this week. The next target on our horizon is creating a short demo of the progress we have made thus far, and believe that we will be able to complete this comfortably in the following week.

No significant risks have appeared from what we have developed this week, but as our individual areas are being completed we are starting to consider the integration of each system. The integration is likely to have some issues arise and to control these factors we have clearly laid out the inputs and outputs of each layer and how they expect communication to happen. Given that we still have ample time left we do not consider this to be a serious risk to our success, but are considering how we can handle this process early to save time in the future.

Overall our progress this week has remained mostly on schedule. By our schedule this week should mark the complete construction of the gantry and the successful operation of our motors. While the gantry is not entirely completed, it will be entirely assembled in the next week, and enough is completed that it is not hindering progress in terms of controlling the motors. Motor control has been completed, and the painting algorithm is also underway.

No changes have had to be made to our design, aside from the development of a software model of the gantry system to allow for easier testing of the painting algorithm. This is not part of our final deliverables, but is a part of the development of the painting algorithm. This change was necessary as it allows for much faster testing of the software layer.

This week, I spent most of my time on the stroke generation algorithm. The first couple days of the week was spent on finishing the design review report with the rest of the team. After we submitted that, we got together and made sure that we had ordered all the parts so that we could build our frame and gantry system as soon as we got back from Spring Break. My main task this week was to work on creating outputs of the mean shift segmentation on all the images from the test bank. This way, we have pre-processed images which we can feed into our physical system.

I then worked on the stroke generation algorithm. This algorithm takes the mean shift segmented image as an input. It then turns the image into a list of strokes. There are two different kinds of strokes: perimeter strokes and fill-in strokes. Each object in the segmented image is made up of one perimeter stroke and one fill-in stroke. The perimeter strokes trace the outline of an object with the color of that object and the fill-in strokes fill in the object with horizontal straight lines of that color. The order of objects to be drawn is based on the length of the perimeter stroke, to make sure that the lowest detailed objects are drawn first. Once the strokes have been created and ordered, they are broken up into line segments which are defined by a starting coordinate, ending coordinate, and color. The color is chosen to be the closest color from the palette in terms of HSV values to the original color. This list of coordinates and colors is the output of this function that is sent to the control layer.

I am currently working on this function, and will use the break to finish it. Once I finish this, I will be on schedule. When I come back to campus after Spring Break, I will be working on helping build the physical system. We will have all of the parts by then and will need to focus on building this mechanical system first.

In the beginning of this week, most of my time was spent working on the team’s design review report. In addition to writing my contribution to the report content, I compiled and formatted the document at the end. The Raspberry Pi order has still not arrived yet, so I am still unable to directly work with the Pi. I created a GitHub repository for the team’s code, and continued working on the code for the motor controls. I am encountering some difficulties in using the library, which I may need to solve by installing Linux on my computer.

My progress is a bit behind schedule, because I was unable to complete most of the motor control code before Spring Break began. To make up for it, I will push myself to complete the code soon after break ends. The process will also surely be expedited by the arrival of the Pi and being able to actually test the code. In the week after break ends, I plan to finish the majority of the code for the motor controls and test it on the Pi once it arrives.

My work this week was mostly focused on contributing to our design document, but outside of this I continued iterating on the 3D part designs and created a template of the routines which need to be developed in python. I printed the initial design of the carriage pat, but due to some size and tolerance issues from printing, the two halves were not able to mate properly. I updated the design to accommodate this but the issue was still present. This issue has been difficult to fix, as each design iteration takes at least a day to verify due to having to wait for the print. Currently, these design issues have set me back slightly and to make up for this I can start printing beta versions of every single part at the start of the next week, to allow for all parts to be finalized and for the full assembly of the gantry to begin.

Additionally I began creating the outline of our software layer and created all of the files we will need to operate the gantry. These files finalize the interface designs between the different components of the software layers and allow us to test each section of the code individually. My contribution to this in the coming weeks will be to develop a simple model replacing the operation of the gantry, which will allow us to test the image processing and stroke generation algorithm.

Overall I have fallen slightly behind on schedule this week, as I have not been able to finalize all of the 3D designs. In the upcoming week I hope to catch up on this, begin assembly of the gantry once the parts arrive, and develop a simple model of the gantry control layer for use in testing the painting algorithms.

This week our team was mostly focused on creating our design document, which consumed most of our time in addition to the discussion we had during class on Monday. None of our new parts arrived, but we have completed ordering all of the requisite parts and should all arrive by after break.

No new risks have developed for our project as a whole, but we are still faced with risks regarding the success of completing the gantry and having an effective mapping from the digital image to series of paint strokes. These portions are being mitigated by us concentrating our efforts on these goals continuously.

Minor changes have been made to our design, and the total design has been finalized with the completion of our design document. Our gantry will now operate on special watercolor paper which should be more robust to warping. This has dimensions of 5×7 inches, meaning we have much more space available in the design of our board. The interface layers have also been finalized, where the gantry control layer exposes an initialization function and a draw stroke function, which are the only means of communication with the painting algorithm, allowing it to be controlled very simply.

This week we were not able to move forward as much as we would have liked due to the significant portion of work of creating the design document and our parts not arriving. An order mixup resulted in us not receiving our bearings, which meant we were not able to test compatibility of the shafts with the bearings or test the 3D printed bearing mounts. Our schedule has also been updated as of the design document and the most up-to-date schedule is included here.

 

 

This week, our team discussed and finalized the final aspects of our design, and spent a good deal of time working on the design review, which involved both preparing for the presentation as well as writing the report. Many of the parts we ordered arrived, so we were able to start experimenting with them, such as comparing the parts with each other and seeing how well they would fit together. Seeing the parts in person also allowed us to confirm and modify features of our design to accommodate our parts.

The most significant risks of our project continue to be successfully completing the gantry as well as having a robust and accurate image processing algorithm, but no new risks to our project have been assessed since our last status report.

Some minor changes were made to the existing design of our system. We changed the dimensions for some of the parts of our gantry system to better accommodate the parts we have received and their measurements; for example, we finalized the dimensions of our gantry frame to be 18 inches by 18 inches. This change does not come at a cost, as the parts we ordered are flexible and can be cut or modified to different lengths and sizes. Minor changes were made to our block diagram for clarity and to better convey the flow of information, but the actual design of our system has not significantly changed.

One setback our team faced this week is that a miscommunication and careless typo on our end caused us to accidentally order an extra Raspberry Pi motor shield instead of the Raspberry Pi. As a result, some of the testing and experimenting that we wished to do with the Pi this week has to be postponed due to not having the Pi yet. In place of working with the Pi, the team instead focused its efforts on other tasks to make up for it. We have updated our schedule to have more detailed tasks as well as reflect our scheduling updates.

This week I spent most of my time working on the design review report. Since I had done the design review presentation, I had a good idea of what to put into the design review report. One of the criticisms from Professor Nace was that our schedules weren’t precise enough, so I spent some time making a more detailed and better formatted schedule.

One of the things we realized was that the time constraint of 8 hours for all images didn’t make sense and was too vague. Therefore I, with Chris’ input, created a new way of measuring how much the time constraint for a specific image should be. It is based on the complexity of the image. The way to measure complexity in the image is based on the ratio of compression of the jpeg version of the image and the size of the image. The more it can be compressed, the simpler the image is. I also cut the shafts to the desired lengths for the gantry system.

My goal for the next week is to run the image segmentation algorithm on the test bank and save the results. The biggest task for me will be to create the stroke generation algorithm. This is the algorithm that takes the output from the image segmentation and turns it into a list of strokes for the robot to do.

I was supposed to start the stroke generation routine and creating the image bank yesterday but I had to push this back by a couple days in order to complete the design review report. I will start working on these tomorrow. In case I don’t finish, I will work on this over Spring Break.

This week I, like the rest of my team, spent a large amount of time working on the design review report and presentation. In addition to contributing by writing slides and sections of the paper, I am formatting the final Word document for our report. As part of finalizing the design for our project, I constructed an image bank consisting of ten increasingly complex images which we will be using to test our project. The image bank is as follows:

 

I also updated our block diagram to clarify the way components interact with each other. As mentioned in the team status report, a mistake on the team’s end caused us to put the wrong order through for the Raspberry Pi, so I was unable to experiment with controlling the motors we received through the Pi. When we realized the mistake, I found the parts online that we needed to order, which included the Pi itself, a power adapter, and a microSD card and its adapter, and placed the orders for it. Although I was unable to work directly with the Pi, I started to use and experiment with the libraries to control the motors, but currently have no way to actually test out any code I write. As a result, my progress is behind schedule. To catch up, I am preparing code this week and next week which will be ready to be tested immediately once our Pi actually arrives. In the next week, I hope to complete the preliminary code for controlling the motors through the Pi, as well as finish up work on the design review report and finalize the format for it.

This week I completed the assembly file of the gantry design and made several modifications to the originally designed parts. Some of these modifications included a complete redesign of the bearing holders, a redesign of the print head to hold a servo vertically, and modification of the frame to move beams to the top. The assembly file is shown in the gif below.

 

Additionally I began printing some of the parts and made some modifications based on the results. We have received the shafts which run along the perimeter of the design and have been able to fix the pulley to the shaft using a small screw. A good portion of my time this week was also spent working on the design report.

This week I remained mostly on schedule, but made an ordering mistake last week which resulted in us not receiving our raspberry pi which slowed us down a bit for this week. I am still mostly on schedule, and plan on completing all designing and 3D printing of the remaining parts.

This week, we finalized the design for our 2D gantry system, as well as the frame of the robot. We started ordering many parts, including the Raspberry Pi, the motors, a paint set, and shafts and bearings for the axis system. We worked on our design review presentation, and created several diagrams and schematics for it. In addition, our team met with Professor Aswin Sankaranarayanan to get input on the image processing aspect of our project and discuss our software algorithm.

The most significant risks of our project continue to include the success of our 2D gantry system, which was discussed in last week’s report. We will continue with the plan we have in place, with our backup plans still ready. Once the parts we’ve ordered have arrived, we can begin constructing the system, and adjust our approach and design based on any problems we may encounter during that process. Another risk was brought up once we discussed our project with Professor Sankaranarayanan, who discussed certain difficulties in creating a painting robot. Specifically, he mentioned the main three things to consider would be the clustering algorithm, the color intensity, and the brush width. He gave us some advice in dealing with each issue, and we discussed the potential approaches and solutions amongst our team. Due to the difficulty in some of the image processing considerations, there are risks in how successfully we can translate a digital image to painting. We decided to keep our initial approach, which is based around the mean shift segmentation algorithm. However, if this fails, we can use any of several other clustering algorithms, with k-means being a viable backup. Regarding color intensity and brush width, there are several alterations we could potentially make to our painting procedure if either of these are an issue. We could modify our approach to only deal with silhouettes, only use edge detection and paint the edges of the input image, or just reduce the detail we recreate in our painting. Our painting head is designed so that it can hold pencils, paintbrushes, etc.

No significant changes were made to the existing design of the system, although we did finalize decisions on the hardware component of our project as well as how to power the robot. No changes have occurred to our planned schedule, as we are all on track with our progress.