Carnegie Mellon ECE Capstone, Spring 2022 – Joseph J., Jasper L., Raymond N.
Jasper is working from home on the poster because he is still down with a Covid-related illness.
Raymond and Joseph are both working in the lab in person to integrate all subsystems together and to test the integrated system. Progress still needs to be made before final video and final demo day. Specifically, new realities about the demo location and feedback from the final presentation necessitated some changes to our design.
We hope to get one meat case done and filmable by May 4.
This week, the team ran into a few roadblocks. Jasper got covid, making it difficult to integrate all the parts together, and the Jetson Xavier we were using decided to break down. Fortunately, after a few days, we were able to flash the Xavier and get it up and running again. We lost a bit of progress, but we made up for it over the course of the week. Additionally, despite some difficulties, we were able to integrate the Computer Vision algorithm to detect meat and the UI together successfully. The Inverse Kinematics for the robotic arm are also going well, allowing the arm to move with the desired range of motion. Finally, the team has been making good headway regarding preparing the slides for the final demo.
We are now working on more extensive testing to make sure all the parts are working as smoothly as planned. As was expected, some aspects are not quite as polished as we would like for the final product, but that was something we planned and made time for in our schedule. We are currently right on track with where we need to be and we are optimistic about our project as a whole.
Raymond has been progressing on his project components and has begun part of the integration step. However, he still needs more data to form a more accurate network. Therefore, the Gantt chart is labelled as 90% complete.
Joseph needs to change the type of inverse kinematics he used due to issues with the Matlab library outlined in the detailed design writeup. He will now be using another inverse kinematics library as one of the risk mitigation options.
Jasper is behind in his progress in the software UI component, as he did not properly test out the success of his user interface interaction with camera.
As of the demo, the most recent updated gantt schedule is included in this post. Catch up may be needed, and some slack time will have to be used up to get the system working.
This week, the team has been diligently working towards being ready for our project demo on Wednesday 4/6. We seem to be in a very good spot with all of our subsystems in a presentable point.
Joseph can demonstrate the robotic arm we plan to use to flip and transport the meats being cooked. The inverse kinematics of it have been difficult to implement, but at least we will be able to show that it moves with the range of motion that we need for our end goal. We have the wiring and other heat-sensitive parts insulated so that they will be protected from the high temperatures of the stove. This has been tested using a soldering iron which did not have any effect on the more delicate internals of the arm.
Raymond has blob detection working with the CV able to detect meats with decent accuracy. He has switched from image classification to object recognition to prevent scenarios where a user places 2 different types of meat on a plate. He has changed parameters of the original blob detection algorithm to be better at identifying meats instead of other objects in the scene, along with improving performance in low light conditions.
Jasper has finished work with the UI and cooking time algorithms. The UI is hopefully clear enough to provide a clean and intuitive experience for the user. It displays information about the detected thickness of a new piece of meat, how long the system decides it needs to cook, and which section of the grill it will go on. There is also an overhead camera feed of the grill which displays which section is which, enabling the user to be confident that the system of working as intended. We plan to add the capability of changing the remaining cooking time for any section that they select, but we do not have that working yet.
Overall, the project is keeping up with the proposed schedule from earlier in the semester, and the team has made great progress.
The computer vision progress for blob detection seems to be progressing smoothly. There seems to be no hiccups in the production schedule for the project. Joseph has been progressing smoothly on the robotic arm, and the grill has arrived. This is a crucial step because now he can better obtain visual feedback on how the arm will interact with the grill environment. Jasper has been working hard on the UI and on some of the more fine tuned aspects of the system, such as the cooking time algorithm.
From the ethics class, one ethical issue we did not consider was the fact that a childs arm might be mistaken as one of the pieces of meat to be picked up by a robot. This is a potential issue that our team will have to look into further, but one solution is to have a manual robotic arm stop in the user interface. Note that this issue will mainly apply to people with a more reddish skin color, perhaps caucasian or sunburnt person. More testing will be needed to see the interaction between the blob detector and skin complexion
For spring break and the first-week back, we started developing each of our respective subsystems. Raymond has made progress on the CV algorithm and is currently trying to get his Blob Detection Algorithm working. Jasper has started working on the UI and deciding on what information to put. Joseph has finished the mechanical robotic arm, as well as the electrical subsystem of the arm. He has also started work on the lower embedded software control of the various stepper and servo motors. Our parts were all delivered and picked up this week. We are all at a decent pace to have the robotic arm and our respective subsystems working by the interim demo. We have not started working on the software controller, but we believe it should be done as part of the integration process, as well as after the interim demo. We have also each worked on the ethics assignment and are ready to discuss it with other groups. We will update our progress in our schedule accordingly.
This week, we completed our designs for each subsystem and compiled all of this information in our design review presentation. Additionally, we have placed our order for parts that we will need for the project and hope to integrate them into our development as soon as we can. The group has also done some work regarding the actual development of our respective parts. Joseph has done work on the robotic arm, including creating CAD models of pieces of the robotic arm that we are making and 3D printing some of those parts. He has also split up work for writing the design report. Raymond has done some work and research on blob detection, including finding a possible new system to increase the accuracy of our CV. Jasper has been considering different ways for us to integrate the touch screen UI into our system, but will have to wait until our parts come to decide what will be best. Finally, as a whole, we seem to be on schedule for a successful project.
This week was our busiest week so far. We had discussions to finalize our design choices for the overall system states and architecture. We estimate that we are each about 70-80% finished with designing our own respective subsystems. We will complete our designs in time to submit the design report. We have not yet documented the fabrication and assembly process of our design, which we will do in our report. We also have to meet again to discuss what the interfaces will be between each of the respective subsystems we are designing and also start thinking about how we will proceed with system integration. We have updated the Gantt chart to show our progress in our design phase. We added slack to our schedule to include more system integration and start our development phase when spring break starts. We decided Raymond will present for our Design Presentation, which is mostly finished and ready to be submitted. Although we are all still busy in the Design phase, it is worth noting that we should look ahead and start discussing the Development phase of this project next week. We hope to continue this pace throughout our project.
One risk we have not considered to the extent we should have until feedback from the instructors for other proposals was slack and integration time. For one, our integration between the software controller and the computer vision algorithm and the integration between the software controller and the robotic arm happen almost simultaneously, which creates the issue of integrating 3 items at the same time, greatly complicating the process of integration. Developing the Computer Vision and software interface might take less time compared to the robotic arm, so one idea proposed is to reduce the development time of computer vision and software by a few days or a week and use that extra time to integrate. Furthermore, testing the robotic arm on an actual grill should take place before the integration step, in the case of a serious misunderstanding of the heat tolerances of the arm. We also made a basic concept drawing of our robotic system. Here is our updated schedule, mostly seen at the end of March and the month of April (I’m sorry for the image quality it cannot be improved for some reason, but we have made some dates for integration longer and software and CV implementation shorter).
