Month: October 2018

Arushi – This week my goals consisted of combining my image processing code with Shreyas’s pid/robotics program to try to start testing our project and refining it for our midpoint demo. We initially had difficulties with a lag in our camera which was fixed by changing a while loop in our code to a for loop that sets a parameter for the pi camera. Once we fixed that, we were able to test rotations in place by moving our target around in a circle around the robot. We had difficulties, however, with the image processing sometimes picking up the top of the door to the room as having hsv values within the threshold range for our target. We tried adjusting the hsv threshold accordingly, but were still having troubles. For possible solutions, I wrote out programs that 1. involved an rgb threshold instead of hsv -> for which we were having troubles identifying the circle under a shadow and 2. changing our target to a chessboard and using a opencv builtin for finding chessboard corners. The chessboard was accurately being picked up, but similar to other HOG based algorithms, it was too slow when it was implemented on the pi. Right as we started to print a new target for which we could specify the hsv, when coloring it, Shreyas tried a threshold that worked. We then were able to test the individual aspects of in-place turns and moving straight. At the end of class we were able to put the parts together. The goal for next week is to further smooth out the PID, connect the pi to a battery source and then test in a variety of lighting to ensure that the image processing does not pick up parts of the image that are not the target.

Shreyas – This week I worked with Arushi on integrating different image processing algorithms into the Raspberry Pi. To avoid repeating details described above, I will talk more about the robot control module side of it. I started with tuning the PID for the X displacement part to make sure rotations were smooth. After this was tuned and after a better image processing technique was found I added logic to find the z displacement based on the area of the target. Then I got the PID controller to also work on the Z displacement part so movement forward and backward was also smooth. The last and most challenging part on my end was figuring out how to move the robot in both the Z and X direction. Since I am sending raw motor power commands based on the output of the PID controller, currently it is not possible to move in the Z and X direction in the same time. The main limitation right now is that the PID controller calculates the motor values independently of each other. Next week I am going to work on having the PID controller give a combined output of Z and X displacement so that movement in both degrees of freedom is possible. However another concern is that may damage the motors. Since the iRoomba is a pivot based robot, sending two different power values to the left and right motors (for curve motion) would put stress on the motors. However for slightly different magnitudes this shouldn’t be a problem. This is something that takes careful testing. Next week my goal is to integrate the iRoomba’s sensors and tune the PID for bi-directional movement.

Pallavi – This week my goal was to get the serial communication between the arduino and the raspberry pi set up. I unfortunately did not complete my goal because I ran into some issues setting up the serial communication on the raspberry pi zero. I’m running a simple python script on the pi just to check whether I can write to the terminal. When I try this, however, the program simply hangs. I looked at this link (https://www.raspberrypi.org/forums/viewtopic.php?t=180951)  and made sure to implement all the solutions discussed. However, my raspberry pi is still not printing out to the terminal. I’m going to work with Shreyas on Monday to figure out what the issue is.

Other than setting up the Pi, I was looking up doing tx, rx communication versus usb communication between the arduino and raspberry pi. I ended up going with tx, rx communication because of less overhead compared with usb communication. I also had to set up the simple voltage divider to properly connect the raspberry pi and arduino, as the raspberry pi has a 3.3V tx and rx pi while the arduino has 5V pins.

This week, I hope to get the serial output working on the pi to ensure I can test my arduino code. I also hope to start integrating my sensors with the combined iRoomba and object detection Shreyas and Arushi have been working on this week.

Arushi – This week, I looked more into homography using checkerboards to further understand the process of when we implement turns. Basically, for us, based on what I learned, we will use the angle between the circles’ center to determine which way the human is turning. Furthermore, we established that in general, when it comes to turns, the cart will only move if the human (and thus the closest circle) is getting smaller, to prevent the cart from turning when a human pivots to pick something off from the shelf. I also worked on porting over my code to the raspberry pi and combining the image processing with Shreyas’ robot control/pid module. Because we had no way to adhere the camera to the robot, we tested with me moving the  target in front of the stationary camera and the robot moved accordingly (in terms of the general direction). Thus the next step is to actually attach the camera to the robot and test it, which is the plan for next week. To be able to do so, Ben helped us in making a wooden base for a post to hold the camera.

Pallavi – This week, I continued to work on the ultrasonic sensors. In order to actual start working on the serial connection between the arduino and the pi, I wanted to actually set up the USB connection between the two. Because Arushi and Shreyas were mainly focusing on integrating the iRoomba and the openCV, the main raspberry pi was being used. Instead, I got a leftover Pi from a project from a previous semester (thanks Ben!) and spent a good amount of time setting it up.  Because the pi leftover only had one USB port, it was kind of difficult setting it up since I could only use the mouse or the keyboard. I’ve got the basics set up, but I plan on going into lab early on Monday to set up ssh on the pi so I can connect the Arduino Uno to the usb port. With that, my goal is to have the entire design – detecting values, sending the interrupt when an obstacle is detected, and the serial communication done by the time I leave lab on Monday.

Shreyas – This week was mostly making the core of the Robot Control Module. This included adding PID control, so I had to figure out the right constraints and tune the controller to our iRoomba. I also added the interfacing between the Robot Control Module and the Image Processing Module. The PID controller took the center of the target found in the image processing module and sent the necessary motor commands to the iRoomba. The x displacement currently works so the next steps are the z displacement. I also will be adding the iRoomba’s sensors obstacle detection as well in the coming week. Another issue I had to work with was optimizing the connection of the raspberry pi camera. Originally the FPS was very low but by figuring out another method to get the frames we got the FPS to be a lot faster (around 30 now). Now the main latency comes from the image processing module.

All – next week we plan to focus on testing of our initial milestone of the cart following the human down a relatively straight path and stopping when the human does. We expect that we will have to change some of the color detecting constants based on the camera and the pid constants as we continue to test.

Pallavi – This week was mostly spent working on the design presentation and design document with Arushi and Shreyas. From delving more into the design specifics, I started looking into more details of how we’re going to use the ultrasonic sensors. As a team, we were going back and forth about what exactly we wanted to send from the sensors to our path planning algorithm. Our first idea was to only send a 1 or 0 corresponding to whether or not an obstruction is there for a given sensor. However, if we want to support the design specification of not only being able to sense obstructions but also move around them and fix the path, we would want more information about the obstruction. In order to do this, we would need to come up with a serial protocol to encode this data, since we would want to know both the sensor’s value and also which sensor is associated with which value. Shreyas and I discussed how exactly we want to encode the data being sent to the Pi. Our idea is to use one byte for each sensor. We plan to use 3 MSB to ID the sensor since we have 6 sensors total, and the remaining 5 bits to encode the data. Because we don’t need to have specifics about the exact number of cm distance from the obstruction and rather a range, we can figure out a scaling from the sensor value to the actual value we send.

Next week, I want to focus on figuring out the exact encoding of data. I also want to start setting up the physical cart and looking into how to use the on-board iRoomba sensors.

Arushi – This week I worked on the design presentation and doc. To do so, first I made sure to further think through the overall design and how my module of the image processing fits into it. Specifically, after hearing concerns of our path planning and turning algorithm, I looked more into it. We plan to characterize our algorithm to address two types of turns. Wide turns in areas such as a human moving to another side of an aisle and sharp turns such as a human walking out of an aisle and turning a corner. To address the first type of turn, a wider one, we plan to have two circles on the fronts and back of the target human. Our algorithm will draw a minimally enclosed circle regardless of if the entire circle is seen.  This way the further back circle will be smaller and we will able to tell which way the human turned. When only one or no circles is seen is indicative that the human has been lost or made a sharp turn out of the aisle. In this case, we will have the robot make a full turn and stop when a circle is seen. We focused on this, for this week although this will not come into play until after the midpoint demo. Thus, for the next week we want to work more on building the cart and have the separate modules interact with each other.

Another aspect I thought through for future milestones is when the human detection tells the cart to go straight but the obstacle detection doesn’t allow it to do so (let’s say a child gets in between). In this case what I proposed is that we check the  closest sensors to the desired direction and move our robot in the direction of the sensor that shows it is further from an obstacle. Then we do this check again as to where the human is and if the sensor in that direction allows the cart to move in that direction safely without hitting the obstacle.

Shreyas – This week was mostly spent preparing the design presentation and the design document as well as figuring out how to integrate the major subsystems. I created a more elaborate system architecture block diagram that includes the hardware, software, and interfaces between the subsystems. I also looked into the benefits of having a dedicated Arduino to handle polling the ultrasonic sensors over having the Raspberry Pi handle it directly. Having the Arduino handle it will prevent wasting CPU time on the Raspberry Pi that can be used for the image processing algorithms. The GPIO pins on the Raspberry Pi also support interrupt driven handling. We can use this to our advantage by having the Robot Control Module only handle obstacles when a interrupt handler gets fired.

I also confirmed the power supply we would be using for the Raspberry Pi. We are going to buy Miuzei Raspberry Pi 3b+ battery pack. This is an expansion board that also provides a casing so it’s more robust. We can use this board to also power the Arduino.

 

All of us – in general this week we worked on further defining how the separate components will interact with each other, which is further talked about in depth above.

Arushi – This week I worked with Pallavi to identify what our target image will be and then we took model videos with the target attached to my back. Based on the videos I was able to iterate on my general computer vision algorithm to be able to more accurately draw a circle, and print its parameters, based  on the target image. The next step on this end would to take a video where the target stays still and the camera moves up until it is the closest safe distance for the cart to be able to calibrate how we can use the target radius to detect the distance between the cart and the human. I also began designing a model on how we are going to place our hardware on the iRobot create. For next week, I plan on working with my teammates to figure out how we will interface our separate components now that we have proof of concepts for all the separate components.

Shreyas – This week I was primarily in charge of setting up the Raspberry Pi and the iRobot. Getting the Pi on the CMU wifi was a slight bottleneck but eventually it connected. I installed basic libraries and text editors that we would use later. Then I looked into the pyCreate2 python library which provides good abstraction layer to interface with the iRobot. I tested sending basic movement commands and they seemed to work. I started testing reading the sensor data from the iRobot, but more elaborate testing needs to be done.

Pallavi – This week I worked with Arushi to take some videos to train our open CV model. After helping with that, I mainly focused on working with the ultrasonic sensors. I started by individually testing each sensor, and I found one sensor that wasn’t reading in valid responses. I then started setting up the sensors in the orientation we want to use them for our final design. Next week, I will work with Arushi and Shreyas to get a better idea of how we will integrate all of our parts, especially now that we have a little bit of the individual parts working. I also want to look more into the library functions actually used to detect objects. Now that I have the general setup taken care of, I want to implement our detection algorithm and test it.

All of us will be working on the design presentation at the beginning of this week and also work on the design proposal concurrently. Working on this presentation will hopefully help us with our other goal of figuring out exactly how we want all of our individual parts to fit together.