Finished integrating the code for project. So, we are now able to place banana on the belt, take picture as the fruit is moving, and get an output from the nano.
We realized the algorithm was slow, so we also sped that up
Almost done getting the servo working; didn’t realize the PCA9685 board needed power. We are waiting on the power bank, but in the meanwhile, i’m trying to find a battery.
Started integration test. Currently, we have been able to take pictures from the cameras, and pass them along to the algorithms and get a final result.
Almost done with servo + shield (should have the system running by monday). After which, we will have full system integrated (i.e from taking picture to controlling gate via the jetson nano). This is the next major step for me.
Wrote a program that streams a video from both the cameras (usb and csi) simultaneously. Pressing ‘j’ takes a picture from both cameras, ‘k’ from csi camera and ‘l’ from usb camera. q quits.
left is csi, right is usb.
Quality of csi is severely lacking. I’ll look more into how gstreamer works since that’s what i’m using for csi. usb cam was able to connect directly, so quality is better for that reason?
Also will try to play around with framerate
Assembled conveyor structure (now only missing the belt)
Got the CSI camera to work in burst mode, with the timing of the burst being configurable.
Trying to get the usb camera to take a picture, but it’s proving to be a lot harder. It’s possible that the usb camera isn’t compatible with the nano, in which case we need to order another usb camera. Because of this setback, I’m slightly behind on schedule, but am confident I can make up for it this week.
3D printed the parts for the conveyor belt, now we just need to assemble the whole thing.
Goal for next week is to get the usb camera working, and make sure that the motor controller board and the 12V power adapter works with the conveyor belt (and to assemble the belt itself).
Got one camera working with the jetson nano. Able to take pictures now automatically. Next step is to process livestream as was recommended in the feedback.
Picked up the rollers for the conveyer belt from Home Depot, and tweaked the design for installing the motor into the rollers (we will now use a wooden piece, glue it to the inside of the pipe, and secure it to the shaft of the motor).
For next week, my main goal is to continue working on the conveyer belt (hopefully have it finished), and to order the second USB camera and instill it.
Worked on getting camera to take pictures every 4s and saving on disk
Worked on design for the conveyer belt more. Finalized dimensions, materials (including parts that need to be 3D printed such as bearings), and how different parts will fit together. Also, sketched different views of the conveyer belt as per the advice of the people in the machine shop. Next step is to prototype the design with cardboard and glue.
Finalized the conveyer belt design to some extent . 1) rack and pinion and 2) treadmill are contingency plans.
Ordered the Nvidia Jetson Nano 2GB and the raspi camera. Started playing around with them, but haven’t gotten hello world to work yet. Ideally, want to have the camera take pictures and save on disk in the next week.
Made design choices for the non-mechanical parts and helped write the design presentation.
Watched tutorials on the jetson nano and learned about all the machine learning / AI capabilities the board has to offer. Found the sdk to be very powerful.
Decided to move away from the rotating plate so that we can incorporate non-round fruits (e.g. bananas)
Developed frameworks for pixel isolation (segmentation and gaussian distribution).
Decided on bananas as the primary fruit since discoloration is more prominent and easier to detect. Still need to learn about HSV.
Started brainstorming ways to isolate fruit through segmentation and detect localized pixels of different colors. This method seems easier than NNs, so preferring this, but keeping options open.
Also started brainstorming on how to differentiate between fruits based on pixel distribution.
Preparing the slides and talking points for proposal presentation
Started looking into Jeston Nano Tutorials and JetPack sdk.
