Category: Justin’s Status Reports

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress.

With deadlines upcoming, I contributed to multiple areas of the project. The most crucial area I worked in was implementing the search algorithm on the robotic system.

The second thing I worked on was helping troubleshoot the training of our detection CNN. We tried training this model on a laptop and on google Colab, but neither of these approaches were feasible. So instead we used my desktop. This ended up being more difficult than anticipated due to incompatibility between our CUDA and Pytorch versions. Afterward, I worked with my group members to install PyTorch on the Jetson Nano to use the detection model.

Finally, I continued to make progress on the photo editing algorithm. Our goal is to have a CNN which outputs amounts to apply various image algorithms. To generate data for this training, we need the algorithms to be reversible to determine the target editing modification. Some inverse algorithms (ex. tint inverse) could be mathematically derived. Others (like sharpening and blurring) required experimentally finding a relationship that minimized reconstruction error.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am on schedule with the most recent team schedule.

What deliverables do you hope to complete in the next week?

Complete test results and modifications for the system.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress.

One of our largest integration problems has been streaming the camera’s video using python 3. The demo code/library for the Pan Tilt Zoom camera was written for python 2 and throws an error when run in python 3. This is a problem because all the code we have written is in python 3. I had previously tried fixing the code by changing the streaming method to write to a pipe or using ffmpeg. After working through the errors in these methods, I realized that the problem was actually with the Threading library. I had been confused by the generic error message combined with a nonfatal error for the GStreamer library which occurred directly before (which I thought was the problem). I have spent the latter half of the week rewriting the JetsonCamera.py file and Camera class to avoid using this library, while still interfacing in the same way.

Whenever I was away from my house and did not have access to the Jetson, I continued working on the photo editing feature. I am working on training a photo editing CNN to automatically apply different image editing algorithms to photos. I plan to use the Kaggle Animal Image Dataset because it has a wide variety of animals and also has only well-edited photos.  I am still formatting the data, but I do not anticipate this task should take too long. This part is not a priority, as we identified it as a stretch goal.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am on schedule with the most recent team schedule. I plan to finish integration tomorrow to present in our meeting Monday.

What deliverables do you hope to complete in the next week?

A full robotic system that we can test with.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress.

This week I focused on the integration of our code onto the jetson nano. This process was admittedly far more difficult than we had anticipated. The ptz camera setup has an official Github with code to run on the Jetson, but the camera streaming only works with Python2. When running with python3 I received an error with the GStreamer framework’s interaction with OpenCV. This is a problem that is well documented, but there is apparently no straightforward solution: https://github.com/opencv/opencv/issues/10324. In order, to remedy this problem, I wrote a replacement for the JetsonCamera.py file that changed the streaming method and Camera class to use ffmpeg instead of GStreamer.  This change was successful in getting the cameras operating with Python 3.

I also added a final image editing function, vibrance, to our library. This algorithm is commonly used in photo editing but was difficult to implement because it is not well defined. The general idea of vibrance is to increase the intensity of dull pixels while keeping colorful pixels the same (to avoid getting washed out). I found pseudocode that decompiled a photo editing library’s implementation, so I coded and adapted this for our proect.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

The integration phase has taken far longer than expected. In order to adjust, I am focusing fully on integration instead of photo editing (a less necessary part of the project). Our group has also adjusted our schedule to have a single round of testing with a Minimum Viable Project instead of multiple rounds of development.

What deliverables do you hope to complete in the next week?

Operational search and detection algorithms running on the robot.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress.

I was able to complete a couple of tasks this week. The first, and most important was the physical setup of the robot. I was able to finish all of this task, except for fixing the camera tower to the tripod. This task involved creating a 3 layered tower (camera, jetson nano, camera) where the cameras are screwed in and able to scan their full range of motion.

The next task I completed was updating the sharpening algorithm. We plan on having an algorithm automatically determine the number of effects to apply to our photos. With this in mind, it is important that the scales for applying the image processing algorithms in our library are natural and allow for enough flexibility. Our initial implementation of the ‘Sharpening’ algorithm would apply a single kernel one time for each level of sharpening. By the second or third iteration, the picture was unusable. This leaves very little flexibility in applying this effect. After research, I changed the algorithm such that the amount of sharpening is the size of the sharpening kernel. This led to a much more natural sharpening scale.

I also did the ethics reading and assignment this week.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

We are on schedule with the Gantt chart shown in the design review and still have an excess of slack. We will test our first design this upcoming week.

What deliverables do you hope to complete in the next week?

Initial testing results for the robot’s search/detection and the editing algorithm.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress.

I spent Sunday working on the design review presentation, and I used the dedicated class time for reviewing other groups’ presentations. My main focus for the week was on the physical setup of the project. Our parts came in this week, so I worked with my team members to set up the Jetson Nano and get the cameras connected and working. I also went on a trip to home depot to find the parts needed for the stand: 3 wooden boards, threaded rods, nuts, and bolts. I had a tripod from a previous project.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

We are on schedule with the Gantt chart shown in the design review. We will finish the setup this week, so we can test the following week.

What deliverables do you hope to complete in the next week?

The final physical tripod setup.