Category: Team Status Reports

This week we mainly fixed up the kinks during integration. We all worked together to try and iron out some of the path planning issues that we were seeing when testing the lead vehicle’s navigation system. Part of the reason for the inconsistency was the frequency of updates of both the path planning node and the localisation node. After decreasing the frequency of updates we were able to observe much better performance .

We also tweaked with the weight distribution of the cars to give it a more smooth motion and achieve less drift overall. Some of the differences in material and other irregularities with the optical optoencoder and wheel dynamics made the vehicles operate slightly differently and it took a little bit of calibration to allow the vehicles to move in a somewhat similar manner. With more time we would have liked to see identical movement by implementing better feedback and maybe passing more specific data between the cars, however, due to time constraints we were left with small hacks to make the motion pretty similar.

Joel also tweaked some issues with the communication to allow for more consistent communications.

The rest of the week was dedicated to testing the system together to try and iron out any issues we saw. Although the vehicles are a little finicky and their behaviour is a little unpredictable due to subtle changes in the positions of the motor axes, as well as accumulation of dirt in the wheels and terrain and small perturbations in the vehicle themselves, we were able to have our two cars successfully navigate a simple track with 5 obstacles.  Since the drift between the two vehicles became too significant after 5m, we ended up shortening our demo video to a track of 5m in length.

This week our team focused on preparing our subcomponents so we could integrate them together properly. In regards to the vehicle mechanics, we switched from standard wheels to mecum wheels, which allows for the vehicle to more easily maneuver around the course and maintain more accurate localization information. Additionally, accommodations to the body of the car were made in order to support the camera being mounted on the vehicle. This week we ran into issues with running our object detection program on the Nvidia Jetson. We ran into an issue where the network we were using was too big and consuming too much power and memory, so the frame rate was too low for our usage. So we moved to a Tensor RT base for the neural network inference and switched to mobile net-v1 since it was too difficult getting v2 to load. This solved the issue and we are now steadily hitting around 30 FPS and no longer run into resource issues. Finally, throughout the week we focused on migrating our code into ROS and looking forward to next week, we will fully integrate our systems.

This week our team focused on perfecting and finalising the individual subsystems to make integration go smoother. Joel finalised and updated the actual vehicle specifications and characteristics, and amended the odometry functionality of the vehicles. Construction of the remaining vehicles will be completed at a later date once integration begins. Joel and Fausto have both begun porting their modules over to ROS and developing a broadcast system to make communication between the subnodes easier. Fausto also begun preparing the communication protocol and establishing limits as to how far the vehicles can be before connection breaks. He found that the connection is extremely robust to distance and once the cars establish a connection at the beginning they are highly unlikely to have that connection broken. Additionally, Fausto has also begun researching more about ROS and figuring out the best way of designing his module in order to have it seamlessly integrate on the ROS platform. This will eventually allow for a more fluid integration, as they can help integrate the object detection node into the ROS platform during next week. On the object detection front, Jeffrey began finalising the pipeline and integrating the Intel RealSense camera with object detection. Rather than using RGB-D image values, only RGB will be inputted to MobileNet v2, and the resulting bounding box with additional padding will be redrawn onto the depth map to extract the distance of the object from the car, and extrapolate what angle at which to turn the wheels in order to avoid the obstacle. The depth map wit the bounding box will also be broadcasted to the localisation node to further develop the map of the environment.

Looking ahead, we want to begin integration next week, and hopefully have an integrated product for interim demos. When integration inevitably runs into issues, our subsystems are ready to demo and are pretty much finalised. This makes integration our main priority, and we will also begin to further develop our communication protocol beginning next week.

This week our team made great progress on individual components for our project. A lot of our hardware parts arrived so we were able to do a lot of hands-on work with them; time was devoted to installing the proper drivers and setting up the new hardware parts. Soon enough we will enter the integration stage of our project, but next week, we will be tying up loose ends in the specific project areas we worked on (detection, mechanics, communication).

In regards to our detection system for our lead car, we experimented with the intel RealSense camera (previously we have been testing our detection algorithm using OpenCV and our laptop camera), We were able to extract RGB values and use them to classify objects detected. Additionally, we tested robustness and how moving the camera around would affect the image feed + process speed of MobileNet v2 and the observed behavior is meeting our expectations.

In regards to our communication architecture, we were pleased to find that after obtaining Bluetooth adapters which integrated with the Jetson Nanos successfully, connection was established easily. We found that stable connection is maintained within 3m and that when a connection was formed it did not break (we need to ensure this is true when both Jetsons are moving in the upcoming week with further tests). We found useful information about the Bluetooth connection and will work on making the Bluetooth software between boards as robust as possible.

In regards to the vehicle mechanics, we received new batteries and UEBC device to power Jetson and we are pleased that they last more than an hour under a stress test ! Additionally, a new CAD design was made to accommodate the new hardware components. The IMU component we are using does give us an sub-optimal sense of displacement and localization so we will be adding an optical wheel encoder for the 2 rear wheels so that our vehicle localization will improve. The drive system is almost done, along with refined code for the vehicle (the serial communication now works such that the Jetson can signal the vehicle motors on how to move).

In the upcoming week we are looking to ensure our specific project areas are functioning as expected and are robust, in order to begin integrating components.

This week everyone on the team was able to get the respective parts needed for their portion of the project. The last week we discovered that we would have issues with power give our previous specs so this tweak was focused on correcting some of those issues. With some research and testing we were able to settle on 2S 7.4v LiPo batteries for our unified power delivery system. We also switched from L298N bridges to TB6612FNG for our motor controller for much more effect motor control design.

For vehicle mechanics we are mostly done with the mockup vehicle and have a few more systems to finalize before we make the real vehicles. Next week we are planning to go to the track with the mock vehicle to do some testing for potential fixes to our current track parameters and potentially some odometer/localization testing.

For object detection have started to work with MobileNetV2 on laptops with small scale examples to simulate what we will see on the track. For now we need to focus on meeting our specs for object detection and moving forward we will get more realistic data to tune the model with. For performance we are considering a few things to ensure we meet what we need. For example we may reduce number of FP bits from 32 to 16.

For communication we have started framing the Bluetooth comms architecture and are able to send bytes between the Jetson and another computer. Moving forward we need to scale up the comms to handle more than one connection and keep connection alive for a more realistic testing scenario. Next couple of weeks we hope to be able to test Jetson-to-Jetson connections.

For this week, we mostly reviewed our feedback from the design review presentation and adjusted our scope and design a little and started implementing small parts. We finalised the CAD model for our RC cars, and the parts arrived late this week, so we will begin assembly this weekend and into next week. After that we will begin testing our RC car design using the metrics we have outlined in our design and proposal presentations. On the object detection front, we have multiple candidate algorithms that we will be implementing and experiment locally first, while hoping to port them onto the Jetson hardware once they arrive.  We will also start experimenting and developing the communication protocols for the V2V communication. We still need to work out how to incorporate the IMU data into the communication and how exactly we will lead the following car to ensure it stays on track and doesn’t drift into any of the obstacles.

Moving forward, we hope to have the cars assembled and begin testing by the end of next week, as well as tinkering and ironing out the details in the object detection algorithm so we can focus on a control algorithm for planning and figuring out some of the convoy mechanics.

This week, our team has made great progress in narrowing down technical design choices for our project. Last week we decided to replace the LIDAR sensor on our lead car with a PS4 camera to simplify the object detection and this week, our team acquired the camera and will soon set it up, such that it can be used on our RC car. Additionally, in regards to the hardware for our RC cars, the CAD work for our car has just about been finished so we will be able to construct the car soon. We have also decided how the motor system will be setup for the car turning mechanism; we will use two LN298 H bridges to control the front and rear motor pair which will hopefully give us the agility in turning that we want.

We will be constructing up to 3 cars (the lead car being equipped with a camera for object detection). The lead car plays a huge role in our project so we will begin working on object detection as soon as possible. This week we read up on a R-CNN algorithm that uses RGB-D information for state of the art object detection, so we will experiment with that. This means we will want to setup the PS4 camera next week to see what we’re working with.

This week consisted of a lot of research for our team, and served as an awesome week for feeling out the scope of our project and getting more comfortable with the technologies we will be working on. We worked hard to come up with metrics we want to hit for our project and worked on the upcoming presentation. As we lay the groundwork for our project, we will also start thinking more deeply on tunable/modifiable design choices (for example, what type of information to send between cars: raw data, instructions, etc.).

This week the group gave the proposal presentation to introduce the project idea and solution approach. As a result of the feedback from the presentation, the group focused on correcting what was subject to critique in our proposal.  We decided to move away from including LIDAR as a form of image sensing for our devices and opt for the use of cameras and odometer of perception and localization. The decision to drop LIDAR was mainly due to concerns about the fidelity of the LIDAR capture at the price point that we could afford for the project as well as the difficulty that may arise in processing the raw data for use in the autonomous vehicle system.

We opted to replace LIDAR for depth sensing with a stereoscopic camera device, namely the PS4 camera. Research from this week has shown us that this camera may serve our purposes adequately at an affordable price point. We were also able to make strides in the vehicle mechanics as we now have a CAD model going with a growing parts list that we hope the finalize by the end of next week. Finally, we have started looking into creating a more explicit set of requirements, goals and testing for our project. We anticipate the we will have progressed far enough by the end of the week to have a rich set of requirements and goals backed by our decision choices in the hardware and software space.

Goals for this coming week include experimenting with the PS4 camera to verify its usability, finalizing CAD model and RC car parts list, developing and documenting more specific goals and requirements which connect qualitative and quantitative measures, and finalizing the design review presentation.

This week the team focused on getting the details of the project finalized. This included discussions with our assigned TA and professor to evaluate the state of our ideas. The meeting concluded with our team settling on the task of a convoy system to demonstrate as a use case for V2V communication. The rest of the week was spent on researching some of the core technologies we intended to use for our project. Some of these insights led us to settle on technologies for communication and perception. We also spent time preparing the our planned schedule for the rest of the semester. This lead us to decide on tasks for each of us to complete which are outlined in greater detail in our proposal documentation.

In the next coming week, the team will be focused on getting the necessary materials finalized according to our budget constraint. This will include our design decision regarding the mechanics of the RC cars. We would also like to setup team infrastructure such as git hub so we can better manage and coordinate our tasks.