Author: goa500

This week, I worked on segmenting the point cloud data, which involved differentiating between walls and objects within a room and categorizing them into distinct models. Additionally, I developed a simple heuristic-based method to classify these segments into furniture types such as tables, chairs, and couches based on their geometric attributes. I also collaborated with Alana to integrate this new functionality into our web application. This integration involved mapping the location data of each model to its corresponding furniture type and compiling this information into a JSON file. This file will facilitate the dynamic display of placeholder furniture models at accurate positions within the web app. 

The project is currently on schedule. We are currently transitioning into the testing phase. Next week, I plan to make improvements to the models based on our results from design. I will also work with my team to prepare the final project documents and get ready for the final demo.

Over the course of the semester, it was necessary to use tools and technologies to handle 3D point cloud processing, mesh manipulation, and web integration. The key tools I used were Open3D for point cloud processing and CloudCompare for mesh visualization. Open3D has been useful for tasks like segmentation, mesh generation, and transformations within 3D space. Tutorial videos were useful in quickly bringing me up to speed with the functionalities and features of Open3D. I also was able to examine how similar challenges were approached in other projects which helped me gain practical insights into what tools I could use for this project. Reading the documentation and API Guides for the RPLiDAR was also helpful for understanding how to make use of Python for the data acquisition from the sensor.

This week, my primary focus was on developing the furniture classification component of our project. I successfully implemented a multi-label image classification model using Keras. However, initial testing revealed that the model’s accuracy is currently low. To address this, I am exploring several strategies to enhance the model’s performance:

• Experimenting with Different Optimizers: I’m testing various optimizers to find the most effective one for our model, aiming to improve the learning process and, consequently, the accuracy.
• Data Augmentation: Recognizing that our training dataset is somewhat limited (comprising approximately 400 images), I’m employing data augmentation techniques to increase the diversity of the training set. This should help the model generalize better to new, unseen images.
• Pretrained Model Evaluation: As an alternative approach, I’ve tested a pretrained model that could potentially be adapted for our use case. This might offer a shortcut to achieving higher accuracy without the need for extensive training from scratch.

Another task I worked on this week was updating our point cloud processing code to accommodate a tilting LiDAR setup. This update is crucial because capturing scans at different angles alters the method required for converting scans into 3D points, due to the LiDAR’s rotation around an axis. This adjustment is essential for accurately generating 3D models from the varied scan data.Due to the additional considerations introduced by the tilting LiDAR, we’ve had to make some adjustments to our project schedule. Specifically, the method for generating meshes from the LiDAR data will need to be revised to account for the varied angles of the scans. This has slightly shifted our timeline and project milestones.

Looking ahead to next week,  I plan to finalize and test the updated code for processing point clouds from the tilting LiDAR. This involves ensuring that scans captured at different angles are accurately converted into 3D points and integrated into our mesh generation process. I will also continue working on improving the accuracy of the furniture classification model.

For verification, I plan to test the point cloud code with LiDAR scans taken at a range of angles. This will help us understand how tilt angles impact the accuracy of the 3D models and identify better scan strategies. For the furniture classification model, implementing cross-validation techniques will ensure the model’s robustness and generalizability by evaluating its performance across different subsets of the data.

This week, I focused on developing the furniture classification component of our project, which involves a multi-label image classification model. This model’s objective is to analyze images of rooms and identify the presence of specific furniture items from a predefined list. I downloaded a dataset of room images that will be used to train the model and a .csv file containing the names of the training images and their corresponding true labels, which are essential for training and validating the model’s accuracy. I began constructing the model architecture using Keras.

Currently, the classification part of the project is behind schedule. The initial steps of creating the model architecture and organizing the training data have been completed, but there’s still significant work to be done in terms of training and testing the model. To get back on track, I plan to begin the process of training the model with the downloaded image dataset and evaluating its performance using the true labels from the .csv file. I aim to reach a point where the model can be tested to assess its effectiveness in classifying furniture within room images accurately.

This week, I worked on a new method for creating 3D meshes with the Open3D library. This method involves creating connections between slices from the scan data and then combining these to form a full mesh. The goal is to produce better-quality scans of rooms with furniture. I also began a new approach for merging scans taken from different locations in the room. The process starts with a central scan to locate room corners, followed by scanning from these corners. The combination of these scans is managed through mathematical adjustments to align the point clouds accurately. The project is currently on schedule. The method for combining scans from different room locations is making progress.

My plan for next week is to develop the scan combination strategy further. I will also try to develop a method for isolating objects from the point cloud data.

This week, I focused on improving how we combine LiDAR scan data from different parts of a room to make a single 3D model. I used the Open3D library, which is known for its 3D data processing capabilities, including point cloud registration. I’ve made some progress, but aligning the scans from different locations is still a problem. The scans don’t match up perfectly, which has put us behind schedule. Because of these alignment issues, this part of the project is behind schedule. Getting the point clouds to line up correctly is crucial for moving forward and affects the quality of the 3D models we’re creating. Next week, I plan to look for new ways to combine point clouds, beyond what Open3D offers. This might include trying out different registration algorithms for better accuracy. Work on algorithms to fill in gaps in the data. The misalignment means there are areas without enough information. I’ll focus on finding ways to fill these gaps effectively.

This week, my focus was on improving LiDAR data processing and starting the furniture classification component. I managed to combine LiDAR data from different room locations into a single point cloud, a crucial step for creating accurate 3D models of the entire room. Additionally, I began working on furniture classification by downloading a dataset with labeled images and writing preliminary code for data preprocessing and setup for model training.

We’re on schedule with the project’s timeline. The integration of LiDAR data from various positions and the start of furniture classification aligns with our planned milestones.

For the next week, my goals are to advance the furniture classification by training the model on a subset of the data and starting to test its accuracy. Alongside this, I’ll work on improving the 3D mesh quality, specifically developing methods to fill in gaps where LiDAR scanning is insufficient.

This week I continued working with the LiDAR data to generate a mesh 3D model for our project. I developed code to filter the data by removing points that are outside of bounds to improve the accuracy of the resulting model. Additionally, I immplemented code to simulate the process of taking slices of a room at different heights. To test the functionality, I captured a horizontal slice of my room using the LiDAR and replicated it used various z values to generate a a point cloud. I used PyVista to create a triangulated surface from the points and saved the resulting mesh to an STL file, which can be used for integration with our web application. This week, I also worked with my team to start the design report.

The progress made in processing data and generating the mesh is currently on schedule. Next week, I aim to improve the model by using code to fill in gaps present in the data and refine the mesh using PyVista to achieve a smoother surface. I also hope to find a way to combine data from the LiDAR when it is placed at different locations in the room.

This week, I focused on initiating the development of the codebase to process data from the LiDAR sensor. As part of this process, I began writing code to convert the data received from the sensor, which includes angles and distance for each sample, into points that form a point cloud. For this process, I used the RPLidar-roboticia module, which is a module designed for working with the RPLidar laser scanner. I successfully collected data from the lidar sensor and converted it into a set of 3D points in an array. 

In the next week, I hope to finalize the code for generating the point cloud from the LiDAR sensor data. I also hope to delve into testing the creation of a 3D mesh using PyVista.

This week, I practiced and delivered the proposal presentation for our team. Following the presentation, I considered the feedback received and discussed with my team to assess the adjustments needed to be made for our project. Based on the feedback, we decided to focus our devices on getting the room scan and generating a digital model of a room, rather than pursuing the robotics aspect and furniture classification. By mounting a LiDAR sensor in a stationary position within the room, we can obtain a 2D scan and map the room. I believe that the RPLiDAR A1 sensor would be a good candidate for room scanning, based on its ability to rotate 360 degrees and supported software packages. Next week, I will begin planning for the codebase to process the sensor data.