Posted on

Grace’s Status Report for 4/6/2024

This week, my primary focus was on developing the furniture classification component of our project. I successfully implemented a multi-label image classification model using Keras. However, initial testing revealed that the model’s accuracy is currently low. To address this, I am exploring several strategies to enhance the model’s performance:

  • Experimenting with Different Optimizers: I’m testing various optimizers to find the most effective one for our model, aiming to improve the learning process and, consequently, the accuracy.
  • Data Augmentation: Recognizing that our training dataset is somewhat limited (comprising approximately 400 images), I’m employing data augmentation techniques to increase the diversity of the training set. This should help the model generalize better to new, unseen images.
  • Pretrained Model Evaluation: As an alternative approach, I’ve tested a pretrained model that could potentially be adapted for our use case. This might offer a shortcut to achieving higher accuracy without the need for extensive training from scratch.

Another task I worked on this week was updating our point cloud processing code to accommodate a tilting LiDAR setup. This update is crucial because capturing scans at different angles alters the method required for converting scans into 3D points, due to the LiDAR’s rotation around an axis. This adjustment is essential for accurately generating 3D models from the varied scan data.Due to the additional considerations introduced by the tilting LiDAR, we’ve had to make some adjustments to our project schedule. Specifically, the method for generating meshes from the LiDAR data will need to be revised to account for the varied angles of the scans. This has slightly shifted our timeline and project milestones.

Looking ahead to next week,  I plan to finalize and test the updated code for processing point clouds from the tilting LiDAR. This involves ensuring that scans captured at different angles are accurately converted into 3D points and integrated into our mesh generation process. I will also continue working on improving the accuracy of the furniture classification model.

For verification, I plan to test the point cloud code with LiDAR scans taken at a range of angles. This will help us understand how tilt angles impact the accuracy of the 3D models and identify better scan strategies. For the furniture classification model, implementing cross-validation techniques will ensure the model’s robustness and generalizability by evaluating its performance across different subsets of the data.

Posted on

Zuhieb’s Status Report for 4/6/2024

This week I worked mainly on improving the lifting mechanism. The pulley system was not very stable and adding a tilt mechanism with different-length strings would be very unreliable. Therefore, I worked on building a guide rail system for the scanner, using a counterweight system to eliminate the issue of weight. I also worked on implementing a tilt mechanism so we can get more scans and not be limited to the x-y plane. Next week I plan to finalize both of these components so we start reliably testing for our metrics. This adjustment has put me a bit behind schedule but I will be working extra this week, using the time off with carnival. In terms of verification, I will be comparing the readings from the distance sensor and the true height to ensure that they match. I will also be stress-testing the entire lift mechanism to ensure its sturdiness through repeated deployment.

Posted on

Team Status Report for 4/6/24

Risks:

So far the biggest risk to the project so far is difficulties getting the proper placement and size of furniture inside a room. This task is made difficult due to our lidar lacking a depth sensor. This results in the furniture location data being backed into the wall data as one model. We have a couple contingency plans to counter this problem. One is to make use of the room data and mathematically modify it to separate the furniture data from the wall data. Another is to make use of bounding box functions in Open3D (our 3D model generator) to get the bounds of the furniture, which could make it easier to extract the data. Another idea is to buy an additional depth sensor to work in tandem with our current setup, though this is the least appealing of the options at this stage of development.

Design Changes:

A small change was made to the lifting mechanism of the scanner. A sturdy rail guide is being built to ensure stability when the scanner is taking scans. This will allow us to implement the tilting mechanism and not interfere with the stability of the initial mechanism.

Schedule Changes and Updates:

No major schedule updates.

Plans For Verification and Validation:

 

Verification of Room Dimension Accuracy:

To verify the accuracy of the dimensions of the room, we plan to measure the dimensions with a tape measure. The manual measurements will be compared to the dimensions represented in the 3D mesh generated from the LiDAR scan. We will calculate the difference in measurements to assess the accuracy of the scan.  We aim to establish the maximum allowable error margin based on our design requirements. To assess the reliability of our scanning process, we’ll conduct multiple scans of the same room under the same conditions to see if the dimensions reported by the system remain consistent across scans.

 

 

Verification of Furniture Classification:

To verify the model’s performance, we will use metrics such as precision, recall, and F1 score. Precision measures the model’s accuracy in identifying furniture items correctly, recall assesses the model’s ability to detect all relevant instances of furniture, and the F1 score provides a balance between precision and recall. We will run a series of accuracy tests on the furniture classification model by processing images from the validation set and comparing the model’s classification results against the true labels. This test helps quantify how often the model correctly identifies furniture items.

Developments:

Demo was this week and with it, we have established all the major connections between web app, the lidar and model generation. From here, it’s a matter of refinement and figuring out the last few bugs with our systems to make it as polished as possible for the final showcase.

 

Picture of the pulley for room scanning:

Posted on

Team Status Report for 3/30/24

Risks:

The stability and reliability of the scissor lift are a risk because instability could result in inaccurate data collection from the lidar. A contingency plan for the scissor lift is using a pulley system to lift the lidar instead.

Another risk is furniture blocking certain areas and being unable to see behind it. This risk will be mitigated by using code to estimate the shapes and areas being blocked.

Design Changes:

We are now developing a pulley system to lift the lidar sensor. This change was necessary because we needed a contingency plan for the scissor lift. We needed to purchase a few new components, but as our budget allowed us to do so, there was not a big cost.

Schedule Changes and Updates:

No major schedule updates.

Developments:

Over the course of the week we stayed true to our word of meeting up more to discuss and work synchronous on the project. We were able to put together some backup plans for some problems (like using pulley) and discuss important steps for moving forward. Getting things in a good state for the interim demo was a big priority.

Posted on

Alana’s Status Report for 3/30/24

Personal Accomplishments:

I spent this week looking at connecting the web app to the model generation code more directly. This was difficult at first because the model generation code is in Python and the web app is written in JavaScript so that it can’t interface directly super easily. I tried a couple of ways of doing this. Originally I tried using a library called react-py that works as a Python interpreter. Type in python and then it gets executed. While I did eventually get that run without any errors, the model generation code we are using was a bit more complicated than react-py could handle. The result was that it took forever for the code to actually execute and even then there were still other issues. After that, I decided to change my approach to using django. 

Django is a Python based backend interface that is often used with HTML. I originally looked for other options as django requires running a server environment and is more commonly used for making forms and interacting with a database. It felt a bit too much for what I needed as I really just needed a single python function to run. Though with react-py not going to plan, django was my next best bet based on my research.

Despite the trouble getting there, I have made progress on getting the web app to react to the python function in the django files. Some adjustments still need to be made to streamline the code and make sure the files go to the right place but there seems to be promise with this method.

I have also spent time fleshing out the web app itself. A lot of small touches like getting the models to highlight when your mouse is over them, changing the tab name of the website to TailorBot Room Designer, experimenting with having text appear over objects in 3D to clarify what they are, making a proper home page for the web app and having the 3D render on another. I have included more meaningful changes like finding a way to more easily move objects in 3D space by simply dragging it and making the code more parameterized to make reusing it easier.

Example of the web app pulling data from elsewhere with django:

 

 

 

 

Showcase model highlighting:

Transform controls on models:

 

 

 

 

 

Progress:

The work I’ve done and my improvement in implementing the Three.js code has made the act of getting new models to appear in 3D space pretty simple. My general react knowledge makes adding to the web app UI a more manageable task. I think the web app itself and my ability to build upon it is in good order. From here it’s really just focusing on those connections with the model generator code and lidar.

Schedule Status: On time.

Next Week’s Deliverables:

Once things with django are sorted out, I want to work on getting the web app to connect to the raspberry pi to start the lidar room scan and then move on to generating the model. I plan to talk more to Zuiheb about that as he has been the one dealing with the raspberries.

Posted on

Zuhieb’s Status Report for 3/30/2024

I spent the first half of the week finalizing the build for the scissor lift, but as I was testing it, I noticed that the weight of the load was too much to allow for the lift to reliably lift the load the distance we wanted. So, I spent the latter half of the week working on implementing the backup plan, a pulley system that would pull the RPLidar up. Other than this hiccup, we are on schedule. Next week, after discussing with the team which method we want to use, I will either tune the scissor lift or improve the design of the pulley system.

Posted on

Grace’s Status Report for 3/30/2024

This week, I focused on developing the furniture classification component of our project, which involves a multi-label image classification model. This model’s objective is to analyze images of rooms and identify the presence of specific furniture items from a predefined list. I downloaded a dataset of room images that will be used to train the model and a .csv file containing the names of the training images and their corresponding true labels, which are essential for training and validating the model’s accuracy. I began constructing the model architecture using Keras.

Currently, the classification part of the project is behind schedule. The initial steps of creating the model architecture and organizing the training data have been completed, but there’s still significant work to be done in terms of training and testing the model. To get back on track, I plan to begin the process of training the model with the downloaded image dataset and evaluating its performance using the true labels from the .csv file. I aim to reach a point where the model can be tested to assess its effectiveness in classifying furniture within room images accurately.

Posted on

Zuhieb’s Status Report for 3/23/2024

This week I was able to get the components to the scissor lift laser cut and get some of the arms linked together. I also worked on moving the code to generate the point cloud from the scan onto a Raspberry Pi, so that there wouldn’t need to be a connection between the scanner and our computer. This week was more focused on setting up a baseline for what we would be able to showcase at the demo next week, so this week I will focus on working with my team to integrate the other parts of the project ahead of the demo. The project is on schedule.

Posted on

Alana’s Status Report for 3/23/24

Personal Accomplishments:

I managed to find a way to get 3D models into the web app. From here it’s just getting the web app to interact with the scan data in real time.  In pursuit of that I’ve also been working on building functions in the code to populate the room with furniture models that the user can move around and interact with, though that’s still in the works.

Uploaded chair and table model:

Uploaded empty room model:

Progress:

I’m at the part of the project where I need to interact with my team members more closely as a lot of the next few steps involve getting the web app to work alongside the scan data and scissor lift controls in real time. I’ve already discussed this with them and plan working with them more as we move forward.

Schedule Status: On time.

Next Week’s Deliverables:

I hope by next week to be able have the web app be able to upload a room model from the lidar scan data in real time.

Posted on

Grace’s Status Report for 3/23/2024

This week, I worked on a new method for creating 3D meshes with the Open3D library. This method involves creating connections between slices from the scan data and then combining these to form a full mesh. The goal is to produce better-quality scans of rooms with furniture. I also began a new approach for merging scans taken from different locations in the room. The process starts with a central scan to locate room corners, followed by scanning from these corners. The combination of these scans is managed through mathematical adjustments to align the point clouds accurately. The project is currently on schedule. The method for combining scans from different room locations is making progress.

My plan for next week is to develop the scan combination strategy further. I will also try to develop a method for isolating objects from the point cloud data.