Posted on

Team Status Report for 3/30/24

Risks:

The stability and reliability of the scissor lift are a risk because instability could result in inaccurate data collection from the lidar. A contingency plan for the scissor lift is using a pulley system to lift the lidar instead.

Another risk is furniture blocking certain areas and being unable to see behind it. This risk will be mitigated by using code to estimate the shapes and areas being blocked.

Design Changes:

We are now developing a pulley system to lift the lidar sensor. This change was necessary because we needed a contingency plan for the scissor lift. We needed to purchase a few new components, but as our budget allowed us to do so, there was not a big cost.

Schedule Changes and Updates:

No major schedule updates.

Developments:

Over the course of the week we stayed true to our word of meeting up more to discuss and work synchronous on the project. We were able to put together some backup plans for some problems (like using pulley) and discuss important steps for moving forward. Getting things in a good state for the interim demo was a big priority.

Posted on

Alana’s Status Report for 3/30/24

Personal Accomplishments:

I spent this week looking at connecting the web app to the model generation code more directly. This was difficult at first because the model generation code is in Python and the web app is written in JavaScript so that it can’t interface directly super easily. I tried a couple of ways of doing this. Originally I tried using a library called react-py that works as a Python interpreter. Type in python and then it gets executed. While I did eventually get that run without any errors, the model generation code we are using was a bit more complicated than react-py could handle. The result was that it took forever for the code to actually execute and even then there were still other issues. After that, I decided to change my approach to using django. 

Django is a Python based backend interface that is often used with HTML. I originally looked for other options as django requires running a server environment and is more commonly used for making forms and interacting with a database. It felt a bit too much for what I needed as I really just needed a single python function to run. Though with react-py not going to plan, django was my next best bet based on my research.

Despite the trouble getting there, I have made progress on getting the web app to react to the python function in the django files. Some adjustments still need to be made to streamline the code and make sure the files go to the right place but there seems to be promise with this method.

I have also spent time fleshing out the web app itself. A lot of small touches like getting the models to highlight when your mouse is over them, changing the tab name of the website to TailorBot Room Designer, experimenting with having text appear over objects in 3D to clarify what they are, making a proper home page for the web app and having the 3D render on another. I have included more meaningful changes like finding a way to more easily move objects in 3D space by simply dragging it and making the code more parameterized to make reusing it easier.

Example of the web app pulling data from elsewhere with django:

 

 

 

 

Showcase model highlighting:

Transform controls on models:

 

 

 

 

 

Progress:

The work I’ve done and my improvement in implementing the Three.js code has made the act of getting new models to appear in 3D space pretty simple. My general react knowledge makes adding to the web app UI a more manageable task. I think the web app itself and my ability to build upon it is in good order. From here it’s really just focusing on those connections with the model generator code and lidar.

Schedule Status: On time.

Next Week’s Deliverables:

Once things with django are sorted out, I want to work on getting the web app to connect to the raspberry pi to start the lidar room scan and then move on to generating the model. I plan to talk more to Zuiheb about that as he has been the one dealing with the raspberries.

Posted on

Zuhieb’s Status Report for 3/30/2024

I spent the first half of the week finalizing the build for the scissor lift, but as I was testing it, I noticed that the weight of the load was too much to allow for the lift to reliably lift the load the distance we wanted. So, I spent the latter half of the week working on implementing the backup plan, a pulley system that would pull the RPLidar up. Other than this hiccup, we are on schedule. Next week, after discussing with the team which method we want to use, I will either tune the scissor lift or improve the design of the pulley system.

Posted on

Grace’s Status Report for 3/30/2024

This week, I focused on developing the furniture classification component of our project, which involves a multi-label image classification model. This model’s objective is to analyze images of rooms and identify the presence of specific furniture items from a predefined list. I downloaded a dataset of room images that will be used to train the model and a .csv file containing the names of the training images and their corresponding true labels, which are essential for training and validating the model’s accuracy. I began constructing the model architecture using Keras.

Currently, the classification part of the project is behind schedule. The initial steps of creating the model architecture and organizing the training data have been completed, but there’s still significant work to be done in terms of training and testing the model. To get back on track, I plan to begin the process of training the model with the downloaded image dataset and evaluating its performance using the true labels from the .csv file. I aim to reach a point where the model can be tested to assess its effectiveness in classifying furniture within room images accurately.

Posted on

Zuhieb’s Status Report for 3/23/2024

This week I was able to get the components to the scissor lift laser cut and get some of the arms linked together. I also worked on moving the code to generate the point cloud from the scan onto a Raspberry Pi, so that there wouldn’t need to be a connection between the scanner and our computer. This week was more focused on setting up a baseline for what we would be able to showcase at the demo next week, so this week I will focus on working with my team to integrate the other parts of the project ahead of the demo. The project is on schedule.

Posted on

Alana’s Status Report for 3/23/24

Personal Accomplishments:

I managed to find a way to get 3D models into the web app. From here it’s just getting the web app to interact with the scan data in real time.  In pursuit of that I’ve also been working on building functions in the code to populate the room with furniture models that the user can move around and interact with, though that’s still in the works.

Uploaded chair and table model:

Uploaded empty room model:

Progress:

I’m at the part of the project where I need to interact with my team members more closely as a lot of the next few steps involve getting the web app to work alongside the scan data and scissor lift controls in real time. I’ve already discussed this with them and plan working with them more as we move forward.

Schedule Status: On time.

Next Week’s Deliverables:

I hope by next week to be able have the web app be able to upload a room model from the lidar scan data in real time.

Posted on

Grace’s Status Report for 3/23/2024

This week, I worked on a new method for creating 3D meshes with the Open3D library. This method involves creating connections between slices from the scan data and then combining these to form a full mesh. The goal is to produce better-quality scans of rooms with furniture. I also began a new approach for merging scans taken from different locations in the room. The process starts with a central scan to locate room corners, followed by scanning from these corners. The combination of these scans is managed through mathematical adjustments to align the point clouds accurately. The project is currently on schedule. The method for combining scans from different room locations is making progress.

My plan for next week is to develop the scan combination strategy further. I will also try to develop a method for isolating objects from the point cloud data.

Posted on

Team Status Report for 3/23/24

Risks:

Once again the greatest risk is getting everything to work together in time for the final showcase. We plan on making extra time to meet up outside of class to make sure all the systems are properly integrated with each other.

Design Changes:

We realized that it would be inefficient to control the scissor lift motor as well as to process the data from the sensor to generate a point cloud on a single Raspberry Pi. Therefore, we separated the two tasks into two Raspberry Pis. We need a second RPI, which we were able to get. In doing this, we have to send the z-position of the sensor to the other RPI.

Schedule Changes and Updates:

No major updates to the schedule.

Posted on

Zuhieb’s Status Report for 3/16/2024

I was able to get the electric components of the scissor lift mechanism working with the Raspberry Pi. There have been some issues preventing me from being able to get the wood necessary to start building the arms, so I focused on getting the electrical systems; namely the linear stepper motor and distance sensor. I need to build out the mechanical component to know what to set the limits of the stepper motor to.  Therefore, my aim next week is to get the wood to cut and assemble the lift and have it working. Due to the unexpected setback, I am a little behind where I would like to have been, but I plan on making the time up this week once the order arrives.

Posted on

Grace’s Status Report for 3/16/2024

This week, I focused on improving how we combine LiDAR scan data from different parts of a room to make a single 3D model. I used the Open3D library, which is known for its 3D data processing capabilities, including point cloud registration. I’ve made some progress, but aligning the scans from different locations is still a problem. The scans don’t match up perfectly, which has put us behind schedule. Because of these alignment issues, this part of the project is behind schedule. Getting the point clouds to line up correctly is crucial for moving forward and affects the quality of the 3D models we’re creating. Next week, I plan to look for new ways to combine point clouds, beyond what Open3D offers. This might include trying out different registration algorithms for better accuracy. Work on algorithms to fill in gaps in the data. The misalignment means there are areas without enough information. I’ll focus on finding ways to fill these gaps effectively.