Personal Accomplishments:
A problem we found with the project is that while we could load in a clean room model pretty consistently, getting clean models of the individual furniture models was a lot trickier as our scanner could detect distance but lacked a depth field. To counteract this, I implemented a system that loads in a room model and replaces the messy furniture models with template furniture models (like the ones you see below) and place them in the appropriate furniture locations.
There were a couple of problems along the way. The first problem was that Javascript doesn’t have a lot of good tools to play around with its file system. What is there is good enough when the files are static, but not when the files change dynamically. Since the scanner would be sending new files to the web app as it scanned a room, this was a problem. I looked into a lot of different ways to get Javascript to do what I wanted but ultimately, I managed to get it done with the help of django. Since django already takes care of running the code that creates and sends the files to the web app, we just had it send a json file that includes the position and object type for each piece of furniture detected in the room. Javascript had a much easier time reading from a json file rather than the other methods I had tried. Getting the position of the furniture was tricky due to the mentioned problem of our scanner’s lack of depth perception but Grace was able to get position data into the file.
Aside from that, I added in a couple more tweaks like some instructions on how to use the site and fixed a few bugs.
Progress:
I managed to complete my goal from last week and we are approaching the endgame for this project. The only thing left to do on the web app side of things is more testing to make sure the models are displayed properly when scanning different types of rooms and fix any of the potential bugs that may arise.
Schedule Status: On time.
What I learned during the project:
Starting this project, I already had a good amount of experience with web app design and React. However, something I had to learn a lot about during the project was how to display 3D models in a web app. The first step was just finding a library that could do most of that for me. When it comes to web app design, there are tons of free libraries there to get specific jobs done. Low and behold, I found Three.js to load up 3D models. Three.js on its own didn’t sync up great with my React setup but then I found React Three Fiber, a free React library that uses Three.js as its base. That saved me a lot of time compared to trying to get base Three.js to work with my React setup. From there I learned how to use React Three Fiber by reading its documentation and watching Youtube tutorials on how it worked. There were instances where I couldn’t find a good Youtube tutorial for what I was trying to do so I had to use the documentation and examples of React Three Fiber projects I found online to reverse engineer the mechanics of the library to get it to do what I wanted it to do in my code. Not to mention the trips to sites like StackOverflow to see if anyone had already posted solutions to specific programming problems I ran into.
Next Week’s Deliverables:
I want to make sure that the code displays the models correctly when using different types of room scans. I’ve been testing the current code on dummy data as it’s hard to collect new data on the fly. That requires having the whole setup with scanner lift in place and that isn’t feasible to do whenever. Further testing will make sure the code works in a majority of cases.