- This week, I worked on finishing the Bluetooth implementation from the laptop’s end. I also tested it by connecting to each characteristic and checking that the outputs were correct via printing from the serial monitor. I also began working on a function that given the number of rotations the motor needs to make, the motor will run. I may need to re-write this function, as our team is working on selecting a new motor, however as we will still be using the same motor shield to interface with it, this code will be useful.
- I was able to catch up to where I am supposed to be on the schedule.
- Next week, I plan to have the code for both motors (the height motor and the linear actuators) fully finished and tested.
Olivia’s Status Report for 3/16
This week, I worked on a function that get the real life distance a laptop would need to rise based on the distance in pixels. To do this, I’m using the camera calibration as described in the OpenCV documentation to find the focal length of my camera and used these in my calculations assuming the distance from the screen is constant.
I also integrated the calibration setup for gathering facial landmarks with the current landmark distance calculation, so that we are using the calibration data points to calculate the reference difference and compare them to current landmarks. I’m able to detect if the camera is at a height lower than the one determined at calibration, higher, or within threshold.
However, when neither the height or pitch of the laptop are known, it is difficult to determine whether the correct course of action for the system will be to raise the height of the stand or to angle to the laptop using the linear actuators since either a change in height or change in angle can produce similar results in vertical landmark difference. In order to solve this issue, I’m working on using a few vectors between landmarks to measure the skew of certain sections of a user’s face to be able to decide if the camera is at an angle or at a height difference.
I am currently on schedule, and next week I will continue working on this height vs. angle determination. I also hope to begin creating a starter GUI to display basic functionalities such as calibration and setup.
Team Status Report for 3/16
The biggest current risk is still the mechanical design as it has many uncertainties and because the software components have been mostly fleshed out already. To mitigate this risk we have gone ahead and ordered the platform jack we plan on using. We will begin testing next Monday. Depending on the friction of the linkages in the jack, we will determine how much torque we need in our stepper motor.
In case the platform jack fails for whatever reason (too much torque required or its top platform is not level), we will purchase a more expensive jack that is manufactured better but is also heavier.
One minor change to the design is that we are now planning on using bluetooth communication between the computer and the arduino since this is simpler than we thought and will contribute to our use case requirement of our product having a smooth and simple user interface. There were no real costs associated with this change since getting the bluetooth code written was minimal work.
Sebastian’s Status Report for 3/16/24
This week I finished the basic proof of concept for the posture detector. Using Google’s BlazeNet pose estimation model, I was able to implement some basic thresholding on a single shoulder landmark to determine when the user is slouching. The video below shows myself sitting up straight briefly so the program can take a “snapshot” of good posture, then whenever I slouch I receive a message at the terminal telling me to fix my posture. If I don’t fix it, I keep receiving messages every 2 seconds. Once I fix my posture, the messages stop.
The link to the video on google drive is here.
I simply calculated the distance between the shoulder landmark in each frame and the “reference” frame showing myself having good posture which was captured at the start of the video. Some problems with this is that it will produce many false positives. If I shift left to right then I will get notified even if my posture remains good. For this reason, next week I will begin training a neural network for posture detection. The ability of a neural network to learn a rich and complex decision boundary makes it suitable for this application. When classifying posture in a person, many types of movement of landmarks on the body do not correspond to bad posture. I hope that the neural network will be able to learn to distinguish between movement that hints at bad posture and movement that is ok. I will be using a google co-lab notebook to train a simple neural network.
Another benefit of the neural network implementation is that our team simply has to learn from posture research what good posture is, take many photos of ourselves with that good posture, and do the same for bad posture. The neural network will do the hard work of learning what the difference is.
I am on schedule.
Team Status Report for 3/9
Currently, the most significant risk to the success of the project is the mechanical side (mainly including the platform jack). We have already made proof of concepts for the posture tracking and facial landmark tracking which will be used for motor control. However, what remains a question is whether the actual stand will hold up to the stress from the computer on top, whether the motor will be forceful enough, whether the stand will stay level, how much friction there is in the platform jack screw, how big and heavy the whole stand will be after our modifications to add a battery/arduino compartment and the linear actuator compartment.
The way this risk is currently being managed is by ordering the stand as soon as possible to begin testing. The stand will be ordered at the start of this week. Additionally, Sebastian has done a lot of research on different stands by reading customer reviews to select the best one. The contingency plan is to simply buy progressively more expensive platform jacks which (according to reviews) are more stable and are machined with greater precision so they have less friction.
The only change we made to the existing design is to modify an off-the-shelf platform jack instead of making one from scratch. This change was necessary because creating a new platform jack was outside the scope of our experience. The only cost is the price of the stand, the design time of the stand has gone down because now Mary only has to design the compartments above and below the stand.
Part A (written by Sebastian):
Our product has an immense global impact because poor posture is a problem affecting almost all industrialized countries where consumer electronics are common. The most widely used consumer electronics are mobile phones and PCs, both of which promote poor posture. This contributes to the global health crisis where people in wealthy nations are becoming unhealthy. As a result, they require more and more expensive medications and treatments, which causes inflation, which lowers the quality of life even more for most people. So, as a result of poor posture, a vicious cycle is created. While this statement is a bit of a stretch, I believe that using technology to collectively improve our posture will considerably improve the general health of ourselves. Now SmartStand will hopefully only make a small dent in the general posture health of the public, but it is a first, and important step.
Part B (written by Olivia):
Our project was created with the target audience being students and people who may work remotely. Since these groups of people often have to work from public places and many different areas, they may not be able to have the most comfortable working environment. This is why we decided to include the height adjustment functionality of a laptop stand, in order to help such individuals improve their productivity and reduce pain from using their laptop for extended periods of time. Additionally, since these groups are often commuting and traveling, we are constantly considering the portability and convenience of our product, which is why we decided to make SmartStand battery powered, compatible with bluetooth, and compactly designed.
Part C (written by Mary Rose):
Seeing as our product is not very large, or particularly hard to produce, it is difficult to determine if it would have any effect on natural resources, or non-human living organisms. In addition, our device is intended to be used indoors, so it will not have much interaction with the environment. However, as one of the main goals of our device is centered around improving human health (helping people to fix their posture), it can be said that our device would have a positive effect on some living organisms (humans).
Olivia’s Status Report for 3/9
This week, I have been working on the Design Review, specifically the Abstract, Introduction, Test, Verification, and Validation, and Project Management sections.
I also wrote a script that will gather all the landmark coordinates of a single frame and write that information to a text file. After our meeting last week, we decided to include 2 calibrations stages: one for gathering reference landmarks and one for gathering posture information. I plan to use this script to achieve the former by saving the user’s facial landmark data from the calibration stage to reference for future use when adjusting the stand’s height automatically.
I am currently on schedule and plan to integrate these distance calculations and reference checks with motor movement in the upcoming weeks.
Sebastian’s Status Report for 3/9/24
Over the past week I personally finished my sections of the design report. This involved completing the system specifications and trade studies sections. This was a useful exercise because it forced me to finalize the design of the mechanical stand. After conducting research on many types of platform jacks, I concluded that we should use an off-the-shelf platform jack and build an attachment for the screw so that a motor can rotate it.
I chose this path because designing a platform jack is a complex task and not the focus of 18500. Additionally, the modification to the stand will not be trivial. Like mentioned before, we will have to attach a motor to the stand, and also create a compartment (3D printed) below the stand to house the battery, arduino, motor shield. We will also need a similar compartment above the stand to store the linear actuators so that we can pitch the computer.
Overall, through writing the system specs and trade studies sections of the report, I became much more confident that our design will be feasible as I have done as much research as reasonably possible to ensure that our design choices will result in a product that meets our quantitative use case requirements. The trade studies section was especially useful to narrow down choices that were still up in the air such as what type of platform jack to go with, whether to go with wired/wireless connection, whether to use platform jack or stepper motors with lead screws, and other such tossups.
We are on schedule as the mechanical design is 90% complete and proof of concepts have been demonstrated on the software side for all software features.
Mary Rose’s Status Report 3/9
- Over the past two weeks, I worked on getting the Bluetooth connected between the stand and the app. I finished the Bluetooth implementation on the Arduino side. I also wrote some basic code for the Bluetooth connection from the laptop app.
- I was able to catch up to where I am supposed to be on the schedule, this is because I moved some tasks around to make them more efficient.
- Next week I plan to finish the Bluetooth from the laptop end and test the connection using print statements.
Sebastian Weekly Status Report Week 7
This week I finalized the design of the computer stand by making a CAD file combining the platform jack design we selected last week with the linear actuators we selected a few weeks ago. I made holes in the off-the-shelf jack design to allow the linear actuators to flip over. I used OnShape for the CAD design.
I also began testing with OpenCV’s posture detection libraries. I used an off-the-shelf program on a video stream of myself and generated the posture landmarks for each frame. I then compared the positions of these landmarks to a “good posture position” which the user would make during the calibration stage. I think there is enough deviation in the landmark positions that a neural network, trained with enough data, can accurately classify between slouching and not slouching. Training the neural network will be done next week.
Olivia’s Status Report for 2/24
This week, I worked on the OpenCV facial landmark detection for the stand height automation. I wrote a Python program to determine the distance between any 2 specified landmarks and tried a bunch of different face angles to see the which landmarks would be good references to use. I found that the distance between the eyebrows and the tip of the nose would be one good set of landmarks to determine head tilt use because it had significant and consistent change in distance when a user is looking up, straight on, or down.
I also worked on the Design slides and practiced for the presentation this week.
By next week, I’d like to have the acceptable thresholds of distances between landmarks determined that will signify that the ideal height has been reached and write a function to decide this.