Progress this week
This week, each team member got one use case running on recorded video. All three workflows now run on laptops, have automated tests with more than 170 passing across the team, and have been submitted for team review. None has run on the Jetson yet.
Treadmill availability (Alan)
Alan’s check uses pretrained models and no training on gym footage. A pose model finds people’s feet, and a treadmill counts as in use when feet are on its belt. For setup, an open-vocabulary detector finds the treadmills in a photo of the empty row, and a segmentation model outlines each belt. Alan tested two team-filmed clips covering seven treadmills, 66 seconds, and six step-on or step-off events.
- Every change was detected within 1.0 to 3.5 seconds.
- There were no false free readings, and people walking past did not change a treadmill’s state.
- A blocked treadmill was reported as unknown rather than free.
Alan also wrote a Jetson deployment guide and a one-command board check. He rehearsed both on a laptop.
Bench (Jerry)
Jerry built a local check for whether a bench is in use and whether plates remain on its bar. He gathered short private-gym recordings and tested the check on them. He also added a browser preview for reviewing predictions and handling stale observations. The tests exposed problems with camera angle and partly hidden equipment, and accuracy remains below the target.
Squat rack (Harry)
Harry added automatic rack setup with editable floor regions, plus detection of people and belongings. The workflow reports available, in use, waiting, review, or unknown. On four short team clips of one rack, 54 of 63 scored states were correct, nine were unknown, and none were wrong. Item presence was correct in 52 of 64 samples, with 12 unknown. These are development results from one camera view. None of the clips contains a full five-minute unattended episode.
Risks and responses
The largest technical risk is running all three checks on one Jetson. The workflows use different models. Treadmill uses a small pose model, squat adds an item detector to person and pose models, and bench uses several models, including its own equipment detector. Bench is expected to need the most computation, but we have not measured their combined speed or memory use on the board.
We will run the treadmill check first, then measure each workflow alone and all three together. If the combined workload does not fit, we can convert models to TensorRT, check racks and benches less often because their timers run for minutes, share a person detector where practical, or support fewer stations.
The bench check is our least mature workflow and remains below its accuracy target. Camera alignment and recognizing plates on the bar are the main problems. Jerry will isolate those failures, improve the pipeline, and evaluate changes on separate footage. Longer recordings will help test the five-minute timer, but they will not by themselves fix detection errors.
Our evaluation data is limited. Results so far come from short clips, often from one camera view, with labels made by one person or with assistant help. Each owner will record longer scripted sessions, hold some clips back for final testing, and ask a teammate to review a sample of the labels.
Recording permission is still unresolved. Team and private-gym recordings remain outside the code repository. We will confirm permission and consent before showing footage publicly. Access to the campus gym is also unconfirmed. If it is unavailable, we will use a controlled setup with consenting participants.
The three workflows were built in parallel and contain some overlapping code. Squat and treadmill, for example, use the same models during setup. We will agree on a shared data format and shared modules, then merge the workflows in a planned order.
Changes to the design
The main change is using models chosen for each use case instead of one shared detector with separate rules. Testing showed that the checks need different evidence. Treadmills depend on feet on the belt, squat racks depend on a person’s floor position and belongings, and benches depend on whether plates remain on the bar. No pretrained class covers plates on a bar. Using more models increases the likely Jetson workload, so we will measure each workflow and share a person detector where the implementations agree.
We made four other changes.
- Automatic setup The treadmill belts and squat rack can now be found from an empty reference frame, with a preview for a person to confirm. This responds to design-review feedback about setup effort. The larger setup models run once on a laptop rather than on every check.
- Unknown states Each workflow reports unknown when its area is blocked, the camera has moved, or observations are stale. An uncertain view should not be shown as free.
- Treadmill confirmation A treadmill is shown as in use after three of the last four checks agree and as free after five of the last six. Previously, both changes required five of six. This reduced step-on delay by about one second in our tests, though it may make brief false detections more likely. None occurred in the tested clips.
- Bench equipment detection The bench check now uses a trained detector because no pretrained class covers plates on a bar. This adds collection, labeling, and evaluation work. Our recordings remain evaluation-only for current and future training, so we do not test the detector on footage it learned from.
Updated schedule
Jetson work is about one week behind. Alan collected the board on October 2, but setup has not started. Camera and tripod requests submitted this week should make future recordings more repeatable. Fall break remains contingency time.
- By October 6 Agree on the shared data format and merge order.
- By October 7 Inventory the Jetson, run the treadmill check on recorded video, then begin running each workflow alone.
- By October 9 Complete the design report. Each owner will write about their implementation, results with sample counts, and risks.
- By October 10 Record longer scripted sessions, reserve held-out clips, and have a teammate review a sample of the labels.
- Week of October 12 Alan will test a live treadmill camera-to-tablet path on the Jetson. Jerry will address bench failures and test changes on separate footage. Harry will test the squat check with people walking past and more types of belongings.
- Before fall break ends Run all three workflows together on the Jetson and measure latency, memory, and temperature.
- November 9 or 11 Demonstrate three checks running on the Jetson with a tablet dashboard