Accomplishments this week
This week I built a working local prototype of the treadmill availability check, from camera frames to a dashboard a tablet can open. It uses only pretrained models; nothing was trained on gym data. A pose model (YOLO11n-pose) finds people and their ankles. Each treadmill belt is stored as an outline, and a belt counts as occupied when someone’s feet land on it. Checks run twice per second. A belt is shown as in use after 3 of the last 4 checks agree, and as available only after 5 of the last 6 agree. Stale data, or a belt the camera cannot see, is shown as unknown, never as free.
I also automated the setup step so that belts no longer need to be outlined by hand. From a photo of the empty row, an open-vocabulary detector (Grounding DINO) finds each treadmill, and a segmentation model (SAM 2) outlines its belt. This found every belt in both of our test clips: 3 treadmills in one and 4 in the other. Two further checks make the system more robust. If the camera shifts, the outlines are re-aligned to the empty reference frame. If a belt is blocked from view, it is reported as unknown instead of free.
(bottom).
I evaluated the pipeline in stages, from simple logic tests up to the full system. I first compared four zero-shot methods on stills from public stock footage: person box, pose, segmentation, and a CLIP image classifier. I then tested on two short clips we filmed from a camera angle similar to our planned mount. These clips include people stepping on and off, two people on adjacent treadmills, and people walking past.
– Across 66 seconds of footage, every step-on and step-off was caught.
– It was shown within 1.0 to 3.5 seconds.
– There were no false “free” readings and no state flips caused by people walking past.
– The CLIP classifier was rejected, because a person on a neighboring treadmill made empty belts look occupied.
– A replay through the dashboard matched the offline results exactly.
– In a synthetic test, a blocked belt changed to unknown within 2 seconds.
These are small-sample laptop results, not a validation of accuracy at the gym. The clips were labeled by one person.
To prepare for the Jetson Orin Nano, I wrote a treadmill deployment guide on top of Jerry’s board runbook. I also wrote a one-command board check that records performance and compares results against our laptop reference, and a model export script for TensorRT. I rehearsed every step on my laptop up to the point that requires the board. That rehearsal found a bug where an unplugged camera stopped the server, which is now fixed. The code is organized into shared building blocks that the bench and squat workflows can reuse. It is in a pull request for team review.
Schedule status and corrective actions
The treadmill software is ahead of schedule for a local prototype. The main gaps are hardware and data. I have not yet run anything on the Jetson, and our test footage is short and comes from a single location and person. Recording permission for that footage still needs to be confirmed.
Next week I will run the board check on the Jetson as soon as I have access. If the per-check time is too high, I will first reduce the cost of camera re-alignment, which is currently the slowest step, before considering further model optimization. I will also record a longer, scripted set of clips with consenting teammates. These will include harder cases, such as someone standing beside or behind a treadmill and a crowded row. Some of these clips will be held back so that the final evaluation does not reuse tuning data.
Deliverables for next week
– Run the treadmill board check on the Jetson Orin Nano. Record per-check time, memory, temperature, and power mode, and compare the results with the laptop reference.
– Export the pose model to TensorRT on the board. Use it only if its results match PyTorch on our labeled clips.
– Run a live test from the C922 camera to a tablet, with scripted step-on and step-off events and a camera unplug test. Measure the time until each change appears on the tablet.
– Record and label additional treadmill clips, keep a held-out set for final testing, and have a teammate independently check a sample of the labels.
– Confirm recording permission for the clips we have filmed so far, and contribute the treadmill sections to the October 9 design report.