Author: alanw2

Alan’s Status Report for 10/10

Accomplishments this week

Most of my time this week went into the design report. Feedback asked us to justify our design choices with real comparisons, so I ran a few quick experiments on my laptop using our existing treadmill clips. No training was involved.

– Image subtraction vs. our pose model: Feedback asked whether simple image subtraction could replace deep learning, so I tested several classical methods on our two treadmill clips. The best one compared each frame with a photo of the empty belt and got 87.8% of checks right. However, it made 14 errors on settled states and flagged 11 of 20 people walking past as using a treadmill. The pose model got 98.6% right with zero errors, because it checks whether someone’s feet are actually on the belt instead of reacting to any change in the image.
– Delay: Our worst case of 3.5 s of delay breaks down into about 1 s where even the label was ambiguous (mid-step), 0.5 s of model lag, and 2 s of voting. Measured from when a person could actually tell, every change showed up within 2.5 s, well under our 5 s target.
– Voting rules and model size: Other voting windows made no fewer errors and only added delay, so we’re keeping 3 of 4 checks for “in use” and 5 of 6 for “available”. The larger YOLO11s-pose model gave the same results at about twice the CPU time, so we’re sticking with YOLO11n-pose.
– Test planning: I worked out how many trials we need to back up our accuracy targets: at least 36 error-free trials to support 92%. This set the size of our test plan.

Classical image-subtraction methods vs. the pose model on our treadmill clips. The pose model (star) gets every check right; the classical methods trade missed people for false alarms.

I also double-checked our treadmill labels against the video.

Schedule status

The treadmill software is still ahead of schedule as a laptop prototype. However, since the design report took priority, I didn’t get to last week’s hardware tasks: the Jetson board check, the live camera-to-tablet test, recording new clips, and the recording permission. These are now my main risk. Our schedule has the first Jetson tests in the week of October 19, after fall break, so I’ll start with the board check since everything else depends on it.

Deliverables for next week

– Run the treadmill board check on the Jetson by October 21 (timing, memory, temperature), and compare it with my laptop results.
– Try a TensorRT export of the pose model, and keep it only if its results match.
– Run a live test from the C922 camera to the tablet, timing how long each change takes to appear.
– Submit the UC gym recording permission request, and confirm our backup gym.
– Record more treadmill clips with teammates, saving some for final testing only.

Team Status Report for 10/3

Progress this week

This week, each team member got one use case running on recorded video. All three workflows now run on laptops, have automated tests with more than 170 passing across the team, and have been submitted for team review. None has run on the Jetson yet.

Treadmill availability (Alan)

Alan’s check uses pretrained models and no training on gym footage. A pose model finds people’s feet, and a treadmill counts as in use when feet are on its belt. For setup, an open-vocabulary detector finds the treadmills in a photo of the empty row, and a segmentation model outlines each belt. Alan tested two team-filmed clips covering seven treadmills, 66 seconds, and six step-on or step-off events.

  • Every change was detected within 1.0 to 3.5 seconds.
  • There were no false free readings, and people walking past did not change a treadmill’s state.
  • A blocked treadmill was reported as unknown rather than free.

Alan also wrote a Jetson deployment guide and a one-command board check. He rehearsed both on a laptop.

Bench (Jerry)

Jerry built a local check for whether a bench is in use and whether plates remain on its bar. He gathered short private-gym recordings and tested the check on them. He also added a browser preview for reviewing predictions and handling stale observations. The tests exposed problems with camera angle and partly hidden equipment, and accuracy remains below the target.

Squat rack (Harry)

Harry added automatic rack setup with editable floor regions, plus detection of people and belongings. The workflow reports available, in use, waiting, review, or unknown. On four short team clips of one rack, 54 of 63 scored states were correct, nine were unknown, and none were wrong. Item presence was correct in 52 of 64 samples, with 12 unknown. These are development results from one camera view. None of the clips contains a full five-minute unattended episode.

Risks and responses

The largest technical risk is running all three checks on one Jetson. The workflows use different models. Treadmill uses a small pose model, squat adds an item detector to person and pose models, and bench uses several models, including its own equipment detector. Bench is expected to need the most computation, but we have not measured their combined speed or memory use on the board.

We will run the treadmill check first, then measure each workflow alone and all three together. If the combined workload does not fit, we can convert models to TensorRT, check racks and benches less often because their timers run for minutes, share a person detector where practical, or support fewer stations.

The bench check is our least mature workflow and remains below its accuracy target. Camera alignment and recognizing plates on the bar are the main problems. Jerry will isolate those failures, improve the pipeline, and evaluate changes on separate footage. Longer recordings will help test the five-minute timer, but they will not by themselves fix detection errors.

Our evaluation data is limited. Results so far come from short clips, often from one camera view, with labels made by one person or with assistant help. Each owner will record longer scripted sessions, hold some clips back for final testing, and ask a teammate to review a sample of the labels.

Recording permission is still unresolved. Team and private-gym recordings remain outside the code repository. We will confirm permission and consent before showing footage publicly. Access to the campus gym is also unconfirmed. If it is unavailable, we will use a controlled setup with consenting participants.

The three workflows were built in parallel and contain some overlapping code. Squat and treadmill, for example, use the same models during setup. We will agree on a shared data format and shared modules, then merge the workflows in a planned order.

Changes to the design

The main change is using models chosen for each use case instead of one shared detector with separate rules. Testing showed that the checks need different evidence. Treadmills depend on feet on the belt, squat racks depend on a person’s floor position and belongings, and benches depend on whether plates remain on the bar. No pretrained class covers plates on a bar. Using more models increases the likely Jetson workload, so we will measure each workflow and share a person detector where the implementations agree.

We made four other changes.

  • Automatic setup The treadmill belts and squat rack can now be found from an empty reference frame, with a preview for a person to confirm. This responds to design-review feedback about setup effort. The larger setup models run once on a laptop rather than on every check.
  • Unknown states Each workflow reports unknown when its area is blocked, the camera has moved, or observations are stale. An uncertain view should not be shown as free.
  • Treadmill confirmation A treadmill is shown as in use after three of the last four checks agree and as free after five of the last six. Previously, both changes required five of six. This reduced step-on delay by about one second in our tests, though it may make brief false detections more likely. None occurred in the tested clips.
  • Bench equipment detection The bench check now uses a trained detector because no pretrained class covers plates on a bar. This adds collection, labeling, and evaluation work. Our recordings remain evaluation-only for current and future training, so we do not test the detector on footage it learned from.

Updated schedule

Jetson work is about one week behind. Alan collected the board on October 2, but setup has not started. Camera and tripod requests submitted this week should make future recordings more repeatable. Fall break remains contingency time.

  • By October 6 Agree on the shared data format and merge order.
  • By October 7 Inventory the Jetson, run the treadmill check on recorded video, then begin running each workflow alone.
  • By October 9 Complete the design report. Each owner will write about their implementation, results with sample counts, and risks.
  • By October 10 Record longer scripted sessions, reserve held-out clips, and have a teammate review a sample of the labels.
  • Week of October 12 Alan will test a live treadmill camera-to-tablet path on the Jetson. Jerry will address bench failures and test changes on separate footage. Harry will test the squat check with people walking past and more types of belongings.
  • Before fall break ends Run all three workflows together on the Jetson and measure latency, memory, and temperature.
  • November 9 or 11 Demonstrate three checks running on the Jetson with a tablet dashboard

Alan Wang’s Status Report for 10/3

Accomplishments this week

This week I built a working local prototype of the treadmill availability check, from camera frames to a dashboard a tablet can open. It uses only pretrained models; nothing was trained on gym data. A pose model (YOLO11n-pose) finds people and their ankles. Each treadmill belt is stored as an outline, and a belt counts as occupied when someone’s feet land on it. Checks run twice per second. A belt is shown as in use after 3 of the last 4 checks agree, and as available only after 5 of the last 6 agree. Stale data, or a belt the camera cannot see, is shown as unknown, never as free.

I also automated the setup step so that belts no longer need to be outlined by hand. From a photo of the empty row, an open-vocabulary detector (Grounding DINO) finds each treadmill, and a segmentation model (SAM 2) outlines its belt. This found every belt in both of our test clips: 3 treadmills in one and 4 in the other. Two further checks make the system more robust. If the camera shifts, the outlines are re-aligned to the empty reference frame. If a belt is blocked from view, it is reported as unknown instead of free.

Treadmill pipeline: one-time automatic belt setup (top) and the per-check path from camera frame to tablet
(bottom).

I evaluated the pipeline in stages, from simple logic tests up to the full system. I first compared four zero-shot methods on stills from public stock footage: person box, pose, segmentation, and a CLIP image classifier. I then tested on two short clips we filmed from a camera angle similar to our planned mount. These clips include people stepping on and off, two people on adjacent treadmills, and people walking past.

– Across 66 seconds of footage, every step-on and step-off was caught.
– It was shown within 1.0 to 3.5 seconds.
– There were no false “free” readings and no state flips caused by people walking past.
– The CLIP classifier was rejected, because a person on a neighboring treadmill made empty belts look occupied.
– A replay through the dashboard matched the offline results exactly.
– In a synthetic test, a blocked belt changed to unknown within 2 seconds.

Member view during replay: 2 of 4 treadmills free.

These are small-sample laptop results, not a validation of accuracy at the gym. The clips were labeled by one person.

Clip 2: ground truth, per-check model output, and the state shown on the dashboard.

To prepare for the Jetson Orin Nano, I wrote a treadmill deployment guide on top of Jerry’s board runbook. I also wrote a one-command board check that records performance and compares results against our laptop reference, and a model export script for TensorRT. I rehearsed every step on my laptop up to the point that requires the board. That rehearsal found a bug where an unplugged camera stopped the server, which is now fixed. The code is organized into shared building blocks that the bench and squat workflows can reuse. It is in a pull request for team review.

Schedule status and corrective actions

The treadmill software is ahead of schedule for a local prototype. The main gaps are hardware and data. I have not yet run anything on the Jetson, and our test footage is short and comes from a single location and person. Recording permission for that footage still needs to be confirmed.

Next week I will run the board check on the Jetson as soon as I have access. If the per-check time is too high, I will first reduce the cost of camera re-alignment, which is currently the slowest step, before considering further model optimization. I will also record a longer, scripted set of clips with consenting teammates. These will include harder cases, such as someone standing beside or behind a treadmill and a crowded row. Some of these clips will be held back so that the final evaluation does not reuse tuning data.

Deliverables for next week

– Run the treadmill board check on the Jetson Orin Nano. Record per-check time, memory, temperature, and power mode, and compare the results with the laptop reference.
– Export the pose model to TensorRT on the board. Use it only if its results match PyTorch on our labeled clips.
– Run a live test from the C922 camera to a tablet, with scripted step-on and step-off events and a camera unplug test. Measure the time until each change appears on the tablet.
– Record and label additional treadmill clips, keep a held-out set for final testing, and have a teammate independently check a sample of the labels.
– Confirm recording permission for the clips we have filmed so far, and contribute the treadmill sections to the October 9 design report.

Alan Wang’s Status Report for 9/26

Accomplishments this week
This week, I prepared the design-review presentation and its supporting material. The package includes ten slides, architecture diagrams, design alternatives, a testing plan, a schedule, speaker notes, and scripts for generating the figures and slides. I also documented our decisions, preliminary evidence, and remaining questions so that this work can support the design report.
I included preliminary experiments investigating the vision approach. The detector check confirms that YOLO11n recognizes people and some common belongings, but its pretrained classes do not include weight plates. On generated gym stand-in images, some plates were assigned an unrelated class. This means the bench check needs its own validation and potentially training; a confident detection does not establish that the object is a weight plate.

Schedule status and corrective actions
The design-presentation work is in place, while camera-placement validation and real-footage testing remain open dependencies. These tasks need attention so that the design report can include evidence from representative scenes rather than only preliminary examples and simulations.
I will resolve the recording location and permission status with the team. If gym access is unavailable, we will use an appropriately authorized controlled setup with consenting participants. I will also help revise the presentation so that estimates, simulations, and measured results are clearly distinguished.

Deliverables for next week
•    Finalize the vision-related claims and figures for the design presentation due September 27.
•    Prepare a scene checklist covering occupied and empty treadmills, belongings left at a rack, an empty bar, plates on a bar, and plates correctly stored nearby.
•    Collect an initial approved set of staged examples and document the camera angle and lighting. If approval remains pending, finish the test protocol without recording unapproved gym footage.
•    Compare simple image processing with a detector-based approach and document failure cases.
•    Determine what additional labeled plate data is necessary before beginning a larger training effort.

Alan’s Status Report 9/19

Accomplished

This week I looked at how much of our detection we get for free from a pretrained model. I went through the classes it covers and wrote a short script to run it over gym images and count what it finds. People, backpacks, handbags, and bottles are all classes it already knows, which covers most of what gets left at a rack. Weight plates are not, and neither are towels, so the bench check will need training on our own labeled images rather than working out of the box. That is what we expected, but it is good to have it confirmed early. I also started looking into whether any public datasets include gym plates so we do not have to label everything from scratch.
I started on camera placement as well. The main issue is that people and equipment block the view, so I have been working through mounting height and angle for each check and what each camera has to be able to see. The bench camera is the tricky one, since it needs both the bar and the plate storage area in frame so we can tell plates that were put away from plates left on the bar.

Progress

Mostly on schedule. The detector work is done and the plate result was not a surprise, since time to collect and label images is already in our schedule. The camera placement plan is started but not finished, and I will finish it Sunday.

Deliverables

Finish the camera placement plan and take photos of the candidate positions. Record a first round of footage, at the gym if permission comes through and at a mock setup otherwise, and start labeling plate images. Help with the design presentation slides.