Skip to content

Latest commit

 

History

11 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Rep counting from pose landmarks

Companion code for How to build a rep counter that does not count your rest on CondadosAI.

Four stages sit between a pose model and a repetition count: landmarks, a joint angle, a filter, a state machine. This repository is the measurement behind each choice, and the code that runs them.

What is in here

src/poserep/
  filters.py     EMA, 1 euro, constant-velocity Kalman, one call signature
  angles.py      joint angles from image landmarks and from world landmarks
  fsm.py         the repetition counter, its guards, and the naive baseline
  landmarks.py   MediaPipe Pose Landmarker in VIDEO mode, video in, arrays out
  signals.py     33 points per frame down to the one number the counter reads
  metrics.py     jitter, lag, count error, and the closed forms they check against
  cli.py         poserep extract / signals / count / jitter / filter-lag
scripts/         one file per experiment, each writing its artifact to output/
web/             a standalone browser lab, no build step, with the knobs exposed
notebooks/       one cell per Python snippet printed in the article
data/clips/      SOURCES.md and labels.csv; the video itself is not committed
docs/            how the conditions found video cannot supply would be recorded

Everything except extract reads a saved .npz instead of the video, so two configurations differ by the configuration and not by what the tracker happened to do on a second pass.

Quickstart

uv sync --extra pose
uv run poserep fetch-model --variant full
uv run python scripts/fetch_clips.py          # from Wikimedia Commons, with licences

uv run poserep extract data/clips/interval_pushups.webm --out output/interval_pushups.npz
# One threshold at 158 degrees, inside the band the arm rests at between reps.
# --guards naive collapses the pair to its midpoint, so 157/159 puts it on 158.
uv run poserep count output/interval_pushups.npz --guards naive --flex-below 157 --extend-above 159 --truth 6
#   -> 20 reps (truth 6, error +14)

# The same clip, the same frames, two thresholds with 70 degrees between them.
uv run poserep count output/interval_pushups.npz --guards hyst --flex-below 80 --extend-above 150 --truth 6
#   -> 6 reps (truth 6, error +0)

The experiments behind the article's tables, each writing to output/:

uv run python scripts/tracker_self_jitter.py                                     # the frozen-frame table
uv run python scripts/noise_profile.py                                           # the autocorrelation
uv run python scripts/filter_sweep.py && uv run python scripts/plot_sweep.py     # the jitter-versus-lag curve
uv run python scripts/threshold_placement.py --track output/interval_pushups.npz --truth 6
uv run python scripts/angle_2d_vs_3d.py                                          # the projection experiment
uv run python scripts/make_figure_svg.py                                         # the figure as inline SVG

Two of the scripts read the article and the site's browser module, and default to finding them in a CondadosAI checkout beside this one. Set CONDADOSAI_SITE if yours lives somewhere else; the tests that need it skip rather than fail when it is absent.

The browser side

web/lab.html is a standalone lab: open the file, no build step, no server. It exposes the knobs the article settled, so the trade-offs can be felt rather than read. The article's own widget is the opposite, a fixed configuration on the reader's camera, and it lives in the site repository.

uv run python scripts/lab_check.py --clip data/clips/interval_pushups.webm --seconds 15

That drives the page in headless Chrome over the DevTools protocol and reads the counter back. A page that throws on load, fails to fetch the model, or silently counts nothing looks fine in a screenshot and does not look fine there.

What is checked, and how

The same two algorithms exist three times: the Python here, the TypeScript the article's widget runs, and the standalone modules in web/. Every number the article publishes describes one arithmetic, so a divergence between them makes the article wrong instead of the code, and tests/test_js_ports.py fails when it happens. It runs all three against the same inputs:

check result
1 euro against the authors' published groundTruth.csv Python, TypeScript and standalone JS all agree to 4.0 × 10⁻⁶
the same error in all three, not merely a small one identical to a relative 10⁻⁶
hardened counters on one signal the same repetitions at the same timestamps
naive counters on one signal likewise, because the reproduced bug has to be reproduced identically
the naive counter still overcounts a held position a guard on the premise of the article

The residual against the authors' ground truth is the CSV's own six-significant-figure printing accumulating through a recursive filter, not a difference in the algorithm: driving the same rows with exact 1/120 timestamps instead of the rounded ones in the file drops the worst disagreement six times further.

Lag is a second place a measurement can be quietly wrong. The EMA has two closed forms and they are not interchangeable: the DC group delay is (1-a)/a samples and the time for a step to reach 90 percent is ln(0.1)/ln(1-a). At a = 0.2 and 30 fps those are 133 ms and 344 ms. The harness is checked against the second, which is the one a user feels, and poserep filter-lag prints both side by side.

Third-party material

Asset Source Licence
Pose Landmarker bundles (lite / full / heavy) Google MediaPipe Apache-2.0, downloaded at run time, not redistributed
tests/oneeuro_groundtruth.csv casiez/OneEuroFilter BSD-3-Clause, (c) Inria
The clips Wikimedia Commons per file, see data/clips/SOURCES.md, generated from the Commons API

The Apache-2.0 licence at the root covers the code in src/, scripts/, tests/ and web/, and nothing else. No clip is redistributed here.

Citation

The filter is not ours:

@inproceedings{casiez2012oneeuro,
  author    = {Casiez, G\'{e}ry and Roussel, Nicolas and Vogel, Daniel},
  title     = {1 € Filter: A Simple Speed-Based Low-Pass Filter for Noisy Input in Interactive Systems},
  booktitle = {Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12)},
  year      = {2012},
  pages     = {2527--2530},
  doi       = {10.1145/2207676.2208639}
}

About

Where repetition counting breaks in a pose pipeline: jitter with memory, the filter that earns its place, and the state machine that does the work. Companion code for condados.ai.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages