JCF — Joint Contact Forces from Monocular Video

Predicts 3D hip and knee joint contact forces from a monocular video, via SMPL pose features (and optional frozen V-JEPA 2 video features). Accompanies "From Pixels to Newtons: Predicting In Vivo Joint Contact Forces from Monocular Video" (arXiv) · code: jeylau/jcf.

Files

• model.pt — force-predictor weights + architecture config
• feat_stats.npz — feature normalisation statistics (required)
• text_vocab.json, text_embeddings.npy — optional, for activity-label conditioning

Usage

pip install git+https://github.com/jeylau/jcf.git

from jcf import ForceModel

model = ForceModel.from_pretrained("jeylau/jcf")
result = model.predict("trial.npz", joint="knee", side="right")
result.forces   # (T, 3) in bodyweight (BW) units;  also .time, .sigma

trial.npz is a preprocessed SMPL sequence; see the repo for the video → features pipeline.

Intended use & limitations

Research use only. Trained on the OrthoLoad instrumented-implant cohort (26 patients, 25 activities) and evaluated zero-shot on an independent cohort (the Grand Challenge knee load competition data) where it matches or outperforms prior published methods. This is the all-subjects checkpoint (the one used for the paper's inverse design experiments); the paper's reported accuracy is leave-one-subject-out, so in-sample subjects look better than those held-out figures. Accuracy outside these tested conditions (other activities or populations) is not guaranteed.

Citation

@article{lauer2026pixels,
  title   = {From Pixels to Newtons: Predicting In Vivo Joint Contact Forces from Monocular Video},
  author  = {Jessy Lauer},
  journal = {arXiv preprint arXiv:2606.06631},
  year    = {2026}
}