Medicine Ball Throw Strength (On-Device)

This project estimates medicine ball (or any known-weight object) throw performance from a single video and produces an upper-body strength score. It runs fully on-device with lightweight TensorFlow Lite models and simple physics.

What you get

• Throw metrics: distance (m), release angle (deg), release speed (m/s)
• Strength score: 0–10 with category (Excellent/Good/Average/Needs Improvement)
• Mode: ball_trajectory (uses ball tracking + calibration) or motion_only (pose-motion fallback when tracking/calibration is unavailable)
• JSON output ready for mobile/Flutter consumption

Folder layout

• data/ raw videos, frames, calibration assets
• models/ TFLite pose, ball detector, strength regressor, checkpoints
• src/ preprocessing, pose, ball tracking, physics, scoring, mobile pipeline, API
• scripts/ helper downloads (e.g., MoveNet)
• tests/ unit tests

Setup

1. Install deps (in a venv recommended):

python3 -m pip install -r requirements.txt

2. Download a pose model (required for motion analysis):

python3 scripts/download_models.py --name movenet_lightning --output models/pose/movenet.tflite

3. (Optional) Add ball detector and strength regressor TFLite models to models/ball_detector/ and models/strength_regressor/.

Recording do's and don'ts

• Do use a fixed camera at shoulder height, side view.
• Do keep the athlete and throw line fully in frame; avoid cutting off the release/landing area.
• Do place two visible ground markers 10 m apart (for calibration) if you want real distances.
• Do ensure good lighting and a high-contrast ball/object against the background.
• Don’t pan or zoom during the throw.
• Don’t let other people/objects occlude the athlete or the ball.
• Don’t use reflective surfaces or heavy motion blur—both reduce pose/ball detection quality.

Run from CLI

python -m src.mobile.video_pipeline_mobile \
  --video data/raw_videos/sample.mp4 \
  --pose-model models/pose/movenet.tflite \
  --ball-model models/ball_detector/yolo_nano_mb.tflite \
  --strength-model models/strength_regressor/strength_mlp.tflite \
  --athlete-age 24 --athlete-sex M --athlete-mass 78 --athlete-height 1.78 \
  --ball-weight 3.0 --object-weight 3.0 \
  --marker-0m 120 --marker-10m 420

Outputs JSON with throw metrics, strength score, and analysis_mode.

REST API / Swagger

Start the server:

python -m uvicorn src.api.app:app --reload --host 0.0.0.0 --port 8000

Open http://127.0.0.1:8000/docs and use:

• POST /analyze/upload: upload video (and optionally models), set athlete info, object_weight (kg), and optional markers.
• POST /analyze: JSON with file paths if you prefer.

Calibration (meters)

If you want real distances, provide pixel x-positions for two ground markers 10 m apart:

• marker_0m: pixel x at 0 m reference
• marker_10m: pixel x at 10 m Without markers, the system falls back to motion-only scoring (no meters).

How the math works

• Calibration: pixels_per_meter = (x_10m - x_0m) / 10
• Projectile (first post-release frames):
  v_x = (x2 - x1)/Δt, v_y = (y2 - y1)/Δt, speed = sqrt(v_x^2 + v_y^2), angle = atan2(v_y, v_x)
  Landing distance: (x_landing - x_0m)/pixels_per_meter
• Baseline strength (ball_trajectory):
  PowerIndex = (distance_m * object_weight_kg) / athlete_mass_kg
  Normalized Z then score = clamp(5 + 1.5 * Z, 0, 10) (optionally blended with MLP regressor).
• Motion-only fallback (when ball tracking/calibration not available):
  Uses centroid keypoint speed from pose.
  relative_speed = max_speed_pxps / 500
weight_factor = (object_weight / athlete_mass) * 75
intensity = relative_speed * weight_factor
score = clamp(intensity * 2, 0, 10)
  Higher arm speed + heavier object → higher score.

Models

• Pose: MoveNet Lightning/Thunder TFLite (download script above).
• Ball detector: YOLO-Nano TFLite or rely on HSV fallback (works best with a high-contrast ball).
• Strength regressor: optional TFLite MLP (models/strength_regressor/strength_mlp.tflite).

Flutter integration guide

Goal: run inference fully on-device in Flutter using TFLite.

1. Add models as Flutter assets

   • Place TFLite models under assets/models/ in your Flutter project.
   • Update pubspec.yaml:

   assets:
     - assets/models/movenet.tflite
     - assets/models/yolo_nano_mb.tflite
     - assets/models/strength_mlp.tflite
   

2. Add dependencies (Flutter)

   • Use tflite_flutter (and tflite_flutter_helper if you want helpers).
   • Use camera or video_player to decode frames; resize to 256x256 for detector, 192x192 for pose.

3. Implement the pipeline in Dart (outline)

   • Load interpreters with Interpreter.fromAsset('assets/models/movenet.tflite') etc.
   • For each frame (15–30 FPS):
     • Pose: preprocess (resize/normalize), run inference, keep keypoints, smooth if desired.
     • Ball: run detector (or HSV fallback) to get ball centers across frames; track continuity.
   • Identify release and landing frames; apply calibration if markers are known.
   • Compute physics: vx/vy, release_speed, release_angle, distance_m.
   • Build features and map to strength score (or use motion-only formula above if no ball/calibration).
   • Return the same JSON shape as the Python pipeline.

4. Optional: call the Python service instead

   • Run the FastAPI server locally or on-device, and call /analyze/upload from Flutter with multipart video if you prefer not to reimplement the pipeline in Dart.

Reference JSON schema

{
  "success": true,
  "athlete_id": "athlete-001",
  "ball_weight": 3.0,
  "throw_metrics": {
    "distance_m": 5.6,
    "release_angle_deg": 41.4,
    "release_speed_mps": 7.9,
    "motion_path_px": 123.4,
    "max_wrist_speed_pxps": 2100.0
  },
  "strength": {
    "score": 8.1,
    "category": "Good",
    "power_index": 0.95,
    "details": {...}
  },
  "metadata": {
    "video_fps": 24,
    "model_version": "v1.0.0",
    "device": "mobile_cpu",
    "analysis_mode": "ball_trajectory" // or "motion_only"
  }
}

Testing

python -m pytest -q