Week 12 · Capstone

3D Markerless Motion Capture (MoCap6)

Six synchronized cameras, no markers, and a full OpenSim skeleton. Capture a movement in 3D, judge whether the result can be trusted, and compare it with the 2D methods you have used all semester.

Pre-lab ≈ 30 minIn-lab 90 min (4 stations)Report ≈ 90 minTools: MoCap6, Pose2Sim, OpenSim, Kinovea, Excel

Overview

In Weeks 2–3 you tracked one point or one angle by hand in a single camera view. Research labs measure the whole body in three dimensions. Our lab's MoCap6 system records hardware-synchronized video from six cameras, calibrates them, detects body keypoints automatically, triangulates them into 3D, and drives an OpenSim musculoskeletal model to produce joint angles. This capstone ties together kinematics, angular kinematics and data quality.

Learning objectives

  1. Explain the markerless motion-capture pipeline: synchronization → calibration → 2D pose → triangulation → filtering → scaling and inverse kinematics.
  2. Interpret calibration and reconstruction quality metrics (reprojection error, IK marker error, camera agreement).
  3. Collect a 3D trial (squats and walking) following the MoCap6 recording workflow.
  4. Export and analyze 3D joint angles in Excel, including ROM and left–right asymmetry.
  5. Compare 3D markerless knee angles with a 2D Kinovea measurement of the same trial and evaluate both methods.
ComponentWhat you doTime
Pre-labHow 3D motion capture works, quality checks, what it cannot do; quiz25–30 min
Lab activityFour stations: capture on the rig, 2D comparison video, review/analyze a processed trial, project meeting90 min
Lab reportJoint-angle curves, ROM and asymmetry table, 2D vs. 3D comparison, quality audit, discussion≈ 90 min

Pre-Lab Reading ≈ 20 min

1. From 2D to 3D

A single camera flattens the world onto a plane: anything moving toward or away from the camera is lost, and any motion out of the calibrated plane is distorted (perspective error). With two or more calibrated cameras viewing the same point, its 3D location can be found by triangulation — intersecting the lines of sight from each camera. More cameras around the subject mean fewer moments when a joint is hidden (occluded) and a more robust 3D estimate.

2. Marker-based vs. markerless

Traditional systems track reflective markers taped to anatomical landmarks. Markerless systems use a deep-learning pose model to detect keypoints (joints such as hips, knees, ankles, heels and toes) directly in ordinary video. MoCap6 uses the open-source Pose2Sim pipeline with the HALPE_26 keypoint set (26 points including the feet), then hands the 3D keypoints to OpenSim. Markerless capture is faster to set up and needs no landmark palpation, but its accuracy depends on video quality, calibration and how well the pose model recognizes the person.

3. The MoCap6 pipeline

StepWhat happensWhy it matters
1. Set upSix Basler cameras are discovered and configured; frame rate and exposure are set (exposure must be shorter than the frame period)Settings the hardware cannot deliver cause dropped or unsynchronized frames
2. SynchronizeCameras synchronize their clocks over the network (IEEE 1588 PTP); shutters fire within tens of nanosecondsTriangulation assumes all six images show the same instant
3. CalibrateIntrinsics (each lens's focal length and distortion) and extrinsics (each camera's position and orientation) from a ChArUco board; floor reference sets the ground planeEvery 3D point inherits the calibration error
4. RecordA session holds one calibration and its trials; a flash test confirms all cameras see the same frameSync is verified from pixels, not just timestamps
5. Process2D pose detection in every view → person association → triangulation → Butterworth filter → OpenSim scaling and inverse kinematics (IK)Produces .trc/.c3d marker files and .mot joint-angle files
6. Review & AnalyzeOverlays of 2D keypoints and reprojected 3D points on each video; joint angles, ROM, repetitions; "Can I trust this result?" checksBad results look just as convincing as good ones — check them

4. Quality metrics you must be able to read

CheckGoodWhat it means
Calibration (mean reprojection error)< 1 px (< 2 px usable)How far a known 3D point lands from where each camera saw it
Level floorPelvis 0.4–1.3 m above floor, uprightWithout a floor reference the model lies on its side
Camera agreementEvery camera used on most framesRejected cameras mean 3D built from one side of the ring
Model fit (IK marker RMS)< 2 cm, worst < 4 cmHow well the scaled skeleton matches the 3D keypoints
Joint limitsNo joint pinned at a model limitA pinned angle shows the model's stop, not the real movement

5. Filtering, again

Raw 3D keypoints jitter from frame to frame. MoCap6 defaults to a 6 Hz low-pass Butterworth filter (suitable for walking and squats); jumping and running need a higher cut-off (≈ 10–15 Hz) or real peaks will be smoothed away. A cut-off above half the frame rate does nothing. This is the same smoothing trade-off you met in Weeks 2, 3 and 10.

6. Joint angles from inverse kinematics

OpenSim scales a generic model to the participant's size (using a standing pose, height and mass), then inverse kinematics finds, frame by frame, the joint angles that make the model's virtual markers best match the 3D keypoints. The output (.mot file) contains hip flexion, hip adduction, hip rotation, knee flexion, ankle dorsiflexion and more, for both sides — angles a single 2D camera cannot measure.

What this system does not doMoCap6 gives kinematics. Joint moments (inverse dynamics) require measured ground reaction forces from force plates. The optional muscle overlay shows muscle length changes from the movement, not muscle activation or force — for activation you need EMG (Week 11).

Before you come to lab

  • Wear fitted, contrasting clothing (avoid baggy clothes that hide joints); no reflective clothing.
  • Bring your Week 3 Excel file (2D knee angle in walking) for comparison.
  • Bring your phone for the 2D comparison video.
  • If you will be a participant: know your height (m) and mass (kg). You will be identified by a code (e.g., S03), not your name.

Pre-Lab Quiz ≈ 10 min

Triangulation in 3D motion capture means:
Each camera defines a ray toward the point; where calibrated rays meet is the 3D position.
Why must the six cameras be synchronized?
A foot moving at 5 m/s travels 8 cm in one 60 fps frame; unsynchronized views would triangulate nonsense.
Extrinsic calibration determines:
Intrinsics describe the lens; extrinsics describe where each camera is and which way it points.
A calibration reports a mean reprojection error of 3.5 px. According to the MoCap6 quality checks, the result is:
< 1 px is good, < 2 px usable; above that, every 3D point inherits too much error.
You are processing a countermovement jump. What should you do with the low-pass filter cut-off compared with the 6 Hz default?
Faster movements contain higher-frequency real content. A cut-off above half the frame rate has no effect.
Which can MoCap6 NOT calculate on its own?
Joint moments need inverse dynamics, which needs measured ground reaction forces from force plates.
Compared with a single 2D camera, the main advantage of the six-camera system for the knee is that it:
3D reconstruction captures frontal and transverse motion and multiple views reduce hidden-joint problems.

Short responses (submit to your LMS)

1. In your Week 3 lab, list two specific sources of error in your 2D knee angle that a calibrated 3D system should reduce, and one new source of error that a markerless 3D system introduces.

2. A result looks smooth and realistic, but the IK marker RMS is 6 cm and two cameras were rejected on most frames. Would you use it in a report? Explain.

Lab Activity 90 min

📍 Set up for your campusWritten for a six-camera markerless system. With any other multi-camera system, substitute its quality checks; with no 3D system, use the provided 3D data sets for the 3D side of the comparison. See the instructor guide.
Instructor preparation (before class)The rig is set up, PTP-synchronized and calibrated before students arrive (lens intrinsics once; camera positions whenever a camera moves; floor reference recorded). A demonstration trial is processed in advance for Station C. See the Instructor Guide.

Station rotation (one six-camera rig)

TimeTeam 1Team 2Team 3Team 4
0:00–0:12Rig tour: cameras, sync status, calibration board, capture volume; live view on the projector. The instructor runs the session's flash test during the tour.
0:12–0:31ABCD
0:31–0:50BCDA
0:50–1:09CDAB
1:09–1:28DABC
1:28–1:30Wrap-up: processing queue. Processed exports are posted in the course LMS within 48 hours.

Stations A and B run together at the rig: while one team captures on MoCap6 (A), the next team films the same trials with a phone (B) for the 2D comparison.

Station A — Capture on MoCap6

One participant per team (≈ 15 minutes of capture). Roles: Participant, Operator (MoCap6 Record page), Clapper (clap at the start of each trial for 2D alignment), Trial logger, Quality checker.

  1. Confirm status on the Record page: synchronized (one PTP master, all others slaved) and the session's calibration loaded.
  2. Participant entry: de-identified ID (e.g., S03), height and mass.
  3. Framing: participant stands in the center of the capture volume; check in the live view that every camera sees the whole body.
  4. Trial 1 — Static: stand still in anatomical position, 3 s (used for model scaling).
  5. Trial 2 — Squats: 5 bodyweight squats to a comfortable depth, 2 s down / 2 s up.
  6. Trial 3 — Walking: walk at preferred speed through the capture volume (at least two full strides inside it).
  7. After each trial, read the sync verdict (frame counts, timestamp cadence, transport losses). Redo the trial if it fails.

Station B — 2D comparison video

  1. Place a phone on a tripod perpendicular to the walking path at knee–hip height (same set-up as Week 3), outside the capture volume so it does not block MoCap6 cameras.
  2. Film Station A's squat and walking trials at ≥ 60 fps. Start each video with the flash or a clap visible to synchronize with MoCap6 later.
  3. Name files Team#_S##_Squat_Phone.mp4, _Walk_Phone.mp4.

Station C — Review & Analyze a processed trial

Work on the review computer with the instructor's pre-processed demonstration trial.

  1. Review page: replay the trial. Identify the 2D keypoints (dots and bones), the reprojected 3D points (rings), and the reprojection gaps (red lines). Which camera has the largest median gap? At which moment?
  2. OpenSim model tab: watch the scaled model; identify the experimental (blue) and model (pink) markers and their IK error lines.
  3. Analyze page: read every line of Can I trust this result? and record the verdict and each check's value.
  4. Use Repetitions to split the squats by knee flexion; note mean ± SD ROM for hip, knee and ankle.
  5. Export CSV to the trial's analysis folder; copy it to the team folder.

Station D — Group Design Project meeting

Hold and document Business Meeting #2 (see Group Design Project). Consider whether 3D motion capture could validate or inspire your design.

Care of equipment and dataDo not touch or bump cameras — moving one invalidates the calibration. Do not rename or move folders in the MoCap6 data root; Pose2Sim needs the exact layout. Participant data are identified by code only.

Lab Report ≈ 90 min

Backup data setAbsent, or your trial failed? Use this simulated data set. It has the same columns as the lab data sheet, so every formula below works unchanged. Tell your instructor in your report that you used it. Participant S03, 1.78 m, 76 kg. To align 2D and 3D, use the sync event: MoCap6 at 0.40 s, phone at 1.15 s.
⬇ 3D joint angles, 5 squats · 3D joint angles, walking · 2D Kinovea knee angle, same walk · Quality-audit values

Your instructor posts processed exports for your team's trials within 48 hours. If your trial failed processing or you were absent, use the backup data set below. The MoCap6 Analyze page CSV contains time and joint angles for both sides.

Step 1 — Quality audit ≈ 5 min

Table 1. For your trial: calibration error (px), floor check, camera agreement, IK marker RMS and worst marker (cm), joints at limit, overall verdict. In two sentences, state whether the result is suitable for your analysis.

Step 2 — 3D joint angles ≈ 25 min

  • Import the CSV. Keep time and the sagittal angles (hip flexion, knee flexion/angle, ankle dorsiflexion) for right and left, plus hip adduction for both.
  • Figure 1. Squat: right and left knee flexion vs. time on one chart; mark each repetition.
  • Figure 2. Squat: hip, knee and ankle angles for one repetition normalized to 0–100%. Use the MoCap6 repetition export, or type the repetition's start and end times in cells K1 and K2 and add a % column: =(A2-$K$1)/($K$2-$K$1)*100 (column A = time).
  • Table 2. ROM for each joint and side: =MAX(D2:D1321)-MIN(D2:D1321) (adjust the column and last row). With right ROM in B and left ROM in C, asymmetry (%) is =(B2-C2)/((B2+C2)/2)*100.

Step 3 — 2D vs. 3D comparison ≈ 30 min

  • Track the knee angle in Kinovea for one stride only (about 66 frames at 60 fps, ≈ 15 min) from the phone video of the same walking trial (Week 3 method). Convert to flexion (180° − included angle).
  • Align the two signals in time using the flash/clap frame, and express both as % gait cycle.
  • Figure 3. Overlay 2D (Kinovea) and 3D (MoCap6) right knee flexion vs. % gait cycle for one stride.
  • Table 3. Peak swing flexion, loading-response peak and ROM for both methods. For agreement, resample both curves at the same % gait-cycle points (0, 2, 4 … 100%). Put 3D flexion in column B and 2D flexion in column C, add D =ABS(B2-C2), and report the mean absolute difference =AVERAGE(D2:D52).

Step 4 — Discussion questions ≈ 30 min

Answer the five required questions. Optional questions earn up to +3 extra-credit points.

  1. Summarize your quality audit. Which check came closest to failing, and what would you do differently in the capture or processing to improve it?
  2. Describe the hip–knee–ankle coordination in the squat (Figure 2). Which joint reached its peak first, and which had the largest ROM?
  3. Report the right–left asymmetry for knee and hip ROM. Is the asymmetry likely real or measurement error? How would you find out?
  4. How closely did the 2D and 3D knee curves agree (Figure 3, Table 3)? Explain where and why they differ (out-of-plane motion, perspective, landmark vs. keypoint definitions, filtering).
  5. Looking back at the semester, rank the measurement tools you used (stopwatch, tape measure, phone video/Kinovea, phone accelerometer, BIOPAC EMG, MoCap6) by validity and practicality for a field setting, and justify your ranking.

Optional questions (extra credit)

  1. Hip adduction is a frontal-plane angle. Why could you not measure it in Week 3, and why might it matter clinically (e.g., knee valgus, Trendelenburg gait)?
  2. MoCap6 does not compute joint moments. What additional equipment and data would be needed, and how would you synchronize them with the cameras?

Submission checklist

  • Excel workbook: imported angles, % normalization, Tables 1–3, Figures 1–3.
  • Document: tables, figures with captions, discussion answers.
  • Kinovea screenshot of your 2D knee angle tracking.

Grading rubric (50 pts)

CriterionPtsFull credit looks like
Quality audit6All checks reported and interpreted
3D analysis & figures16Correct normalization, ROM and asymmetry, clear figures
2D vs. 3D comparison10Time-aligned overlay with quantitative agreement
Discussion16Critical evaluation of methods, integrates semester concepts
Presentation2Professional and complete

Software: MoCap6 (lab-built; Basler acA720-290gc cameras), Pose2Sim, OpenSim. Methods details for each processed trial are saved in meta/processing.json.

References

  • Pagnon, D., Domalain, M., & Reveret, L. (2022). Pose2Sim: An open-source Python package for multiview markerless kinematics. Journal of Open Source Software, 7(77), 4362. doi.org/10.21105/joss.04362
  • Delp, S. L., Anderson, F. C., Arnold, A. S., Loan, P., Habib, A., John, C. T., Guendelman, E., & Thelen, D. G. (2007). OpenSim: Open-source software to create and analyze dynamic simulations of movement. IEEE Transactions on Biomedical Engineering, 54(11), 1940–1950. doi.org/10.1109/TBME.2007.901024
  • Puig-Diví, A., Escalona-Marfil, C., Padullés-Riu, J. M., Busquets, A., Padullés-Chando, X., & Marcos-Ruiz, D. (2019). Validity and reliability of the Kinovea program in obtaining angles and distances using coordinates in 4 perspectives. PLOS ONE, 14(6), e0216448. doi.org/10.1371/journal.pone.0216448