3D Markerless Motion Capture (MoCap6)
Six synchronized cameras, no markers, and a full OpenSim skeleton. Capture a movement in 3D, judge whether the result can be trusted, and compare it with the 2D methods you have used all semester.
Overview
In Weeks 2–3 you tracked one point or one angle by hand in a single camera view. Research labs measure the whole body in three dimensions. Our lab's MoCap6 system records hardware-synchronized video from six cameras, calibrates them, detects body keypoints automatically, triangulates them into 3D, and drives an OpenSim musculoskeletal model to produce joint angles. This capstone ties together kinematics, angular kinematics and data quality.
Learning objectives
- Explain the markerless motion-capture pipeline: synchronization → calibration → 2D pose → triangulation → filtering → scaling and inverse kinematics.
- Interpret calibration and reconstruction quality metrics (reprojection error, IK marker error, camera agreement).
- Collect a 3D trial (squats and walking) following the MoCap6 recording workflow.
- Export and analyze 3D joint angles in Excel, including ROM and left–right asymmetry.
- Compare 3D markerless knee angles with a 2D Kinovea measurement of the same trial and evaluate both methods.
| Component | What you do | Time |
|---|---|---|
| Pre-lab | How 3D motion capture works, quality checks, what it cannot do; quiz | 25–30 min |
| Lab activity | Four stations: capture on the rig, 2D comparison video, review/analyze a processed trial, project meeting | 90 min |
| Lab report | Joint-angle curves, ROM and asymmetry table, 2D vs. 3D comparison, quality audit, discussion | ≈ 90 min |
Pre-Lab Reading ≈ 20 min
1. From 2D to 3D
A single camera flattens the world onto a plane: anything moving toward or away from the camera is lost, and any motion out of the calibrated plane is distorted (perspective error). With two or more calibrated cameras viewing the same point, its 3D location can be found by triangulation — intersecting the lines of sight from each camera. More cameras around the subject mean fewer moments when a joint is hidden (occluded) and a more robust 3D estimate.
2. Marker-based vs. markerless
Traditional systems track reflective markers taped to anatomical landmarks. Markerless systems use a deep-learning pose model to detect keypoints (joints such as hips, knees, ankles, heels and toes) directly in ordinary video. MoCap6 uses the open-source Pose2Sim pipeline with the HALPE_26 keypoint set (26 points including the feet), then hands the 3D keypoints to OpenSim. Markerless capture is faster to set up and needs no landmark palpation, but its accuracy depends on video quality, calibration and how well the pose model recognizes the person.
3. The MoCap6 pipeline
| Step | What happens | Why it matters |
|---|---|---|
| 1. Set up | Six Basler cameras are discovered and configured; frame rate and exposure are set (exposure must be shorter than the frame period) | Settings the hardware cannot deliver cause dropped or unsynchronized frames |
| 2. Synchronize | Cameras synchronize their clocks over the network (IEEE 1588 PTP); shutters fire within tens of nanoseconds | Triangulation assumes all six images show the same instant |
| 3. Calibrate | Intrinsics (each lens's focal length and distortion) and extrinsics (each camera's position and orientation) from a ChArUco board; floor reference sets the ground plane | Every 3D point inherits the calibration error |
| 4. Record | A session holds one calibration and its trials; a flash test confirms all cameras see the same frame | Sync is verified from pixels, not just timestamps |
| 5. Process | 2D pose detection in every view → person association → triangulation → Butterworth filter → OpenSim scaling and inverse kinematics (IK) | Produces .trc/.c3d marker files and .mot joint-angle files |
| 6. Review & Analyze | Overlays of 2D keypoints and reprojected 3D points on each video; joint angles, ROM, repetitions; "Can I trust this result?" checks | Bad results look just as convincing as good ones — check them |
4. Quality metrics you must be able to read
| Check | Good | What it means |
|---|---|---|
| Calibration (mean reprojection error) | < 1 px (< 2 px usable) | How far a known 3D point lands from where each camera saw it |
| Level floor | Pelvis 0.4–1.3 m above floor, upright | Without a floor reference the model lies on its side |
| Camera agreement | Every camera used on most frames | Rejected cameras mean 3D built from one side of the ring |
| Model fit (IK marker RMS) | < 2 cm, worst < 4 cm | How well the scaled skeleton matches the 3D keypoints |
| Joint limits | No joint pinned at a model limit | A pinned angle shows the model's stop, not the real movement |
5. Filtering, again
Raw 3D keypoints jitter from frame to frame. MoCap6 defaults to a 6 Hz low-pass Butterworth filter (suitable for walking and squats); jumping and running need a higher cut-off (≈ 10–15 Hz) or real peaks will be smoothed away. A cut-off above half the frame rate does nothing. This is the same smoothing trade-off you met in Weeks 2, 3 and 10.
6. Joint angles from inverse kinematics
OpenSim scales a generic model to the participant's size (using a standing pose, height and mass), then inverse kinematics finds, frame by frame, the joint angles that make the model's virtual markers best match the 3D keypoints. The output (.mot file) contains hip flexion, hip adduction, hip rotation, knee flexion, ankle dorsiflexion and more, for both sides — angles a single 2D camera cannot measure.
Before you come to lab
- Wear fitted, contrasting clothing (avoid baggy clothes that hide joints); no reflective clothing.
- Bring your Week 3 Excel file (2D knee angle in walking) for comparison.
- Bring your phone for the 2D comparison video.
- If you will be a participant: know your height (m) and mass (kg). You will be identified by a code (e.g., S03), not your name.
Pre-Lab Quiz ≈ 10 min
Short responses (submit to your LMS)
1. In your Week 3 lab, list two specific sources of error in your 2D knee angle that a calibrated 3D system should reduce, and one new source of error that a markerless 3D system introduces.
2. A result looks smooth and realistic, but the IK marker RMS is 6 cm and two cameras were rejected on most frames. Would you use it in a report? Explain.
Lab Activity 90 min
Station rotation (one six-camera rig)
| Time | Team 1 | Team 2 | Team 3 | Team 4 |
|---|---|---|---|---|
| 0:00–0:12 | Rig tour: cameras, sync status, calibration board, capture volume; live view on the projector. The instructor runs the session's flash test during the tour. | |||
| 0:12–0:31 | A | B | C | D |
| 0:31–0:50 | B | C | D | A |
| 0:50–1:09 | C | D | A | B |
| 1:09–1:28 | D | A | B | C |
| 1:28–1:30 | Wrap-up: processing queue. Processed exports are posted in the course LMS within 48 hours. | |||
Stations A and B run together at the rig: while one team captures on MoCap6 (A), the next team films the same trials with a phone (B) for the 2D comparison.
Station A — Capture on MoCap6
One participant per team (≈ 15 minutes of capture). Roles: Participant, Operator (MoCap6 Record page), Clapper (clap at the start of each trial for 2D alignment), Trial logger, Quality checker.
- Confirm status on the Record page: synchronized (one PTP master, all others slaved) and the session's calibration loaded.
- Participant entry: de-identified ID (e.g., S03), height and mass.
- Framing: participant stands in the center of the capture volume; check in the live view that every camera sees the whole body.
- Trial 1 — Static: stand still in anatomical position, 3 s (used for model scaling).
- Trial 2 — Squats: 5 bodyweight squats to a comfortable depth, 2 s down / 2 s up.
- Trial 3 — Walking: walk at preferred speed through the capture volume (at least two full strides inside it).
- After each trial, read the sync verdict (frame counts, timestamp cadence, transport losses). Redo the trial if it fails.
Station B — 2D comparison video
- Place a phone on a tripod perpendicular to the walking path at knee–hip height (same set-up as Week 3), outside the capture volume so it does not block MoCap6 cameras.
- Film Station A's squat and walking trials at ≥ 60 fps. Start each video with the flash or a clap visible to synchronize with MoCap6 later.
- Name files
Team#_S##_Squat_Phone.mp4,_Walk_Phone.mp4.
Station C — Review & Analyze a processed trial
Work on the review computer with the instructor's pre-processed demonstration trial.
- Review page: replay the trial. Identify the 2D keypoints (dots and bones), the reprojected 3D points (rings), and the reprojection gaps (red lines). Which camera has the largest median gap? At which moment?
- OpenSim model tab: watch the scaled model; identify the experimental (blue) and model (pink) markers and their IK error lines.
- Analyze page: read every line of Can I trust this result? and record the verdict and each check's value.
- Use Repetitions to split the squats by knee flexion; note mean ± SD ROM for hip, knee and ankle.
- Export CSV to the trial's
analysisfolder; copy it to the team folder.
Station D — Group Design Project meeting
Hold and document Business Meeting #2 (see Group Design Project). Consider whether 3D motion capture could validate or inspire your design.
Lab Report ≈ 90 min
⬇ 3D joint angles, 5 squats · 3D joint angles, walking · 2D Kinovea knee angle, same walk · Quality-audit values
Your instructor posts processed exports for your team's trials within 48 hours. If your trial failed processing or you were absent, use the backup data set below. The MoCap6 Analyze page CSV contains time and joint angles for both sides.
Step 1 — Quality audit ≈ 5 min
Table 1. For your trial: calibration error (px), floor check, camera agreement, IK marker RMS and worst marker (cm), joints at limit, overall verdict. In two sentences, state whether the result is suitable for your analysis.
Step 2 — 3D joint angles ≈ 25 min
- Import the CSV. Keep time and the sagittal angles (hip flexion, knee flexion/angle, ankle dorsiflexion) for right and left, plus hip adduction for both.
- Figure 1. Squat: right and left knee flexion vs. time on one chart; mark each repetition.
- Figure 2. Squat: hip, knee and ankle angles for one repetition normalized to 0–100%. Use the MoCap6 repetition export, or type the repetition's start and end times in cells K1 and K2 and add a % column:
=(A2-$K$1)/($K$2-$K$1)*100(column A = time). - Table 2. ROM for each joint and side:
=MAX(D2:D1321)-MIN(D2:D1321)(adjust the column and last row). With right ROM in B and left ROM in C, asymmetry (%) is=(B2-C2)/((B2+C2)/2)*100.
Step 3 — 2D vs. 3D comparison ≈ 30 min
- Track the knee angle in Kinovea for one stride only (about 66 frames at 60 fps, ≈ 15 min) from the phone video of the same walking trial (Week 3 method). Convert to flexion (180° − included angle).
- Align the two signals in time using the flash/clap frame, and express both as % gait cycle.
- Figure 3. Overlay 2D (Kinovea) and 3D (MoCap6) right knee flexion vs. % gait cycle for one stride.
- Table 3. Peak swing flexion, loading-response peak and ROM for both methods. For agreement, resample both curves at the same % gait-cycle points (0, 2, 4 … 100%). Put 3D flexion in column B and 2D flexion in column C, add D
=ABS(B2-C2), and report the mean absolute difference=AVERAGE(D2:D52).
Step 4 — Discussion questions ≈ 30 min
Answer the five required questions. Optional questions earn up to +3 extra-credit points.
- Summarize your quality audit. Which check came closest to failing, and what would you do differently in the capture or processing to improve it?
- Describe the hip–knee–ankle coordination in the squat (Figure 2). Which joint reached its peak first, and which had the largest ROM?
- Report the right–left asymmetry for knee and hip ROM. Is the asymmetry likely real or measurement error? How would you find out?
- How closely did the 2D and 3D knee curves agree (Figure 3, Table 3)? Explain where and why they differ (out-of-plane motion, perspective, landmark vs. keypoint definitions, filtering).
- Looking back at the semester, rank the measurement tools you used (stopwatch, tape measure, phone video/Kinovea, phone accelerometer, BIOPAC EMG, MoCap6) by validity and practicality for a field setting, and justify your ranking.
Optional questions (extra credit)
- Hip adduction is a frontal-plane angle. Why could you not measure it in Week 3, and why might it matter clinically (e.g., knee valgus, Trendelenburg gait)?
- MoCap6 does not compute joint moments. What additional equipment and data would be needed, and how would you synchronize them with the cameras?
Submission checklist
- Excel workbook: imported angles, % normalization, Tables 1–3, Figures 1–3.
- Document: tables, figures with captions, discussion answers.
- Kinovea screenshot of your 2D knee angle tracking.
Grading rubric (50 pts)
| Criterion | Pts | Full credit looks like |
|---|---|---|
| Quality audit | 6 | All checks reported and interpreted |
| 3D analysis & figures | 16 | Correct normalization, ROM and asymmetry, clear figures |
| 2D vs. 3D comparison | 10 | Time-aligned overlay with quantitative agreement |
| Discussion | 16 | Critical evaluation of methods, integrates semester concepts |
| Presentation | 2 | Professional and complete |
Software: MoCap6 (lab-built; Basler acA720-290gc cameras), Pose2Sim, OpenSim. Methods details for each processed trial are saved in meta/processing.json.
References
- Pagnon, D., Domalain, M., & Reveret, L. (2022). Pose2Sim: An open-source Python package for multiview markerless kinematics. Journal of Open Source Software, 7(77), 4362. doi.org/10.21105/joss.04362
- Delp, S. L., Anderson, F. C., Arnold, A. S., Loan, P., Habib, A., John, C. T., Guendelman, E., & Thelen, D. G. (2007). OpenSim: Open-source software to create and analyze dynamic simulations of movement. IEEE Transactions on Biomedical Engineering, 54(11), 1940–1950. doi.org/10.1109/TBME.2007.901024
- Puig-Diví, A., Escalona-Marfil, C., Padullés-Riu, J. M., Busquets, A., Padullés-Chando, X., & Marcos-Ruiz, D. (2019). Validity and reliability of the Kinovea program in obtaining angles and distances using coordinates in 4 perspectives. PLOS ONE, 14(6), e0216448. doi.org/10.1371/journal.pone.0216448