COMPUTER VISION · 3D PERCEPTION

DepthMotion

Monocular 3D Perception and Short-Term Trajectory Forecasting

A monocular computer vision pipeline that estimates object positions in 3D, tracks their motion and predicts short-term trajectories from RGB video.

View Demo GitHub Repository

SYSTEM DEMO

From RGB video to motion in 3D

Object detection and tracking in the RGB view with estimated trajectories and short-term forecasts visualized in bird's-eye view.

RESEARCH QUESTION

Can monocular depth estimation provide sufficiently useful 3D spatial information for short-term trajectory forecasting in a real-time vision system?

The project investigates how errors in monocular depth estimation propagate through 3D localization, motion estimation and trajectory forecasting.

PIPELINE

System architecture

01 RGB Video
→
02 YOLO
→
03 ByteTrack
→
04 Metric Depth
→
05 3D Geometry
→
06 Kalman Filter
→
07 Forecast

EXPERIMENTS

Depth evaluation

Metric depth was evaluated against Virtual KITTI 2 ground truth to quantify the spatial uncertainty entering the trajectory pipeline.

4.01 m MAE
6.06 m RMSE
13.14% AbsRel

KEY FINDING

Depth uncertainty increases substantially with distance.

Mean absolute depth error increases from approximately 0.66 m at 0–10 m to 7.64 m at 40–50 m. This creates distance-dependent uncertainty in downstream 3D motion estimation and forecasting.

IMPLEMENTATION

Technology

Python PyTorch OpenCV YOLO ByteTrack Depth Anything V2 Kalman Filter Virtual KITTI 2

Work in progress

Quantitative trajectory evaluation is currently in progress. The final study will evaluate short-term forecasting error in metric 3D space and analyze how monocular depth uncertainty affects downstream motion prediction.