About the role
Responsibilities:
- Metric 3D hand pose. Build the in-house hand model that is stable in the world frame, and decide the representation (MANO topology, 6D vs axis-angle) rather than inheriting it.
- Egocentric pose that survives the camera. We have narrowed the suspects (rectification residual, FOV, shutter type) and currently route between several public SLAM / VIO systems per clip. Replace routing-by-heuristic with a model that makes any capture device deliverable, including learned despiking and drift correction.
- Ground truth where we have none. Today we validate pose without ground truth (loop-closure drift, static-window jitter). Stand up real GT fiducial-based (AprilTag / ArUco), gravity-aligned via IMU under a hard constraint: our subjects are real workers doing their own jobs, so any protocol must add zero operator burden.
- Our own depth model. Public depth models do not survive our rigs. We need a depth model of our own: metric, stereo-native, valid across the full frame on wide FOV, and cheap enough to run on every video hour we deliver. Own the training data, the architecture call, and the accuracy-versus-throughput trade.
- Learned QC instead of thresholds. Our gates are hand-tuned numbers: epipolar residual, depth QC metrics, MCAP QA checks. Train error-detection models that flag a bad label before it reaches a customer, per clip, and quantify their catch rate against known-bad deliveries.
- Training and serving, both. Train on AWS and ship on AWS: batch GPU orchestration, throughput tuning, daily delivery SLAs. Cost per labelled video hour is one of your numbers.
Requirements:
- MS / PhD in computer vision, robotics or ML or equivalent published research output.
- You have trained a perception model that beat an off-the-shelf baseline on a real domain and shipped it. Data curation, loss design and eval decisions were yours.
- Deep hands-on work in at least two of: 3D hand / human pose reconstruction, stereo or monocular depth, VIO / SLAM, open-vocabulary detection and segmentation, video VLMs.
- Multi-view geometry is fluent, not looked up: rectification, epipolar residual, Q matrix and depth sign conventions, triangulation, world vs camera frame.
- You can read a calibration report and say whether the problem is the camera or the model and be right.
- Strong Python and PyTorch; you own training and inference code, not notebooks.
- Production inference experience on AWS: containerization, batch pipelines, orchestration.
- 4+ years on problems in this space.
Nice to Have:
- Publications in egocentric vision, 3D hand pose, VLA / VLM models, or robot learning.
- IMU-synced multimodal data, gravity alignment, camera-IMU extrinsics.
- Robotics data formats: MCAP / Foxglove, ROS, LeRobot, RLDS.
- Worked with Ego4D, EgoExo4D, Open X-Embodiment or comparable large egocentric corpora.
- Open-source contributions to vision or robotics projects.