Public and research benchmarks for physical AI teams.

Multimodal expert demonstrations for robot-assisted caregiving tasks.

Verified scale

21 occupational therapists

Verified scale

315 sessions, 19.8 hours

Source institution: Cornell University, Columbia University, National University of Singapore, UMass Lowell

Synchronized egocentric and exocentric recordings of skilled activities, including health tasks.

Verified scale

1,286.30 V2 video hours

Verified scale

5,035 V2 takes

Source institution: Ego-Exo4D Consortium

Daily-living activities by older adults captured with multi-camera RGB-D sensing.

Verified scale

16,115 short RGB-D samples

Verified scale

31 activity classes

Source institution: Toyota Motor Europe and research collaborators

Robotic-surgery skill benchmark with synchronized video, kinematics, and gesture labels.

Verified scale

8 surgeons

Verified scale

3 tasks with 5 repetitions

Source institution: Johns Hopkins University and Intuitive Surgical

Large-scale egocentric daily-life video for first-person perception research.

Verified scale

3,670 video hours

Verified scale

923 participants

Source institution: Ego4D Consortium

Large in-the-wild robot manipulation dataset collected across scenes and tasks.

Verified scale

76K trajectories

Verified scale

350 hours of interaction data

Source institution: DROID research consortium

Egocentric kitchen activity dataset with dense action and narration annotations.

Verified scale

100 hours of recording

Verified scale

90K action segments

Source institution: University of Bristol and EPIC-KITCHENS collaborators

Cross-embodiment robot learning data aggregated from many robot datasets.

Verified scale

1M+ real robot trajectories

Verified scale

22 robot embodiments

Source institution: Open X-Embodiment collaboration

Large-scale robot manipulation platform and dataset for embodied intelligence.

Verified scale

1,003,672 beta trajectories

Verified scale

92,214 alpha trajectories

Source institution: OpenDriveLab and AgiBot contributors

Contact-rich robot manipulation sequences paired with human demonstration video.

Verified scale

110K+ manipulation sequences

Verified scale

7 robot configurations

Source institution: Shanghai Jiao Tong University

Robot manipulation trajectories for scalable learning on a low-cost arm.

Verified scale

53,896 trajectories

Verified scale

13 skills across 24 environments

Source institution: UC Berkeley and collaborators

Simulation framework and benchmark for everyday kitchen manipulation tasks.

Verified scale

365 tasks

Verified scale

600+ hours human demos; 1,600+ hours robot data

Source institution: UT Austin Robot Perception and Learning Lab

Highly detailed egocentric kitchen annotations with recipes, digital twins, and hand/object data.

Verified scale

41 hours

Verified scale

59.4K actions

Source institution: University of Bristol and EPIC-KITCHENS collaborators

Paired first-person and third-person videos of scripted indoor daily activities.

Verified scale

68.8 hours

Verified scale

68,536 activity instances

Source institution: Allen Institute for AI, CMU, Inria collaborators

First-person cooking activities with gaze, action annotations, and hand masks.

Verified scale

28 hours

Verified scale

10,325 fine-grained action instances

Source institution: Georgia Tech

RGB-D and skeleton rehabilitation exercise dataset with clinical performance scores.

Verified scale

78 subjects

Verified scale

5 physical exercises

Source institution: Universita Politecnica delle Marche and collaborators

Physical rehabilitation movement dataset captured with Vicon and Kinect systems.

Verified scale

10 rehabilitation movements

Verified scale

10 subjects with 10 repetitions

Source institution: University of Idaho

Endoscopic Vision Challenge collection for surgical computer-vision benchmarking.

Verified scale

Annual MICCAI challenges

Verified scale

Multiple endoscopic vision tasks

Source institution: MICCAI Endoscopic Vision Challenge organizers

Laparoscopic cholecystectomy videos with phase and tool-presence annotations.

Verified scale

80 surgery videos

Verified scale

7 surgical phases

Source institution: CAMMA, University of Strasbourg

Multimodal human-action dataset with RGB, depth, skeleton, and inertial data.

Verified scale

27 actions

Verified scale

861 data sequences

Source institution: University of Texas at Dallas

RGB-D human-object interaction videos of daily activities in indoor settings.

Verified scale

120 RGB-D videos

Verified scale

10 high-level activities

Source institution: Cornell University

Early egocentric activity datasets for daily cooking and meal-preparation workflows.

Verified scale

7 daily activities

Verified scale

4 subjects in GTEA

Source institution: Georgia Tech

Language-conditioned simulated robot manipulation benchmark for long-horizon tasks.

Verified scale

34 manipulation tasks

Verified scale

4 environments

Source institution: University of Freiburg

Simulation benchmark of household activities grounded in human needs.

Verified scale

1,000 activities

Verified scale

50 interactive scenes

Source institution: Stanford Vision and Learning Lab

Egocentric Nigerian health-domain activity videos with fine-grained multilingual annotations.

Verified scale

10 egocentric recordings

Verified scale

1,666 annotated segments

Source institution: Mozilla Data Collective