Meta announced two major advancements toward general-purpose embodied AI agents capable of performing challenging sensorimotor skills.

The first advancement is an artificial visual cortex (called VC-1) that supports a diverse range of sensorimotor skills, environments, and embodiments. VC-1 is trained on videos of people performing everyday tasks from the Ego4D dataset. VC-1 matches or outperforms sota results on 17 different sensorimotor tasks in virtual environments.

The second advancement is a new approach called adaptive (sensorimotor) skill coordination (ASC), which achieves near-perfect performance (98 percent success) on the challenging task of robotic mobile manipulation (navigating to an object, picking it up, navigating to another location, placing the object, repeating) in physical environments.

These improvements are needed to move the field of robotics forward and match the current pace in AI which will need bodies at some point.