Abstract

Recent work has shown promising results for learning end-to-end robot policies using imitation learning. In this work we address the question of how far can we push imitation learning for challenging dexterous manipulation tasks. We show that a simple recipe of large scale data collection on the ALOHA 2 platform, combined with expressive models such as Diffusion Policies, can be effective in learning challenging bimanual manipulation tasks involving deformable objects and complex contact rich dynamics. We demonstrate our recipe on 5 challenging real-world and 3 simulated tasks and demonstrate improved performance over state-of-the-art baselines.

ALOHA Unleashed

We introduce ALOHA Unleashed, a general imitation learning system for training dexterous policies on robots. We demonstrate results on ALOHA 2, which consists of a bimanual parallel-jaw gripper workcell with two 6-DoF arms. ALOHA Unleashed consists of a framework for scalable teleoperation that allows users to collect data to teach robots, combined with a Transformer-based neural network trained with Diffusion Policy, which provides an expressive policy formulation for imitation learning. With this simple recipe, we demonstrate autonomous policies on 5 challenging real world tasks: hanging a shirt, tying shoe laces, replacing a robot finger, inserting gears, and stacking randomly initialized kitchen items. We also show results on 3 simulated bimanual tasks: single peg insertion, double peg insertion, and placing a mug on a plate.

Examples: Real world challenge tasks

Shoelace Tying

Center the shoe on the table, straighten thelaces, then perform a maneuver to tie the laces in a bow.

Robot Finger Replacement

Remove a robot finger from a slotted mechanism, pick up the replacement finger, roerient the finger, then precisely insert the finger back into the slot with millimeter tolerance.

Shirt Hanging: with diverse examples

Hang a shirt on a hanger. The detailed steps include flattening and orienting the shirt, picking a hanger off a rack, performing a handover, picking up the shirt, precisely inserting both sides of the hanger into the shirt collar, then hanging the shirt back on the rack.

Our model is capable of handling many different shirt types, including a shirt previously unseen in the dataset.

Unseen Shirt

Gear Insertion

Insert 3 plastic gears onto a socket with millimeter precision with a friction fit, while ensuring that the gear is fully seated and the teeth on the gear mesh with neighboring gears.

Random Kitchen

Clean up a randomly initialized table by stacking bowls, cups, and utensils and place the stack at the center of the table.