LeRobot v0.6.0: Imagine, Evaluate, Improve
More from the blog
Looking beyond natural sequences
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
Record, train, and deploy from one place with Strands Agents, LeRobot, and Hugging Face Storage Buckets
A Blog post by Amazon on Hugging Face
Toward a future that preserves benefits of neurotechnology for all
PhD student Rachel Sava, winner of the Envisioning the Future of Computing Prize, explores transformative improvements and dystopian risks of neural technology.
From the Hugging Face Hub to robot hardware with Strands Agents and LeRobot
Related papers
Mask2Real-WM: Segmentation Masks as a Sim-to-Real Bridge for Controllable Dexterous World Models
Mask2Real-WM is presented, a two-stage action-conditioned world model for dexterous manipulation that decouples pixel prediction into a dynamics model and a rendering model that shows that mask conditioning and simulation pretraining are both required for per-DoF action controllability across all 23 degrees of freedom.
Planning-aligned Token Compression for Long-Context Autonomous Driving
This work proposes COMPACT-VA, a planning-aligned working memory framework built on conditional VQ-VAE, compressing extended context into bounded representations, and evaluates on high-signal dynamic scenarios where historical context is most critical for behavior correctness, and accordingly design behavioral metrics.
FleetScape: A Mixed Reality Sandtable for Spatial Supervision and Control of Scalable Drone Fleets
As autonomous drone deployments scale from individual units to coordinated swarms, the human operator's role shifts from direct piloting to high-level supervision. Current interfaces often treat multi-drone control as a scaled-up version of single-drone operation. We instead investigate how reframing fleet supervision as spatial interaction can better support the spatial, temporal, and safety demands of complex missions. We present FleetScape, a Mixed Reality (MR) sandtable system that externalizes layered real-time mission, safety, and environmental data while enabling fluid transitions between manual intervention and autonomous supervision. We developed a high-fidelity building inspection simulation that generates and streams synchronized multi-drone and environmental data for MR visualizations. We used this prototype to conduct a user study with six experienced drone pilots managing fleets of up to 15 drones. Our findings show that FleetScape supports situational awareness through layered spatial representations and clarifies control mode transitions. However, a limit to situational awareness was observed as fleet size increases, leading to different supervisory strategies. Finally, we derive design implications for supporting scalable drone fleet supervision.
GigaBrain-WBC-0.5: A Behavior World Model for Robust Whole-Body Control with Environment Interaction
GigaBrain-WBC-0.5, the first Behavior World Model for humanoid whole-body control, is presented, which trains a causal Transformer to jointly predict its next action, next state, and the distribution over its next latent behavior command, so the network that acts also models how the environment shapes what it can do next.