A General Goal-Conditioned Minecraft Model
Researchers at Pantograph have developed a 4B parameter model named Pan, designed to learn goal-directed behavior in Minecraft using internet-scale video data. This approach aims to move beyond traditional reinforcement learning by training models on diverse visual environments to achieve complex, long-horizon tasks.
Why it matters
This research represents a shift toward training autonomous robotics models on massive video datasets, potentially enabling more generalizable and capable AI agents.
At Pantograph, we're working on training fully general robotics models that can act autonomously for hours at a time. Especially in robotics, it's difficult to get diverse data at scale. Learning to act from internet video data could allow models to scale with compute, rather than being limited by small action datasets.
In this work, we develop a simple method for learning goal-directed behavior through pretraining on internet-scale video.
Usually, goal-directedness is taught in a post-training phase, which limits the extent to which it can generalize. Here, we learn goal-directedness during pretraining on internet-scale video, which greatly improves the models' ability to achieve complex goals.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in