HN Reader

>Moreover, we follow previous work in accelerating block breaking because learning to hold a button for hundreds of consecutive steps would be infeasible for stochastic policies, allowing us to focus on the essential challenges inherent in Minecraft.

https://tasvideos.org/

Isn't something like finding dimonds in minecraft something that old-school AI could already do decently?

If you watch a guide on how to find diamonds it's really just a matter of getting an iron pickaxe, digging to the right depth and strip mining until you find some.

to a bright future

where we focus 100% on work

and AI will play our games

/s

Adding more than 1 GPU didn't improve speed but that's pretty standard as we don't have fancy interconnect. Bit annoying they didn't use tensorboard for logging, but overall seems like a pretty cool lib - will leave it a few days and see if it can learn (no other algo has so I dont have much hope).

If all you’re going to do is parrot things like human consciousness or human ingenuity then I will never be impressed so long that it’s just parroting.

... but the first video only shows the player character digging downwards without using any tools and eventually dying in lava. What?

https://www.youtube.com/@EmergentGarden

I very much like the comparative approach this guy takes looking at how different LLMs fare... including how they interact together. Worth a look.

But I remember the alpha version, and NOBODY knew how to make a pick ax. Humans were also very bad at figuring out these steps.

People were de-compiling the java and posting help guides on the internet.

How to break a tree, get sticks, make a wood pick. In Alpha, that was a big deal for humans also.

It seems rather tenuous to keep pounding on 'training via pixels' when really a game's 2D/3D output is an optical trick at best.

I understand Sergey Brin/et al had a grandiose goal for DeepMind via their Atari games challenge - but why not try alternate methods - say build/tweak games to be RL-friendly? (like MuJoCo but for games)

I don't see the pixel-based approach being as applicable to the practical real world as say when software divulges its direct, internal state to the agent instead of having to fake-render to a significantly larger buffer.

I understand Dreamer-like work is a great research area and one that will garner lots of citations for sure.

>> In Minecraft, the team used a protocol that gave Dreamer a ‘plus one’ reward every time it completed one of 12 progressive steps involved in diamond collection — including creating planks and a furnace, mining iron and forging an iron pickaxe.

And that is why it is never going to work in the real world: games have clear objectives with obvious rewards. The real world, not so much.

I do wonder though: if you swapped Minecraft for a cloud-based synthetic world with similar physics but messier signals, like object permanence or social reasoning, would Dreamer still hold up? Or is it just really good at the kind of clean reward hierarchies that games offer?

Can you look at the world model, like you can look at Waymo's world model? Or is it hidden inside weights?

Machine learning with world models is very interesting, and the people doing it don't seem to say much about what the models look like. The Google manipulation work talks endlessly about the natural language user interface, but when they get to motion planning, they don't say much.