HN Reader

> Do they merely memorize training data and reread it out loud, or are they picking up the rules of English grammar and the syntax of C language?

This is a false dichotomy. Functionally the reality is in the middle. They "memorize" training data in the sense that the loss curve is fit to these points but at test time they are asked to interpolate (and extrapolate) to new points. How well they generalize depends on how well an interpolation between training points works. If it reliably works then you could say that interpolation is a good approximation of some grammar rule, say. It's all about the data.

Lots of problems with this paper including the fact that, even if you accept their claim that internal board state is equivalent to world model, they don't appear to do the obvious thing which is display the reconstructed "internal" board state. More fundamentally though, reifying the internal board as a "world model" is absurd: otherwise a (trivial) autoencoder would also be building a "world model".

They are learning a grammar, finding structure in the text. In the case of Othello, the rules for what moves are valid are quite simple, and can be represented in a very small model. The slogan is "a minute to learn, a lifetime to master". So "what is a legal move" is a much simpler problem than "what is a winning strategy".

It's similar to asking a model to only produce outputs corresponding to a regular expression, given a very large number of inputs that match that regular expression. The RE is the most compact representation that matches them all and it can figure this out.

But we aren't building a "world model", we're building a model of the training data. In artificial problems with simple rules, the model might be essentially perfect, never producing an invalid Othello move, because the problem is so limited.

I'd be cautious about generalizing from this work to a more open-ended situation.

I’m reminded of the Holographic Principle in physics: https://en.m.wikipedia.org/wiki/Holographic_principle

Sometimes a sufficiently good model of a surface is completely identical to a model of the volume.

It turns out our word for "surface statistics" is "world model".

Big thread at the time https://news.ycombinator.com/item?id=34474043

Idk from when even id this article? Got me LLMs currently are broke and the majority is already aware of this.

Copilot fails the cleanly refactor complex Java methods in a way that I’m better of writing that stuff by my own as I have to understand it anyways.

And the news that they don’t scale as predicted is too bad compared to how weak they currently perform…

Honestly, I think it’s somewhere in between. LLMs are great at spotting patterns in data and using that to make predictions, so you could say they build a sort of "world model" for the data they see. But it’s not the same as truly understanding or reasoning about the world, it’s more like theyre really good at connecting the dots we give them.

They dont do science or causality theyre just working with the shadows on the wall, not the actual objects casting them. So yeah, they’re impressive, but let’s not overhype what they’re doing. It’s pattern matching at scale, not magic. Correct me if I am wrong.

This is irrelevant, and it's very frustrating that computer scientists think it is relevant.

If you give a universal function approximator the task of approximating an abstract function, you will get an approximation.

Eg.,

    def circle(radius): ... return points()
    aprox_cricle = neuralnetwork(sample(circle()))
    
    if is_model_of(samples(aprox_circle), circle)): print("OF COURSE!")

This is irrelevant: games, rules, shapes, etc. are all abstract. So any model of samples of these is a model of them.

The "world model" in question is a model of the world. Here "data" is not computer science data, ie., numbers its measurements of the world, ie., the state of a measuring device causally induced by the target of measurement.

Here there is no "world" in the data, you have to make strong causal assumptions about what properties of the target cause the measures. This is not in the data. There is no "world model" in measurement data. Hence the entirety of experimental science.

No result based on one mathematical function succeeding in approximating another is relevant whether measurement data "contains" a theory of the world which generates it: it does not. And of course if your data is abstract, and hence constitutes the target of modelling (only applies to pure math), then there is no gap -- a model of "measures" (ie., the points on a circle) is the target.

No model of actual measurement data, ie., no model in the whole family we call "machine learning", is a model of its generating process. It contains no "world model".

Photographs of the night sky are compatible with all theories of the solar system in human history (including, eg., stars are angels). There is no summary of these photographs which gives information about the world over and above just summarising patterns in the night sky.

The sense in which any model of measurement data is "surface statistics" is the same. Consider plato's cave: pots, swords, etc. on the outside project shadows inside. Modelling the measurement data is taking cardboard and cutting it out so it matches the shadows. Modelling the world means creating clay pots to match the ones passing by.

The latter is science: you build models of the world and compare them to data, using the data to decide between them.

The former is engineering (, pseudoscience): you take models of measures and reply these models to "predict" the next shadow.

If you claim the latter is just a "surface shortcut" you're an engineer. If you claim its a world model you're a pseudoscientist.

I think they learn how to become salespeople, politicians, lawyers, and résumé consultants with fanciful language lacking in facts, truth, and honesty.