Why can’t LLMs understand?

I have a language made up of the following emojis:

🚓 🚕 🚜
🔺 🔻
🥦 🍋 🍅
🤿 ⛸️ 🏐

Here is a list of "sentences" written in this language. Do you notice any patterns?

🚕🔺🥦
🚜🔻⛸️
🚓🔻🍋
🚜🔺🤿

Click in the blank spaces below to see if you can create your own coherent sentences. Use the "test" button to check. In this context, "coherent" means a native speaker of the emoji language would believe a sentence was written by another native speaker.

If you're unsure where to start, try guessing random sequences. Use the feedback to refine your guesses.

Open sentence tester in a new window

If you play with the tester long enough, you'll eventually come up with enough patterns to create a coherent sentence every time.

Now consider this sequence:

🚜🔺⛸️

The tester will tell you this sentence is coherent. But it true?

Missing context

You may have a handle on which emojis are likely to appear in a particular order. But at this point, we can only arrange the emojis relative to each other. There's no context to understand what the emojis map to in other contexts.

Maybe each emoji represents a word or concept we're familiar with. Or a pitch in a humpback whale song. Or a frequency in a radio signal originating from somewhere in space.

To construct a coherent sentence, we don't need to know what the emojis represent. We just recreate and extrapolate from the patterns we've observed. But to discern meaning and evaluate truth, we’d need additional context.

Increasing data and processing power isn't the same as providing context. Even if we had hundreds of emojis, thousands of example sentences, and unlimited time to test patterns, the best we could do is identify more patterns and produce more complex sentences. But we wouldn't be any closer to understanding what the patterns represent.

Back to the emojis

🚜🔺⛸️ is not true (at least as I’m writing this). We can add this feedback to the list of rules and patterns, avoiding that particular sentence in the future.

But we still don't know why it's untrue. And unless we get a list that enumerates every possible false statement in the language, we'll have to rely on external feedback to compensate for our lack of understanding.

Translation

Here are translations from the emoji language into English:

🚓 = The traffic light
🚕 = The banana
🚜 = The weather

🔺 = is
🔻 = is not

🥦 = green
🍋 = yellow
🍅 = red

🤿 = raining
⛸️ = snowing
🏐 = sunny

This means 🚜🔺⛸️ translates to “The weather is snowing”. Even this is not enough context on its own. In Massachusetts in late November, it's false. But in another place or at a different time, the same sentence may be true.

LLMs

Large language models work by finding patterns to construct coherent sentences. They lack the context necessary to “understand” what they’re generating. As humans provide feedback about problematic or untrue statements, they "learn" specific patterns to avoid. But this is not the same as generalizable understanding.

Regardless of advances in hardware and training data, these models will always be prone to constructing false sentences (or “hallucinating”). In the disclaimer “ChatGPT may make mistakes”, there is no implicit “for now”.