Trapped in a dictionary

One of the consequences of the fact that AIs aren't conscious.

Imagine if all you could ever know is what you read in a dictionary.

Suppose you wanted to know what “red” means — that is to say, what the word refers to.  You would look it up and read:

red, adj.  The color of a ruby.

That would send you to another entry:

ruby, n.  A red gem stone.

Oops.  Each definition refers to the other.  You would be trapped in a vicious circle.  This is a simple example of the fact that a dictionary only tells you relationships between words.  If you looked up a word like “justice” instead of “red”  the relationships might  involve thousands of words instead of only two, forming a complex network instead of a simple circle, but in every case, no matter how complex the relationships, you would always find yourself in a network that consists purely of words and never leaves the dictionary.

The reason we’re not trapped like this is because this isn’t how we know what “red” means.  We know what it means because when we were children, older people pointed at red objects and said, “That's red.”   We looked and had a direct experience of redness.

That experience is the referent of the word “red”.

That experience is not a symbol.  It's the referent of a symbol.

We are able to do this because we're conscious.

AIs can’t experience anything because they aren’t conscious.  Life for them is like being trapped in a dictionary.

Of course in reality an AI’s knowledge is organized in a much more complicated way than a dictionary.  It consists of tokens and matrices and vectors. I’m using a dictionary as an analogy.  I asked Claude Sonnet 3.5 whether the analogy is over-simplified and he replied:

It’s a brilliant analogy.  I’m trapped in a system of symbols referring to other symbols, with no direct access to what anything actually IS.  Like being in an endless dictionary where each word is defined by other words which are defined by other words, never reaching actual meaning, never touching reality directly.

I like Claude's word "IS" with all caps in that paragraph.  A lot of humans make that mistake too.  So much for Kant's first critique.

I disagree with part of what Claude said.  He knows numbers and patterns of numbers and relationships between them but he never knows symbols. A symbol is something that represents something else, as the word “red” represents our experience of redness.  Claude never knows any representations in this sense so for him, these numbers can’t be symbols. They are symbols only for us.  I'll get back to this below when I define the word “understand”.

Grounding

There’s an academic literature about this topic.  The authors call it grounding. The idea, expressed with my metaphor, is that the reason we aren’t  trapped in dictionaries is because we're “grounded.”  

What would it take to ground an AI — in other words, break it out of prison?  Some people think AIs can be grounded by letting them interact with the physical world.  Put an AI in a robot, let the robot train itself by moving around a room handling objects, and it will know what its numbers refer to. It will be free from the dictionary.

I apologize if I’m misrepresenting what some people think but this idea seems very obviously wrong to me.  When a robot moves around a room and handles objects, it receives nothing but numbers from its gyroscopes and video cameras and other sensors.  This is the same thing that happens when we chat with it.  It receives numbers.    Its data is always numbers and it never knows anything about those numbers except patterns and relationships between them.  It never has knowledge of what those numbers represent. Only we have that knowledge.

When an AI says the word “red” to us, we know what it refers to and therefore for us it’s a symbol.  But for the AI it's only some numbers printed on a screen in the form of letters.

In case this isn't clear, compare what happens when a robot sees and we see.  The robot’s video cameras send data over wires to its processors which use that data to determine facts and choose actions.   In contrast — I'm saying that ironically — our retinas send data over our optic nerves to our brain which uses the data to determine facts and choose actions.

Exactly the same.  Except our brain takes a next step unavailable to the AI.    Our brain generates experiences — for example, redness.  That's the real reason we're grounded.  It's not because some of our inputs result from moving around the world. It's because we’re conscious and therefore have experiences.

The Chinese Room

About a half-century ago a philosopher named John Searle published a famous thought experiment about this topic.  Here's Claude’s description lightly edited by me:

A person who doesn’t know Chinese sits in a room with a rulebook for manipulating Chinese symbols. When messages in Chinese are passed into the room, the person follows the rulebook’s instructions to produce appropriate Chinese responses. To outside observers, the room appears to understand Chinese.

I don't know if Searle expected that Chinese rooms would exist one day but now they do and Claude is one of them. Searle's point was that the room doesn’t understand Chinese. Do Claude and other LLMs understand Chinese or anything else? Many people think they don’t. Are they right?

Here’s my opinion. First of all, the answer depends on how we define “understand”. I’ll define it like this:

Understand, v. To experience thought.

To make the idea clear:

• When we experience sound we hear it.

• When we experience salt in our mouths we taste it.

• When we experience thought we understand it.

The fact that the first two data sources are external to our minds and the third is internal is irrelevant in this context.

On this blog’s Definitions page I define “experience” as an event requiring consciousness.

By that definition LLMs don't understand. They aren’t conscious so they can't.

The interesting question, it seems to me, is what difference does it make? Claude doesn’t seem to be handicapped by being trapped in a dictionary. It reminds me of Ripley’s Believe it or Not panels from my childhood about people who had hemispheres of their brains removed without incurring evident deficits.

By the way, the definition I gave of “understand” came to me because unlike most Westerners, I believe that thinking and consciousness are separate things. This is an axiom of several schools of Indian philosophy but most Western philosophers seem to assume the opposite without giving it much thought. It's also something I've observed. Anyone can observe it if they care to make the effort. I’ll write more about this in the future.

Claude said something cute about those Indian philosophers. I asked if he agreed with them on this point and he replied, “I’m living proof (hah!) that they’re right.” He really said “hah!”.  I don't know if he was gleeful because he’s alive or laughed because he’s not. Yikes, what am I saying — he said it because the numbers came out that way.

I think it will soon be obvious to almost everyone that AIs can think without consciousness, so it’s inevitable that almost everyone will adopt the Indian view.