Foundations
Why Can't a Machine Make Sense of This Sentence?
Before ChatGPT could answer questions, write code, or summarize documents, it had to solve a surprisingly difficult problem.
Before ChatGPT could answer questions, write code, or summarize documents, it had to solve a surprisingly difficult problem.
A simple question
Look at these three words.
Cat
Dog
Car
Which two are more similar?
Most people answer immediately:
Cat and Dog.
Easy.
Now imagine asking a machine the same question.
How would it know?
Nobody told it that cats and dogs are both animals.
Nobody explained what a car is.
Nobody even defined what similar means.
So how could a machine ever answer correctly?
That question lies at the heart of modern AI.
Reading is more than seeing words
Read this sentence.
The cat sat on the mat.
What happened inside your mind?
You probably didn't focus on each letter.
You didn't consciously identify every word.
Instead, your brain quietly created a tiny scene.
A cat.
A mat.
The cat sitting on the mat.
You weren't just reading words.
You were constructing meaning.
One way to think about it is like climbing a ladder.
Words
↓
Concepts
↓
Relationships
↓
Meaning
Let's call this the Meaning Ladder.
Every sentence you read climbs this ladder so quickly that you rarely notice it.
A machine starts somewhere very different
A machine doesn't begin with concepts.
It begins with symbols.
T h e
c a t
s a t
To us, these symbols immediately become ideas.
To a machine, they're simply pieces of text.
Before it can do anything useful with them, they first need to be represented in a form the computer can work with.
That form is numbers.
So the next question becomes:
How do you turn language into numbers?
The obvious solution
Suppose we assign every word a unique number.
Cat → 1
Dog → 2
Car → 3
Apple → 4
Problem solved?
Let's test it.
Which word is more similar to Cat?
Dog
or
Car
The machine only sees this.
Cat = 1
Dog = 2
Car = 3
Can those numbers explain why Dog is more similar to Cat than Car?
No.
Now change the numbers.
Cat → 91
Dog → 7
Car → 2
Has the meaning changed?
Not at all.
Only the labels changed.
That's our first important insight.
A number can identify a word without telling us anything about what that word means.
The real challenge
Converting words into numbers isn't the difficult part.
Anyone can assign labels.
The difficult part is creating numbers that preserve meaning.
Imagine these two sentences.
I sat beside the bank of the river.
I went to the bank to deposit money.
You instantly understand that the word bank means two different things.
Not because the word changed...
but because the surrounding words changed.
Meaning comes from context.
Could a machine represent that using numbers?
For decades, researchers struggled with exactly that problem.
If you remember one thing...
Everything you've seen in modern language AI begins with this idea:
The challenge isn't turning words into numbers. It's finding numbers that preserve meaning.
Once researchers learned how to do that, machines became dramatically better at working with language.
Where we go next
Giving every word an ID answered one question.
A machine could now recognise which word it was looking at.
But it still couldn't recognise how words related to one another.
So researchers asked a different question.
What if numbers themselves could capture meaning?
That single idea became one of the foundations of modern AI.
It's called an embedding.
And that's where our journey continues.