The moment machines learned to generalize: the epic breakthrough of artificial intelligence

A few days before the release of ChatGPT, Łukasz Kaiser, the inventor of the Transformer, writes to me. He calls me and says: “Listen, Marco, check out how it translates,” and he sends me GPT-3.5. I start doing some translations and say: “Luca, yeah, it’s interesting, it’s quite fluid, but, how should I put it, it’s not state-of-the-art. We’ve been working on this stuff together for 10 years, and to me, this system seems like a step backward. But why are you showing it to me?”

And he replies: “Marco, it has never seen a translated text.” That’s when I teared up.

What had happened was that the machine had begun to generalize concepts and had figured out on its own what translating meant, without anyone ever having taught it translation.

For me, that was an epic moment. I got emotional because I realized: wow, we’re not the only ones capable of generalizing concepts.

Marco Trombetti (Podcast: The Other Uncle Sam — Episode 1) — https://podcast.ilsole24ore.com/serie/l-altro-zio-sam-AF0V6GwC

Originally published at Levysoft.