LinguisticIntelligence

ThoughtStorms Wiki

This is really good. One of the best, short, clear explanations of what the networks are capable of, that I've heard. (See video)

In order for LanguageModels to get good at working with language to be able to "predict the next text", they have to build a deeper model of the world.

ThreadView

Does GPT understand the world?

Here is what @ilyasut, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia:

(1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is… pic.twitter.com/ZbmqZggArq
— Alex Ker 🔭 (@thealexker) October 15, 2023

More thoughts (originally on https://twitter.com/interstar/status/1714284386424865240)

This point, that networks trained simply on text inevitably have to acquire a deeper world-model in order to handle it well, is probably ALSO a good intuition about the role of language in the evolution / development of human intelligence.

When humans invented language, we made a system of symbols that superficially described the world. But in order to get good at working with those symbols we had to internalize a deeper and more sophisticated model of the world that meant the linguistic descriptions could track it successfully, even when being passed from person to person.

This helped us create a model out of more abstract, fungible elements. Out of "ConceptualContent" (ie. the ability to reason about counter-factuals to our representations) etc.

All to be able to work well with language.

This is not going to surprise many people in PhilosophyOfMind, but AFAIK it's a quite "traditionalist" position there.