LLMsAndPostmodernism

Context: LanguageModels, PostModernism

(ReadWith) LinguisticIntelligence

I found myself writing a couple of tweets

https://twitter.com/interstar/status/1695981310202851449

Interestingly the vector encoding mechanism of LLMs is surprisingly close to the Saussure model of language as a network of differances which underlies Lacan, Derrida etc.

And Lacan takes Saussure's language as a model for mind, including the subconscious. In other words, in building LLMs, which are "minds" made of nothing but a statistical crunch of language, we've actually produced something that is very very like the way the POMO philosophers understood the human mind to be.

They wouldn't be surprised at all 🤣

To elaborate. What the Saussure model of language and the LLM people have in common is a notion that words get their meaning, or have meanings that can be understood through, the network of relations they have with other words. This is in contrast to a theory of word meaning that attributes it to some special / mystical connection between the mind using those words and the world. (Eg. something like "Intentionality" that isn't naturalised to a physical thing)

Sure, the details are different. Saussure's relations between words are differance or some criteria for disambiguation (cat means cat because of all the ways it differs from dog and squirrel and polyester). While the vector encoding is more about the statistical probability of correlations. This word is the probability of all the other words likely to precede or follow it.

Nevertheless, despite this difference, the similarities are still striking. And it wouldn't surprise me if you could tweak Saussure's model to be more like the vector encoding one and much of the usage derived from the Saussure model would still hold. Language would still be seen as holistic. And many concepts would still require contrasting or opposite concepts to be made sense of.