GPT

 This is an excerpt from one of my conversations with ChatGPT:

It's not that I'm bothered by its answer to my first question, and that it got it wrong. What bothers me is that GPT, despite being an incredible milestone in conversational AI, is still incapable of understanding context, which is what humans are instead so good at. 

The snippet above (I have many more) convinces me more than anything else that large language models, however impressive, do not display any kind of true intelligence or comprehension, and never will, due to their structural constraints.

GPT is capable of providing plausible sounding replies to a wide variety of questions by scanning through an immense knowledge corpus, but if the question is misleading or unanswerable, GPT will still give it its best shot by conjuring up some kind of plausible sounding answer, instead of recognizing the fact that there might be no answer, or that we lack the information to provide a meaningful one. This is what a really intelligent agent should do.

The way to trick GPT is to ask a common question which has no definite answer, and watch it come up with a credible guess which many non-experts would buy, but specialists know is fundamentally wrong and completely misses the point. Then, when you point that out, GPT acknowledges your rebuttal (probably by tapping into some other source in its immense corpus) without ever realizing that means its original answer to your question was completely off-mark and wrong.

What that tells me is that GPT is a glorified search engine which has no unitary self (Aristotle would have called this a soul) with a set of beliefs, opinions and centralized governance, which is the hallmark of intelligence and sentience. GPT is very good at spitting out facts from disparate sources, but doesn't have the ability to integrate all of the information from those different sources into one unified comprehensive understanding of the world. Intelligence is not about dots, but the ability to connect them.

This brings us back to the structural deficiencies of GPT and all large language models. A language model is essentially a probability distribution over words or sequences of words. So what GPT essentially does is to try to predict the next sentence based on the previous ones, given some pre-trained probability distribution.

This is not what humans do. I don't know about you, but I am not trying to predict the next word or sentence when I'm typing or speaking. What's actually happening is that I have an idea (which forms in my unitary self), and I try to spell it out the best way I can with words. Words and sentences flow out of the unitary idea and depend on it: they exist by means of and for the idea. 

Large language models instead have no unity, and therefore no central idea they are trying to convey, and no central self those ideas come from. They are just guessing what the next best sentence should be based on how other people speak in the same context. It's a giant statistical model, and I can't believe that language generation in humans follows a statistical model. There has to be something more at play there.

There is this unfounded hope that the Deep Learning paradigm will lead us into the promised land of AGI sooner or later. We just need more parameters, more training data, a better model. GPT, as impressive a technical feet as it is, proves us in the strongest possible term that that is not the case. We need to re-evaluate our theoretical principles if we want to get something truly intelligent which is capable of comprehension instead of just mimicking it. 

Human minds, it seems to me, have rich structures; they are not just tabulae rasae, as the deep learning paradigm wants us to believe. We are not just learning some statistical representation of the world, we already have one that is deeply ingrained in us. If we study that deep structure which resides within us we might have a better shot at creating sentience in other beings, if that's even possible in principle. 

We always assume that sentience is computational, somehow, but there is very little evidence for that. The current paradigm is based on the assumption that 

Sentience = Software

but it might as well be (and is actually more likely if you ask me) that

Sentience = Hardware 

If sentience (and the real comprehension that comes from it) is a biological property of living organisms, the whole program of not only deep learning but computationalism itself is doomed from the start. Perhaps we should start seriously looking into this possibility.