The argument “a sufficiently capable autocomplete must contain a level of general intelligence” is correct but also not very useful. It is a lot like saying “a sufficiently fast horse can fly”.
It is technically correct that when you take things to the extreme you can accomplish great things, but we may not reach those levels. We may require completely different technology to reach those levels of autocomplete, and we have simply reached a new plateau at this point in time.
The argument is simpler than that. Prediction requires a model, completely accurate or not. There's a projection of the world in text. A model of the text data we feed it is a model of the world as humans see it. The trend of loss is more and more accurate models of the dataset. So it won't stop at any arbitrary competency level. Indeed, there are already a few abilities GPT possess that are deemed Super Human. It's not a distinction that matters to the machine. It's all just data to be modelled.
We have reached those levels lol. That's why we're having this argument.
I think the trouble is that "model" is a very general term. If you had a computer doing simulations of artillery shots back in the 50s, then it would have a "model" of the world in terms of variables tracking projectiles, but this model doesn't generalize to anything else. If a computer does image recognition from the 90s and 2000s to recognize faces, then the computer has a "model" of visual information in the world, but this model only lets it recognize faces.
ChatGPT has a model of all the text information on the internet, but it remains to be seen what the hard limits of this model are. Does this model let it do logic or predict the future well, or will no amount of training give it those abilities? Simply being good in one task doesn't imply a general ability to do everything, or even most of everything. LLM's would simply be the last advancement in a field with a lot of similar advancements.
>ChatGPT has a model of all the text information on the internet, but it remains to be seen what the hard limits of this model are.
Before training is complete and loss is maxed, there will be limits on what the "learned so far" model can do that say absolutely nothing about the limits of a perfect(or very close to it) model.
It really looks like anything will converge with enough compute. I don't think architecture is particularly important except as "how much compute will this one take?" question.
I've noticed that when I speak I really don't control each word.
I have an idea that I want to convey, but how each word comes to my mind as I form a sentence has always felt like it's controlled by an unconscious algorithm.
So I don't understand why people find this prediction mechanism so alien.
It isn't clear to me how much of communication is really in our control.
With the current tools, it feels like we still provide the ideas we want the AI to convey, and it may be using a nearly identical mechanism to us to form the words.
Consciousness would be the computer being able to come up with the ideas.
So, it seems to me we've gotten close enough on the communication side of intelligence.
But the machine is not conscious. When it is, it seems like it will generate its own ideas.
Are people debating whether the machine is conscious?
Otherwise, it feels very straightforward to grasp what we've made up to now.
It is technically correct that when you take things to the extreme you can accomplish great things, but we may not reach those levels. We may require completely different technology to reach those levels of autocomplete, and we have simply reached a new plateau at this point in time.