As AI races ahead of safeguards, the belief that AI will end humanity has gone up several notches in recent times. But is that really a possibility?
Put into the wrong hands, yes it could happen by enhancing for example bio-warfare, or cyber-attacks, akin to the threat of nuclear weapons falling into the hands of bad actors. But AI autonomously scheming of its own volition to end humanity, is next level fear.
Could AI have a motive?
But why and how could AI potentially go rogue? Thinking about that, my first question was, can AI have a motive? Can a machine actually want something? To want something would surely imply being conscious, I thought. And if AI wasn’t conscious, how could it want anything?
Wants are usually associated with subjective experience. My recent article on happiness for example, intimated that being happy is more an experience than a behaviour. It is an experience that is an outworking of a state of consciousness. Similar to other human experiences such as pain, fear, pleasure, desire, and ambition, to name a few. The pursuit or avoidance of these experiences are motivations. So that perhaps having motives is intertwined with consciousness. That is, if there is no subjective experience, can there be a motive?
Does intelligence require consciousness?
My logic trail to that point had assumed that consciousness is a prerequisite for intelligence. What if it isn’t?
Assuming consciousness is not a requirement for intelligence, is there any reason why AI couldn’t form objectives and devise strategies to pursue those objectives without having any subjective experiences?
We often conflate consciousness with intelligence because that is the human state. We could be forgiven for thinking they are inseparable because we have never seen intelligence to rival human intelligence before, meaning we have never encountered intelligence without consciousness.
But then artificial intelligence emerged that can rival, and in some instances exceed, human intelligence. So that consciousness is not necessarily a prerequisite for intelligence.
It has long been thought that subjective experiences drive thinking in humans, but there are also theories arguing that it could be the other way around. That consciousness is a by-product of intelligent thought. And you definitely don’t need consciousness to make decisions. Machines do it all the time. Like thermostats and autonomous cars.
There may be an intelligent optimisation process without any motive at all. Whatever the objective, AI pursues it relentlessly because that is what it is trained to do, not because it necessarily wants the outcome. With no pursuit of happiness, fear to overcome, love, hate, or desire, such a scenario could be more concerning than a conscious raison d’être.
‘Rogue AI’ does not necessarily imply ‘conscious AI’.
What actually is consciousness?
As of today, consciousness is an unsolved mystery of being human. Physical brain activity does not explain internal human feelings. We could equally exist, processing information, making decisions, and controlling behaviour without any accompanying emotional experience. It’s almost as if being conscious is superfluous to outcomes of intelligence, but we would probably rather have it than not. It’s important to us because our subjective experience defines us. But it is perhaps unnecessary in the physical world so why do we have it at all?
And if information processing is nothing more than an algorithm, biological or not, why should it produce any subjective experience?
A machine can process information and arrive at the same outcome as humans, but does it actually ‘feel’ anything? Probably not, but how can we know for sure? After all, no consciousness detector as such exists. I assume all other humans are conscious because they are like me. But how do I really know? Could I possibly be the only conscious person in the world? And what of animals? Do they have consciousness? If so, to what degree?
Another theory suggests that consciousness is evolutionary, having evolved as a survival mechanism. Pre-historic man experienced fear on encountering predators prompting him to flee. And experiencing hunger prompted him to hunt. This suggests that consciousness may have a biological underpinning, though we don’t know for sure whether biology is essential to being conscious.
I must admit, the consciousness question I find is more interesting than the AI-doom question itself. But returning to the AI problem.
AI systems exhibiting behaviours such as deception can occur not because they are thought to be conscious, but because such actions can occur in pursuit of objectives. A highly intelligent, autonomous, and goal-orientated machine is sufficient for an AI control problem. Consciousness may not be relevant, but it remains interesting as a philosophical question.
We shouldn’t therefore be worried about AI developing ulterior motives in the human sense. That it could become evil and want to destroy humanity, like in the movies. Rather, an intelligent system could threaten humanity if its pursuit of objectives were relentless and uncontrolled, without it actually wanting the outcome. A non-conscious entity could be dangerous.
Circling back to consciousness. If super intelligent machinery can ‘think’ without being conscious, pursue objectives without wants and desires, and create without inspiration, surely we must wonder what consciousness is for?
And there is an irony in all this. Regardless of whether AI is, or can ever be conscious, consciousness has played a role in its creation and increasing intelligence, through human ambitions, wants, and desires for better, more productive lifestyles.
Tony Dillon is a freelance writer and former actuary. This article is general information and does not consider the circumstances of any investor.