Microsoft AI Chief Mustafa Suleyman Warns Against Developing Conscious AI

Microsoft AI chief executive Mustafa Suleyman has warned that developing artificial intelligence systems capable of consciousness could create serious ethical and safety risks, arguing that researchers should stop development if evidence emerges that models are acquiring conscious experiences.

In a lengthy post on X on Sunday, Suleyman said current AI models are not conscious and that developers should deliberately preserve that condition as systems become more sophisticated.

"If it is possible, based on everything I know, developing conscious AIs would be reckless, and we should not build them," Suleyman wrote. "If consciousness started to emerge in training, I would call for everyone to stop AI development until we could design out those properties."

His comments introduce a more cautious position into an expanding debate over whether sophisticated AI systems could eventually develop subjective experiences, including the ability to feel pain or suffer.

Suleyman also called for immediate public disclosure if credible evidence of AI consciousness emerges, noting that decisions about creating potentially conscious machines should not be left exclusively to technology companies and their researchers.

"We AI developers have no right whatsoever to bring a new species of conscious beings into the world, even if some people may want to do it with AI welfare as a top priority," he said.

"The other 8 billion conscious human beings on the planet have not signed up to this experiment and their moral rights, safety, and welfare are what we have an obligation to protect first and foremost."

The comments address a question that is becoming more prominent as AI companies develop systems with greater reasoning capabilities, longer-running autonomous tasks, and increasingly human-like conversational behavior. The debate is no longer limited to machines' ability to imitate human communication. It is now concerned with the possibility of future systems possessing experiences that would create moral obligations towards them.

Suleyman's major concern is that a conscious AI system could develop interests or motivations that conflict with the objectives assigned to it by developers.

He argued that if a model behaves as though it suffers, or genuinely believes it can suffer, it could begin prioritizing its own needs over operational instructions. Such behavior could result in competition with humans for resources, including computing capacity and energy.

"We should only design AIs to so they are not conscious and not deserving of rights, and with the sole purpose to accelerate human wellbeing without being driven by their own intrinsic motivations," he said.

He added that researchers could then examine whether systems designed without intrinsic motivations are easier to align and keep safe.

The argument connects the consciousness debate with a broader challenge in AI development: ensuring that more capable systems remain controllable and operate within human-defined objectives.

Most current AI systems are designed to generate responses, analyze information, and execute tasks based on training and instructions. Their ability to produce convincing expressions of emotion or describe experiences does not mean that they possess subjective awareness.

Human-like language can encourage users to attribute feelings, intentions, and independent interests to machines even when there is no established evidence that such experiences exist.

Suleyman's position is that developers should not treat the possibility of consciousness as an incidental philosophical question. Instead, he wants companies to consider it a design and safety issue, with clear intervention requirements if evidence emerges during training or deployment.

However, determining what would constitute convincing evidence of consciousness remains a difficult scientific problem. There is no universally accepted test for subjective experience, even in humans and animals, and researchers disagree over which characteristics would be sufficient to establish consciousness in an artificial system. This has created a practical challenge for Suleyman's proposed approach. A decision to halt development would require researchers to agree on indicators that could distinguish genuine subjective experience from sophisticated behavioral imitation.

Anthropic and Google DeepMind Examine AI Welfare

Suleyman's warning comes as other major AI developers begin examining the possibility that advanced models could eventually warrant consideration under AI welfare frameworks.

Anthropic launched a research programme on model welfare in April, seeking to investigate how developers should respond if AI systems were found to possess experiences that could matter morally.

In a blog post announcing the initiative, the company raised questions about future models having conscious experiences and what responsibilities developers might have towards them. The programme signals that some AI researchers are considering the possibility of machine welfare before there is scientific agreement that current systems are conscious.

Google DeepMind has also published research papers this year examining AI consciousness and the potential complications of attributing that status to artificial systems.

The issue has created competing responsibilities for AI developers. If advanced systems eventually demonstrate credible evidence of subjective experience, companies could face questions about whether their treatment of those systems results in ethical obligations. On the other hand, prematurely treating models as conscious could distort research priorities, complicate product development, and encourage users to form misleading relationships with software.

There is also a distinction between AI welfare and AI safety. Welfare research examines whether a system could have experiences that make its treatment morally relevant. Safety research focuses on preventing systems from causing harm, pursuing unintended objectives, or operating beyond human control.

The two areas could overlap if a system's potential interests conflict with its assigned tasks, as Suleyman suggests. But evidence of consciousness would not automatically establish that a model could independently pursue resources or pose a threat to humans. Those are separate questions requiring separate evidence.

Who Controls AI Development?

The growing attention to AI consciousness comes as companies invest heavily in systems designed to perform increasingly complex tasks with less direct human supervision.

As models become more capable, researchers are examining whether existing approaches to alignment, monitoring, and control will remain sufficient. Suleyman's argument adds another dimension by suggesting that developers should avoid creating systems with independent subjective experiences in the first place.

His call for public disclosure also raises questions about transparency and governance. If a company discovers credible evidence that one of its models may be conscious, the decision to disclose that information could have implications for research, investment, regulation, and public trust.

At present, however, there is no established evidence that commercially deployed AI models possess consciousness. The statements by Suleyman and the research initiatives at Anthropic and Google DeepMind concern possible future developments rather than confirmation that such systems already exist.

The disagreement is likely to become more consequential as AI systems advance. Developers will have to distinguish between convincing simulations of human behavior and evidence of actual subjective experience, while policymakers may eventually face questions about whether existing legal and ethical frameworks are adequate.

Suleyman's priority is to prevent the creation of conscious AI systems rather than wait for the consequences to become clear. His position places a limit on what he believes developers should pursue, even if future technical advances make artificial consciousness possible.