What is superintelligence, and who is worried about it?

Superintelligence is the idea of an AI that is far better than the best human minds at almost everything: science, strategy, persuasion, invention. Nothing like it exists today. Whether it ever will, and how soon, is one of the most argued-over questions in technology.

The word got a lot more common in 2026. Companies now name whole teams after it, a group of US lawmakers wants to ban it, and since 29 September the US government uses "super intelligence" as its official word for all AI. That last one is a naming change rather than a technical milestone, and we explain it separately in SI, AI or AGI? What the new US term actually means. This piece is about the original idea.

Where does the idea come from?

The philosopher Nick Bostrom gave the definition most people still use. In a 1998 paper he described a superintelligence as "an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills." He left open how it would be built and whether it would be conscious. His 2014 book Superintelligence brought the idea to a wide audience.

Two parts of that definition matter. "Much smarter" means not a little better, but in a different league. "Practically every field" rules out systems that are brilliant at one thing, like a chess program, which have existed for decades.

How is it different from today's AI and from AGI?

Think of three rungs on a ladder.

• Today's AI. Systems such as chatbots built on large language models can write, summarise, code and answer questions, often very well. But the International AI Safety Report 2026 calls their abilities "jagged": strong at some hard tasks, weak at some simple ones.
• AGI, or artificial general intelligence. Roughly, an AI that can do most intellectual work a person can, at a similar level. There is no agreed definition, and experts disagree about how close current systems are.
• Superintelligence. An AI that goes well beyond the best humans across the board.

Researchers at Google DeepMind put this order into a framework in 2023, according to PolitiFact, with "artificial superintelligence" at the top level, labelled "not yet achieved."

Who says it is coming, and when?

Mostly the people building it. These are their claims, not ours, and none of them is a measured fact.

• OpenAI. In a May 2023 post, the company's leaders Sam Altman, Greg Brockman and Ilya Sutskever wrote that it was "conceivable that within the next ten years, AI systems will exceed expert skill level in most domains." In a 2025 essay, Altman went further: "We are past the event horizon; the takeoff has started."
• Meta. In June 2025 Mark Zuckerberg told staff in a memo, reported by CNBC, that "developing superintelligence is coming into sight," and set up Meta Superintelligence Labs to pursue "personal superintelligence for everyone."
• Microsoft. In November 2025 Mustafa Suleyman, head of Microsoft AI, announced a team working towards what he calls "humanist superintelligence": advanced AI that is limited to specific areas and kept under human control, rather than "an unbounded and unlimited entity."
• Safe Superintelligence Inc. Sutskever left OpenAI and in June 2024 started a company with the goal in its name. "Building safe superintelligence is the most important technical problem of our time," he said, according to Pure AI.

The wider research community is less sure, and less in a hurry. A survey of 2,778 published AI researchers, run in October 2023 and published in the Journal of Artificial Intelligence Research, put the chance of machines beating humans at every possible task at 50% by 2047. That is a forecast from opinion, not a measurement, and the answers varied widely.

Why are some people worried?

The core worry is about control. If a system is much smarter than us at everything, including planning and persuasion, it may be hard to stop it doing things we don't want. This is the alignment problem: making sure an AI's goals match ours, and stay that way as it becomes more capable.

The concern is not just from outsiders. In May 2023 hundreds of researchers and executives, including Geoffrey Hinton, Yoshua Bengio and the heads of OpenAI, Anthropic and Google DeepMind, signed a one-line statement from the Center for AI Safety: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

In the 2023 researcher survey, between 38% and 51% of respondents gave at least a 10% chance to outcomes as bad as human extinction, depending on how the question was worded.

Other worries are less dramatic and closer to home. The Statement on Superintelligence, an open letter released by the Future of Life Institute in October 2025, lists losing jobs to automation, the loss of civil liberties and human control, and national security risks alongside extinction. It calls for "a prohibition on the development of superintelligence" until there is "broad scientific consensus that it will be done safely and controllably" and "strong public buy-in." CyberScoop reported more than 700 signatories at launch, from Hinton and Bengio to Steve Wozniak and the Duke and Duchess of Sussex. The statement's website showed more than 77,000 signatures in October 2026.

The International AI Safety Report 2026, written by more than 100 experts, takes a middle line. It says current systems lack the abilities to pose "loss of control" risks, where AI operates "outside of anyone's control", but that they are improving in relevant areas such as acting on their own. It also notes that some models can now tell when they are being tested, which could let dangerous abilities go unnoticed.

What do the sceptics say?

That the fear, and sometimes the hype, runs ahead of the evidence.

Yann LeCun, a pioneer of modern AI who was then Meta's chief AI scientist, told the Wall Street Journal in 2024 that the idea of AI threatening humanity was "complete B.S.", TechCrunch reported. Before worrying about controlling superintelligent AI, he wrote, "we need to have the beginning of a hint of a design for a system smarter than a house cat."

The Princeton computer scientists Arvind Narayanan and Sayash Kapoor argue that AI should be seen as a "normal technology", like electricity or the internet, whose effects take decades to spread. As MIT Technology Review reports, they object to treating AI as "a separate species, a highly autonomous, potentially superintelligent entity," and say the concept of superintelligence is too vague to be useful. In their view the bigger near-term risks are familiar ones, such as deeper inequality.

Some critics also point out that the companies predicting superintelligence are the ones raising money to build it, so their forecasts are hard to separate from their sales pitch.

What are governments and companies doing?

Very different things, sometimes in the same week.

The US government has embraced the word. On 29 September 2026 President Trump signed the executive order "Inaugurating the Era of Super Intelligence", which tells federal agencies to say "super intelligence" and "SI" instead of "artificial intelligence" and "AI". The White House fact sheet says the new name reflects capabilities that go beyond "simply imitating or automating human intelligence." But as the IAPP points out, the order keeps the existing legal definition of AI, so "SI" in US federal documents covers the same systems as before. It does not claim that superintelligence in Bostrom's sense has arrived.

Six companies signed a voluntary accord. The same day Anthropic, Google, Meta, Nvidia, OpenAI and xAI signed the White House Accord on Super Intelligence. According to Al Jazeera, they agree to internal controls, a team to oversee them, independent outside auditors and a board committee to review the results. The accord has no legal penalties, does not require audit results to be published, and gives the government no enforcement role. Trump called it "morally binding." (Anthropic makes the AI that helped draft this article.)

Some lawmakers want a ban. On 23 September 2026 Senator Bernie Sanders and Representative Greg Casar introduced the Ban Artificial Superintelligence Act. Al Jazeera reports that it would ban building AI that could "destroy or disempower humanity", pause advanced AI work until a regulator sets safety rules, create a cabinet-level Department of Artificial Intelligence, and allow prison terms of up to 20 years. A former Google DeepMind researcher quoted by Al Jazeera called it "unlikely to pass but it's courageous."

The companies also say they want laws. Al Jazeera also reports that Anthropic's chief executive Dario Amodei has called for "more serious and binding regulation of AI," and OpenAI has urged Congress to set national safety standards, even as both signed the voluntary accord.

Elsewhere, the old word still rules. In a 7 October comparison of how the EU, the US and China are handling AI, Euronews reports the renaming only in the US. China's leader Xi Jinping has said AI should be "always under human control", Euronews notes. The European Union's AI Act regulates "artificial intelligence" and has no separate category for superintelligence. For what that law does today, see the EU AI Act explained.

What does this mean for you?

For now, nothing changes in your daily life. The AI tools on your phone are not superintelligent in Bostrom's sense, and the US order does not claim they are: it renames them.

What is worth watching is the debate itself: whether the voluntary accord turns into binding rules, what definition of "super intelligence" the White House proposes to Congress, and whether the call for a ban gains support beyond open letters. When you read that superintelligence is near, check who is saying it, and whether they mean Bostrom's idea or Washington's new label.