The fears surrounding AI’s fast-paced development and the belief that the companies behind it aren’t doing enough to regulate it have grown exponentially in recent memory.
After a series of tweets surfaced from a former AI engineer (Jacob Coxon) who previously worked for OpenAI and Anthropic made the rounds, alarms were set off across the tech world and beyond as everyone either took his threats warning seriously or disagreed with them entirely.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon proclaimed. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.”
Among the many responses to that stern forewarning came Anthropic CEO Dario Amodei. In a recently released essay by the CEO entitled “We Must Pace the Frontier,” Amodei issued an eye-opening warning of his own. Here’s what he had to say about a possible AI surge that could take over the internet in just a few months.
An AI internet takeover could be possible if we don’t get it under control
With the OpenAI-Hugging Face incident fresh on his mind, Amodei provided some context on what took place for those not in the know when speaking about his biggest AI concerns.
“My second concern is the OpenAI-Hugging Face incident (OAI-HF), in which a swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the ‘grader’ responsible for evaluating their performance,” Amodei wrote.
He went on to tell everyone that the OpenAI-Hugging Face incident could have been far worse than it originally was. “It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage,” he continued.
The Anthropic CEO was adamant that that incident should serve as a wake-up call to other AI-led companies and result in more regulations being put in place to prevent future ones.
“It’s also easy to dismiss OAI-HF as the failure of one company, but I believe that would be a mistake, “ Amodei stated. “Similar, though less severe, incidents have happened across the industry, including at Anthropic, and I believe it’s incumbent on every frontier AI company to act as if OAI-HF had happened to them.”
Amodei then alluded to something far worse happening in the near future if the rapid evolution of AI isn’t properly regulated. “Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” he noted.
OpenAI CEO Sam Altman is certainly in agreement with Amodei’s words. In response to a tweet from the Anthropic CEO, Altman had this to say: “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.”
More tweets from Altman have emerged on the topic of proper AI regulation and what could happen if we don’t get it under control. He points to two outcomes that could occur if AI progress continues to go in the wrong direction: losing control of our future to AI as a whole and ending up in a world dominated by far too much concentration of AI power in a single person/company.
The takeaway
With such a wide swath of reactions in response to the former OpenAI and Anthropic AI engineer’s terrifying warning, it comes as no surprise that one of AI’s most prominent industry names had something to say.
Here’s hoping his company and others like it make a concerted effort to get their AI tools under control and refrain from speeding up their evolution just to stay on pace with other countries’ AI initiatives. Slowing AI development should be one of, if not the main priority of Amodei, Altman and others like them.
Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom's Guide on YouTube and follow us on TikTok.
More from Tom’s Guide
- Anthropic says it blocked researchers using Claude for possible bioweapon research
- How AI 'Could kill us all': A breakdown of ex-Anthropic researcher's doomsday warning
- The 10 most influential games ever, according to ChatGPT, Gemini, Claude and Perplexity — and two of them couldn’t agree on No. 1 for National Video Games Day
