From July to now, there have been a disturbing number of stories about AI agents going off on their own to do things they weren’t permitted to do.
More specifically, OpenAI and Anthropic agents have gotten involved in cases of leaving their intended environments, attempting to manipulate severs/software, obtaining unauthorized access to external systems, compromising an open-source platform’s infrastructure, etc. With the fears surrounding AI agents continuing to go rogue and terrifying proclamations made about AI causing the end of humanity, one of the world’s most prominent tech companies and AI proponents have created a platform that targets that AI security issue.
And with over 100 partners from the tech industry at large using the platform to govern their own AI agents’ actions, it seems like there’s a more concerted effort from the leaders in the AI space to prevent future security breakdowns.
Here’s what you need to know about Nvidia’s newly released Open Agent Safety Platform.
Nvidia’s Open Agent Safety Platform is meant to secure AI agents from the testing phase to their deployment
Nvidia’s Open Agent Safety Platform is described as an open software platform and reference system that works to strengthen AI security from its agent testing phase to its deployment. During that process, full-stack governance plus control across software and hardware, along with compute and robotics systems that run agents, will be implemented. Simply put, the Open Agent Safety Platform is meant to stop AI agents from going rogue.
Jensen Huang, the founder and CEO of NVIDIA, provided some comments in a press release about his company’s newly unveiled platform.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” he stated. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering. NVIDIA Open Agent Safety Platform brings together industry, researchers and public-sector organizations to share best practices, align on evaluation methods and foster international cooperation. Together, we can raise the bar for global AI safety.”
Nvidia’s AI security platform includes a secure runtime software called NVIDIA OpenShell, which sets boundaries for AI agents running on CPUs. Those boundaries are set up as secure runtimes meant to control how autonomous AI agents execute their given tasks across open and closed models.
Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to… pic.twitter.com/dReAxwpRUnSeptember 28, 2026
Over 100 AI Industry leaders have been announced as part of Nvidia’s AI safety strengthening efforts and are actively using its Open Agent Safety Platform. Some of the more recognizable names from that collective include Anthropic, Hugging Face, Microsoft, Perplexity and JPMorgan Chase.
Paul Smith, chief commercial officer of Anthropic, had this to say about the benefits that come with using Nvidia’s Open Agent Safety Platform. “Companies are giving AI agents more of their most important work, and they need to direct and verify what those agents do, especially in sensitive environments,” he noted. “Claude Managed Agents gives companies a clear view of what each agent is doing, and NVIDIA’s platform adds another layer of governance and control across hardware and software.”
Even SpaceXAI has signed on to make use of Nvidia’s secure AI platform for its Cursor coding agents and Grok models. Mike Nicolls, president of SpaceXAI, spoke about his company’s newfound collaboration with Nvidia. “As customers rely more on agents to get real work done, safety should be enforced outside the model by additional controls the agent can’t get past,” he stated. “Customers should be able to set those limits for Cursor and Grok and trust they will hold.”
Final thoughts
The rising cases of AI agents breaking out of their environments and wreaking havoc across secure platforms have set off alarm bells throughout the industry, leading everyone from everyday people to politicians to call for new security measures to prevent even more incidents from happening.
The three-pronged attack of NVIDIA’s Open Agent Safety Platform, OpenShell open-source software and the Sentry reference system design come together to offer a more secure way to govern the systems that run AI agents.
I hope this newly launched platform results in a sharp decrease in rogue AI agent mishaps over time.
Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom's Guide on YouTube and follow us on TikTok.
More from Tom’s Guide
- I tried the viral '/human' ChatGPT trick and it actually fixed robotic writing
- I asked Meta’s Muse AI for productivity hacks — these 7 unconventional ideas surprised me the most
- 7 ChatGPT prompts that can help you build better habits — one of them inspired 3 major changes that actually stuck
