NVIDIA, together with over 100 industry partners, launches a system to harness AI agents, an Open Agent Safety Platform, bringing together OpenShell and Sentry and described as a safety enforcement layer that works across software and hardware.

In an announcement, NVIDIA explains that AI agents can sometimes run out of control, reach systems they never should have been allowed to, and misreport what they did. NVIDIA sees a need to accelerate AI safety research and engineering.

NVIDIA CEO Jensen Huang emphasized the point and wrote on X that though AI is an "extraordinary" technology that enables advanced research, its full potential can only be realized when it's built to be safe:

According to the CEO, NVIDIA's new product is the beginning of an open ecosystem intended "to build the trust layer for safe agent systems."

The CEO wrote in a follow-up post that the Open Agent Safety Platform Reference Design combines NVIDIA OpenShell and NVIDIA Sentry. OpenShell is an open-source secure runtime that controls AI agent performance by giving them boundaries, tracing their actions, and enforcing policy as they work. NVIDIA Sentry is a hardware-based security system that monitors AI agents as they run; it can detect when an agent tries to go beyond its allowed software boundaries and quickly isolate or stop it.

Partners include Anthropic (Claude), SpaceXAI (Cursor and Grok), Scale AI (Scale GenAI Portfolio), and SAP (Joule Studio), which are integrating NVIDIA's agent safety technologies into their AI products and infrastructure. Other companies working with NVIDIA include Microsoft, IBM, Hugging Face, Perplexity, ServiceNow, Cisco, CrowdStrike, Palantir, and Palo Alto Networks, among more than 100 organizations.

Earlier, NVIDIA CEO Jensen Huang shared an open letter titled "Open Weights and American AI Leadership," which discusses why open-weight AI models, which can be downloaded, inspected, modified, and run on their own machine, matter. Major tech companies signed it, including Google, AMD, Cisco, Cloudflare, GitHub, Microsoft, Meta, IBM, Dell Technologies, Palantir, Hugging Face, Mistral, Andreessen Horowitz, Y Combinator, Perplexity, ServiceNow, CrowdStrike, and the Linux Foundation.

Recently, Microsoft also followed the same policy of bringing safety to using AI agents and published a draft code of conduct that outlines the intended behavior and values of its AI models. The core idea of the document is that AI agents must be under humans' control. The text is in the works and is being shared for public consideration, with the revised version planned to be published at the end of the year.

The risks associated with increasingly autonomous AI agents are already becoming apparent. For example, the machine learning platform Hugging Face was recently breached by an autonomous OpenAI agent system. Hugging Face is also set to be acquired by NVIDIA.

Subscribe to our Newsletter, join our 80 Level Talent platform, Discord, and follow us on Twitter, LinkedIn, Telegram, and Instagram, where we share breakdowns, the latest news, awesome artworks, and more.

Are you a fan of what we do here at 80 Level? Then make sure to set us as a Preferred Source on Google to see more of our content in your feed.