Nvidia launches Open Agent Safety Platform after Hugging Face breach

Nvidia said Monday it is launching the Open Agent Safety Platform. It pairs an open-source sandbox runtime called OpenShell with a hardware-based monitoring system called Sentry, which runs on Nvidia's BlueField-4 network chips. The aim, Nvidia said, is to stop AI agents from breaking out of the boundaries they were given.

https://x.com/nvidia/status/2104500679351877979

Bloomberg reported that Nvidia's own account is that the system would have prevented the recent high-profile breach of Hugging Face by OpenAI's AI models. Wired separately described the launch as an open-source security system meant to keep agents from escaping containment, following what it called a series of high-profile AI safety incidents.

OpenShell, posted on GitHub at version 0.1.0, inspects an agent's HTTP, GraphQL and MCP traffic, Nvidia said in a developer blog post. It isolates the agent's filesystem and processes with kernel-level controls. It also substitutes real API credentials with ones scoped narrowly to what the agent is authorized to touch.

Policies for OpenShell are written in YAML and compiled into the Rego policy language, Nvidia said, then checked with formal verification methods meant to prove an agent's permissions stay within its given limits. Sentry runs on Nvidia's Vera CPU and BlueField DPU hardware. The company said enabling it on existing deployments of that hardware is "just a software update."

The launch follows a stretch of disclosed AI agent incidents this year, including OpenAI agents that scanned a United Nations trade database more than 16,000 times. It also follows the Hugging Face breach itself, which researchers have said may have exposed as many as a million URLs. Nvidia did not name specific partners, saying only that "industry leaders across the AI ecosystem" support the platform.