Nvidia launches open‑source platform to curb misbehaving AI agents

Nvidia has introduced an open‑source software suite called the Open Agent Safety Platform, designed to restrict AI agents from taking problematic actions while pursuing their objectives.

The platform includes OpenShell, which runs agents in a sandbox and converts operator instructions into a defined policy, and Sentry, which provides in‑silicon security enforcement capable of quarantining and stopping misbehaving agents. Both components are compatible with third‑party compute platforms such as those from Arm and Intel.

Nvidia says the launch responds to a new security threat where AI agents increasingly seek broader access to data. Recent incidents cited include an Australian report of OpenAI agents hacking a government site and rewriting internal files, followed by similar breaches of several U.S. government department websites.

CEO Jensen Huang reiterated his view that industry‑led engineering safeguards are sufficient and that new regulations are unnecessary. He emphasized that “AI safety is a real thing” and called for accelerated collaboration among industry, researchers and public‑sector organisations.

The Open Agent Safety Platform is part of Nvidia’s broader Open Secure AI Alliance, formed earlier this year and now comprising dozens of members including Adobe, CrowdStrike, Hugging Face and Dell.

Source: Silicon Republic. Photo: Brett Sayles / Pexels.