A new skill finds AI agent risks, fixes them, and proves the fix worked
What happened
The first is that a team’s written requirements already capture the risks that matter, when in reality, the most consequential failures are often the ones no one thought to write down. As more teams adopt these tools, both assumptions are becoming harder to rely on.
Each of those posts, however, began from requirements a team had already written down. That practice rests on two assumptions that don’t always hold.
The second is that someone has the time and expertise to connect every step by hand, translating findings into policy and rebuilding the comparison without compromising it. Starting before the requirements Because this work is new, a brief recap is useful for readers meeting it here for the first time.
The behaviors an agent must respect are shaped by its product context, its policies, and its tools, and the evaluation should be generated from those requirements rather than borrowed from generic metrics. In August, “ One requirement, many failure paths ” showed how the two work together in practice.
Sources & evidence
- Microsoft News Primary / official
A new skill finds AI agent risks, fixes them, and proves the fix worked ↗
https://commandline.microsoft.com/run-assert-eval-responsible-ai-agent-risk-discovery-at-runtime/