Principles for Building One-Shot AI Agents

Summary

EdgeBit is a security platform that helps application engineering teams to find and fix security vulnerabilities. The Dependency Autofix feature contains an extremely accurate reachability engine to identify impact to your app. Most updates have no impact, so engineers using EdgeBit can dedicate efforts on impactful upgrades. This translates to more time spent “on-mission” instead of managing dependencies. This post will cover how to identify areas that can 1) use focused tools, 2) smartly handle errors and 3) harness the persistence of an AI agent to unlock massive efficiency gains, like we have done for Dependency Autofix, with some data to back it up. What is a One-Shot Agent What is a “one-shot” AI Agent? A one-shot AI agent enables automated execution of a complex task without a human in the loop. EdgeBit’s Dependency Autofix is built around one-shot code maintenance workflows - no human input is required. Unlike the typical AI within an IDE experience, where a developer manually accepts or rejects changes, our AI agents handle updates autonomously with a high degree of confidence. Our confidence comes from three sources: static analysis that deeply understands how your app uses its dependencies calculating and executing dependency updates agentic workflow that is consistent and correct (what this post is about) Confidence is extremely important because one-shot agents must do something correctly or bail out before they do damage or ask a human for review. Since we want to keep engineers focused “on-mission”, we don’t want to call for review often. Agent vs Pipeline Prior to introducing an agentic workflow for updating dependencies, our workflow was pipeline-based. This meant it was fairly deterministic since it’s given a concrete list of inputs and proceeds linearly. Experiments with an unrestricted and fully agentic workflow for automated dependency updates yielded a gain in “fuzziness” around inputs (what to update) and outputs (adapt code to API changes), b...