Show HN: Magnitude – open-source, AI-native test framework for web apps

Summary

Magnitude: The open source, AI-native testing framework for web apps End-to-end testing framework powered by visual AI agents that see your interface and adapt to any changes in it. How it works ✍️ Build test cases easily with natural language 🧠 Strong reasoning agent to plan and adjust tests 👁️ Fast visual agent to reliably execute runs 📄 Plan is saved to execute runs the same way 🛠 Reasoning agent steps in if there is a problem 🏃‍♂️ Run tests locally or in CI/CD pipelines ↕️ Magnitude test case in action! ↕️ test ( 'can add and complete todos' , { url : 'https://magnitodo.com' } ) . step ( 'create 3 todos' ) . data ( 'Take out the trash, Buy groceries, Build more test cases with Magnitude' ) . check ( 'should see all 3 todos' ) . step ( 'mark each todo complete' ) . check ( 'says 0 items left' ) Setup Install Magnitude 1. Install our test runner in the node project you want to test (or see our demo repo if you don't have a project to try it on) npm install --save-dev magnitude-test 2. Setup Magnitude in your project by running: npx magnitude init This will create a basic tests directory tests/magnitude with: magnitude.config.ts : Magnitude test configuration file : Magnitude test configuration file example.mag.ts : An example test file Configure LLMs Magnitude requires setting up two LLM clients: A strong general multi-modal LLM (the "planner") A fast vision LLM with pixel-precision (the "executor") For the planner, you can use any multi-modal LLM, but we recommend Gemini 2.5 pro. You can use Gemini via Google AI Studio or Vertex AI. If you don't have either set up, you can create an API key in Google AI Studio (requires billing) and export to GOOGLE_API_KEY . If no GOOGLE_API_KEY is found, Magnitude will fallback to other common providers ( ANTHROPIC_API_KEY / OPENAI_API_KEY ). To explicitly select a specific provider and model, see configuration docs. Currently we support Google AI Studio, Google Vertex AI, Anthropic, AWS Bedrock, OpenAI, and OpenAI-compatible p...