Auditomatic

Auditomatic (auditomatic.org) is a local-first web app esign designing and running systematic evaluations of Large Language Models, in your browser, no coding required.

With Auditomatic, you can test how LLM responses vary across systematic changes. Create a prompt template with placeholder variables, run it across multiple models, and see patterns in the results. No programming required.

Perfect for bias audits, prompt engineering, and understanding LLM behavior patterns for single tasks. This is a local-first web app in which we at auditomatic.org never see your data, as requests to LLM provider APIs are sent directly from your browser to OpenAI, Anthropic, OpenRouter, or whatever provider you configure.

There is both a web app at lite.auditomatic.org and a desktop version for Mac, Win, and Linux. The desktop app has identical features as the web version, but runs faster and is more secure from browser-based hacks.