The biggest bottleneck in AI right now isn’t building the app. It’s figuring out if the app actually works.
In traditional software, tests pass or fail. In AI, the output is probabilistic. To check if an LLM is hallucinating, most teams resort to “vibes-based evaluation” — engineers manually eyeballing spreadsheets of responses. It is slow, tedious, and impossible to scale.
Athina AI is killing the “eyeball test”.
Founded by Shiv Sakhuja (ex-Google) and Himanshu Bamoria (IIT Delhi, serial entrepreneur), Athina is an end-to-end evaluation and monitoring platform for LLM developers.
Backed by Y Combinator (W23) and heavyweights like the founders of Perplexity and Snorkel AI, Athina allows teams to systematically measure the quality of generative AI.
The core insight? Evaluating AI isn’t just an engineering problem. It requires domain expertise.
Athina provides a collaborative, spreadsheet-like IDE where Engineers, Product Managers, and QA teams can work together. They can run prompts, compare models side-by-side, configure 50+ preset evaluations (like testing for faithfulness or answer relevance), and monitor production traffic — all without writing complex evaluation scripts from scratch.
The market? Every team moving AI from “cool prototype” to “production-grade product” (including their current users like You.com, Meesho, and Perplexity).
Shiv and Himanshu realized that without rigorous testing infrastructure, AI will remain an unreliable toy. They are building the safety net that lets developers ship with confidence.
Let’s celebrate the builders.
w/ Jay Ingle & Dikshant Joshi
#DevTools #AIBoomiAnnual26
