.jpg)
AI-enabled features are becoming a normal part of software products. Support chatbots, document summarizers, RAG-based assistants, and AI-powered workflows are moving into production, often alongside the same testing practices teams use for traditional software.
That creates a problem.
Most software testing is built around predictable behavior. Given a particular input, we know what the system should return, and we can test the actual result against that expectation.
With an LLM, the relationship between input and output is less predictable. The same prompt can produce different responses across runs, even when nothing obvious has changed. The wording may differ slightly, or the difference may be significant enough to affect accuracy, relevance, or user experience.
This doesn't make AI systems impossible to test. But it does change what good testing looks like.
When there isn't always one correct response, testing needs to look beyond exact matches.
Is the response accurate? Is it grounded in the information the system was given? Does it follow the required format? Does it behave consistently when users phrase the same question differently? Does it know when it should refuse to answer?
These questions require different types of evaluation, and no single method can answer all of them well.
Some aspects of an AI response can still be checked with simple, deterministic tests. Others require evaluating meaning, context, safety, or quality. Human judgment still matters too, particularly when the definition of a good response depends on nuance or domain expertise.
The challenge is deciding how those methods should work together without making evaluation prohibitively slow or expensive.
The test data matters just as much as the evaluation method.
A test set built around a collection of clean, expected prompts may tell you very little about how an AI feature will perform once real users encounter it.
People make spelling mistakes. They phrase the same question in unexpected ways. They provide incomplete information. They ask things the product was never intended to answer. Sometimes they deliberately try to push the system beyond its intended boundaries.
For RAG-based applications, there is another layer to consider: the answer may depend on which information was retrieved in the first place.
Testing therefore needs to cover more than whether an answer looks correct. It needs to account for the range of conditions under which the feature will actually operate.
One of the more important differences with AI-enabled software is what happens after deployment.
The model behind a feature can change. The knowledge base supporting a RAG application can change. User behavior can change. Over time, production inputs may look quite different from the examples used when the feature was originally tested.
A feature can therefore continue running normally while the quality of its responses gradually changes.
That makes production behavior part of the quality process. What teams learn from real interactions needs to inform what they test next.
We created The AI Testing Playbook for engineering and QA teams working through these questions.
It looks at the AI feature lifecycle from definition through production, and provides a practical framework for deciding what to evaluate, how to evaluate it, and where different testing methods belong.
It also addresses some of the questions teams encounter once they move beyond basic AI testing: when LLM-as-judge is useful and where it can go wrong, how to approach adversarial testing, how evaluation should change with risk, and how production monitoring feeds back into the test suite.
The playbook isn't intended to replace the fundamentals of quality engineering. Test design, edge-case thinking, domain knowledge, and an understanding of real user behavior still matter.
AI adds another layer to that work.
83% faster delivery. Under 10% rework. See exactly how Xceptor got there.
