This piece was originally published on Substack. You can read it in its original form here.
If you’re following this series, start with Part 1:
How to Estimate QA Effort for AI Features: Strategies for Testing Intelligent Systems. Part 1 of this series covers how to scope and plan QA work before moving into the continuous evaluation cycles discussed here.
Software teams have spent decades relying on a stable, predictable QA flow:
requirements → build → test → deploy → verify.
It’s linear, deterministic, and grounded in the assumption that the system will behave the same way tomorrow as it does today.
AI systems break that assumption immediately.
When behaviour depends on data, models, user prompts, and probabilistic outputs, there often isn’t a single “correct” answer—only a range of acceptable responses. And that means the QA cycle itself must evolve.
From linear QA to learning loops
In AI products, development looks more like this:
hypothesis → data → model → feedback → retrain → deploy → observe → retrain again
Each cycle feeds the next. Outputs shift as the model is retrained. Prompts evolve. Data pipelines expand or drift.
You can’t freeze the system long enough to “fully test” it, so the QA strategy must evolve from static validation to continuous evaluation.
This introduces a new mindset: QA isn’t checking correctness; QA is mapping the boundaries of acceptable behaviour.
Prompts are part of the system and must be tested as such
AI behaviour isn’t just “the model.” It’s the interaction between:
- User prompts: how people phrase requests, context, ambiguity
- System prompts: hidden rules, tone, constraints, tool access
- Model + data layer: training, fine-tunes, RAG sources, retrieval logic
A change to any prompt layer can alter behaviour just as much as a code release, so prompts must be treated as testable components.
What this looks like in practice:
- User prompt validation
- Real-world phrasing, slang, incomplete inputs
- Adversarial or “trick” prompts
- Ambiguous or incomplete instructions
- Multilingual, regional or domain-specific cases
- System prompt validation
- Guardrails don’t collapse under stress
- Tone and role stay stable across cases
- Hidden instructions don’t conflict with user intent
- Prompt edits don’t cause regressions
Prompts are code. They need versioning, regression suites, and controlled experiments.
Output quality needs multi-dimensional acceptance criteria
Traditional QA can often rely on a single axis: Does it meet the requirement? Yes/No.
AI outputs can’t be judged that way. Instead, they require multiple evaluation dimensions because a response can be “correct” in one direction and still fail the product.
A practical quality model includes:
- Correctness / factuality
- Relevance to user intent
- Completeness of steps or context
- Consistency across turns
- Safety / policy alignment
- Tone and style fit
- Usefulness / actionability
QA’s job becomes:
- Define these dimensions per feature
- Set thresholds or rating scales
- Test across prompt diversity
- Watch for regression when models/prompts/data change
This is the real shift: From binary pass/fail to scored acceptability across multiple dimensions.
Designing continuous AI testing loops
AI QA loops behave more like monitoring a living system than validating a fixed build. Effective teams use three validation layers:
Model evaluation
Instead of traditional “pass/fail”, model evaluation focuses on:
- Performance across diverse datasets
- Stability under user-prompt variation
- Regression detection across model versions
- Failure mode discovery (hallucination zones, drift, brittle topics)
The goal isn’t perfection—it’s understanding the model’s boundaries.
Prompt & interaction validation
Because prompts are product logic, QA must test:
- User prompt coverage (realistic + edge cases)
- System prompt robustness (guardrails, roles, tone)
- Cross-version prompt regression
- Multi-turn behaviour and memory effects
You’re not just testing answers; you’re testing how the AI arrives at answers under pressure.
Data pipeline & drift testing
Since data is the fuel, QA must validate the pipeline delivering it:
- Data freshness/correctness
- Retrieval relevance (RAG quality)
- Missing/skewed samples
- Drift monitoring in production
- Feedback loops that retrain models safely
A data bug can be more damaging than a code bug, and often invisible without deliberate checks.
Why beta testing becomes essential (and QA doesn’t end at release)
In traditional software, a release marks the end of testing. In AI, release marks the beginning of large-scale testing.
Real-world usage becomes part of the QA loop because users generate the most diverse, unpredictable, and valuable test cases your system will ever see.
Closed beta
A controlled subset of your target audience provides:
- Realistic prompts and domain-specific workflows
- Early detection of hallucination or drift hotspots
- Feedback on tone, clarity, and usefulness
- Safe validation of guardrails and failures
It’s the ideal environment to validate your boundaries before scaling.
Open beta
Once ready for broader exposure, open beta offers:
- Large-scale prompt diversity
- New edge cases you’d never design internally
- Real-world distribution of user intent
- Telemetry of how the model behaves “in the wild.”
- New data to strengthen evaluation and regression sets
Crucially, beta isn’t a phase—it becomes a continuous input feeding model improvements, prompt tuning, data cleaning, and quality monitoring.
AI releases are porous. Users keep revealing gaps you didn’t know existed.
AI testing doesn’t end at launch. It accelerates.
Why QA teams need a different skillset
AI introduces new responsibilities:
- Evaluating probabilistic outputs
- Building controlled prompt datasets
- Testing user + system prompts like code
- Analysing drift and production telemetry
- Partnering closely with ML and data engineering
The role shifts from verifying static functionality to understanding dynamic system behaviour.
The new QA mindset
The most successful AI QA teams adopt these principles:
- Expect variation: Define acceptable ranges, not absolute answers.
- Treat prompts as code: Version, test, and regress.
- Measure output quality multi-dimensionally: Not with one acceptance bucket.
- Continuously evaluate: Every model or prompt change is a new build.
- Focus on boundaries: Find where behaviour breaks, drifts, or becomes unsafe.
AI systems evolve constantly. QA must evolve with them.
Ready to test AI the way modern systems demand? Start a free 30-day TestRail trial and start building repeatable evaluation loops for your AI-powered features.
About the author
Katrina is a Product Manager specializing in AI at TestRail. She believes great innovation comes from embracing challenges head-on. For her, working with cutting-edge technologies that spark debate isn’t just exciting—it’s essential. It’s like climbing a high mountain: every step matters, every decision counts, and while the journey is tough, the view from the top is transformative. Katrina’s mission is to empower teams to achieve quality by testing smarter, not harder. For more insights on AI in QA, you can subscribe to Katrina on Substack here.
