- Product, engineering, or risk leaders preparing an AI workflow for its next environment
- Teams that need a reusable regression harness instead of a one-time demonstration
- Decision owners comparing a model, agent, prompt, or workflow against a baseline
Sprint / 02 · AI-enabled systems
AI Evaluation and Test Harness Sprint
Evaluate an AI workflow against agreed criteria using a test harness: repeatable checks that show where it performs well and where it fails.
- 01Scope
- 02Investigate
- 03Review
- 04Handoff
How evidence is built
From the initial question to the handoff.
A convincing demonstration does not establish performance across the cases, failures, and operating conditions that matter.
- 01
Define intended use and failure
Turn the AI workflow, operating conditions, and consequential failure modes into measurable evaluation criteria.
- 02
Build a repeatable harness
Create controlled fixtures, baseline comparisons, and executable checks that can be rerun after changes.
- 03
Report readiness and limits
Analyze errors and record where the evidence supports proceeding, where revision is required, and what remains untested.
Included work
One intended use. An executable evaluation.
- Intended use, failure modes, and acceptance criteria
- Evaluation data or fixtures and a repeatable harness
- Baseline comparisons and error analysis
- Traceable results, limitations, and deployment boundary
What you receive
Repeatable tests and readiness findings.
- An executable evaluation harness
- Baseline and failure-mode results
- A reproducible evidence summary
- A proceed, revise, or stop recommendation
Engagement fit
The decision this Sprint informs.
Whether the AI-enabled system meets defined criteria for its intended use—and where it fails.
- An AI model, agent, or workflow has a defined intended use
- Representative test cases can be approved for evaluation
- A decision owner can define acceptable performance
- General AI strategy or ideation without a system and decision to evaluate
- Claims of universal safety, fairness, compliance, or readiness
- Production monitoring across operating conditions outside the agreed test boundary
5–10 business days