Simulations run scripted conversations against any agent version — production, staging, or a draft — so you can prove behavior before customers see it. Each turn carries pass/fail criteria, which makes regressions visible instead of anecdotal.
In this section
- Author a simulation as a sequence of user turns with expected outcomes.
- Run a suite against multiple agent versions and compare results side by side.
- Interpret failures using the same retrieval trace you get in Admin Mode.
- Gate releases on a passing suite before flipping an agent live.
What's inside
- 01
Scenario
A named script: the persona of the simulated user, the channel, and the ordered list of user turns.
- 02
Assertions
Per-turn criteria — must contain, must not contain, must call a named function, must escalate, must stay under a latency or cost budget.
- 03
Run
Execute against a chosen agent version. Every turn records the reply, the skills called, the retrieval sources, and the assertion result.
- 04
Report
Pass rate per scenario and per assertion, a diff against the previous run, and a link into each failing conversation.
At a glance
| Suite | What it covers | Run it when |
|---|---|---|
| Smoke | Five to ten core questions the agent must always answer correctly. | Every change, before publishing |
| Knowledge | Questions that map to specific documents in the vector store. | After any Train tab change |
| Actions | Flows that must trigger a Function, Webhook, or Calendar call. | After changing a skill or schema |
| Guardrails | Prompts that must be refused, redacted, or escalated. | Before a compliance review |
| Regression | Every scenario that previously failed in production. | Weekly, and before major releases |
Tip
Turn every production incident into a scenario. A suite built from real failures is worth far more than one written from imagination.