Goal: Inspect a test reply and identify what needs review.
Prerequisites: Use a demo agent and test data; review the side-effect warning below before sending anything.
Expected result: You can distinguish the reply, its cited sources, and its model-usage details.
Why We Test Before Going Live
Deploying an AI agent without testing it is like putting a new employee on the phones without training. No matter how good your Instructions or how clean your Vector Store is, the agent will interpret things in ways you did not expect.
Use Copilot to test replies with a demo agent and test data you control. Copilot is not automatically an inert sandbox: if the agent has skills, functions or integrations enabled, a test conversation can perform real actions. For text-only tests, use approved test recipients and turn off tools with side effects. Then test on the channel you will actually deploy, because it can look and behave differently.
Choose questions that cover common tasks, exceptions and missing information. Write an expected answer for each, compare replies with current sources, then revise and retest before enabling a live channel.
Three Ways to Work in Copilot
raia Copilot is not just a dashboard — it is your primary workspace for collaborating with AI agents.
| Suggested task | Purpose | When to Use |
|---|---|---|
| Monitor conversations | Watch conversations from configured channels available to your role | Your default state — look for trends, monitor sentiment, ensure smooth handling |
| Check a reply | Under a reply, click Sources to see which files it cited, or the model name to see which model answered and how many tokens it used | When an agent gives an unexpected answer |
| Step in when needed | Click Take Over where supported — the agent pauses and you step in; check user-visible behavior on each channel | When a conversation needs empathy, complex negotiation, or falls outside guardrails |
These are suggested tasks, not three buttons or modes in Copilot. Admin Mode, below, is the real switch.
What Admin Mode Changes
Admin Mode is a switch in your Copilot profile menu. When it is on, Copilot shows the operator view: the conversation filters, the controls for assigning a conversation and setting its status, and conversations beyond your own, depending on your role. Turning it off returns you to your own view. It does not change the agent.
Looking Behind a Reply
Each agent reply has a row of controls underneath it. Two of them help you check a reply:
- Click Sources. A Sources panel opens. Its Files tab lists the files the reply cited.
- Click the model name (for example
openai/gpt-5.6-luna). Model usage opens and shows Provider, Model, Reasoning effort, Reasoning mode, and Input, Output, Reasoning and Total tokens.
- Highlight 1: The reply names its source when the instructions ask it to.
- Highlight 2: The model name: click it to see Model usage.
- Highlight 3: Sources: click it to see the files the reply cited.
- Highlight 1: Click Sources under the reply.
- Highlight 2: The Files tab.
- Highlight 3: The cited file: northstar-support-guide.md.
- Highlight 1: Click the model name.
- Highlight 2: Model usage: provider, model, reasoning effort and mode, and token counts.
What appears can vary with the agent's configuration, the model and your permissions. Copilot does not show a confidence score, relevance scores, or the model's internal thinking. The Sources list shows the files a reply cited, not every piece of text that was searched.
The Diagnostic Framework
If the agent gives a bad answer, start with its Sources. These categories suggest what to investigate; citations and symptoms do not prove a root cause:
| What you see | Possible area to investigate | What to check |
|---|---|---|
| Sources lists the wrong file, or none | Knowledge or retrieval | Check that the intended document is included, current and complete; also check the question and cited passage (Module 4) |
| Sources lists the right file, but the answer is wrong | Instructions and use of retrieved content | Compare the actual passage with the answer, check conflicting facts and behavior rules, then correct and retest (Module 3) |
| An action you expected never happened | Skills configuration, access, or execution | Check whether the required skill is enabled, its credentials and scope, and the available action records (Module 5) |
Important: Can customers see Sources or the Admin Mode view? It depends on how your roles and channels are set up. Operator controls such as Admin Mode depend on each person's permissions. Check what a real end-user account on each channel can see before launch, and remember that anything included in the agent's reply may be visible there.
Practice check
Recommended practice: ask a question with a known answer. Compare the reply with the source document, then inspect Sources and model usage. Neither citations nor token counts prove the answer is correct.