Building an Support AI Agent
How to Build and Deploy an Internal Support AI Agent in raia
🧭 Overview
An Internal Support AI Agent helps your support team get fast answers from company documentation, automate routine questions, and integrate with support systems to log or retrieve tickets.
It is:
Trained on your internal documentation
Deployed to Copilot for daily use and continuous learning
Connected to your support systems via Functions or Webhooks
Capable of scoring, summarizing, and notifying based on interactions
Eventually deployed to Live Chat to serve external users with internal oversight
🏗️ Phase 1 – Prepare & Train the Agent
✅ Use raia Academy to Prepare Content
Upload your support content (PDFs, Docs, Notion exports, etc.) into raia Academy, which:
Cleans formatting, removes artifacts
Breaks large documents into AI-digestible chunks
Outputs structured Markdown or JSON
Pushes content into your agent’s vector store
This ensures your agent has clean, relevant, and retrievable knowledge from the start.
✅ Define Agent Instructions
Use Launch Pad to define the agent’s behavior, tone, and responsibility.
Example Instructions:
You are an Internal Support AI Agent for ACME Corp. You help the support team answer questions about products, policies, systems, and tools. You respond clearly and concisely using your trained knowledge. If you're unsure, admit it and suggest escalation. You can access ticketing systems via integrated functions.
✅ Upload & Organize Training Material
Documents to upload:
Internal support documentation
Troubleshooting guides
SOPs
Tool-specific how-to docs
Ticketing process docs
System architecture or API documentation
Organize content into Training Packs for easier future updates.
🧪 Phase 2 – Deploy Internally via Copilot
🔄 Copilot for Testing and Training
Let support agents use the AI daily to:
Ask real questions
Rate responses 👍/👎
Edit answers and mark “Include in Training”
Add tags or summaries
Simulate complex issues
👥 Takeover Mode
In any live conversation, a support user can instantly take control of the thread using Copilot’s human-in-the-loop mode.
This allows:
Manual handling of sensitive or complex topics
Escalation to SMEs
Seamless handoff back to AI
🔌 Connect to Support Systems
Use the following skills and integrations:
🔄 Function Execution
Set up secure API access to:
Look up ticket status
Create new support tickets
Retrieve knowledge from internal tools
🌐 Webhook Skill
Push data from conversations into:
Jira, Zendesk, Freshdesk, or other systems
Shared inboxes or support CRMs
Internal databases
Triggered at:
OnMessageEnd(per reply)OnThreadEnd(after conversation closes)
📬 Notification Skill
Automatically email or SMS a human support agent when:
A conversation begins (e.g., urgent request)
A conversation ends (e.g., needs follow-up)
A specific tag or keyword is detected
📈 Phase 3 – Add Scoring, Summarization & Monitoring
📊 Scoring Skill
Use this skill to:
Auto-score conversations for quality and confidence
Tag interactions that require human follow-up
Filter top issues or low-performing responses for improvement
🧾 Auto Summarization
Each interaction can be summarized by the AI and:
Added to the training dataset
Shared with team members for handoff
Logged into your CRM or ticketing system via webhook
🌍 Phase 4 – External Live Chat Deployment
Once the agent achieves high internal accuracy (≥95%) and positive team feedback:
✅ Enable Live Chat Skill
Embed chat widget on internal or external site
Set welcome message and branding
Define escalation paths and operating hours
🔁 Use Copilot for Dual Purposes
Continue internal Q&A and training
Monitor external chats in real time
Intervene or take over if needed
✅ Launch & Governance Checklist
Documents transformed via Academy
✅
Agent created in Launch Pad
✅
Clear instructions defined
✅
Packs uploaded and structured
✅
Skills enabled (Copilot, Function, Notification, Scoring, Webhook)
✅
API connections tested
✅
Copilot rollout complete
✅
Scoring + summaries enabled
✅
Live Chat deployment planned/tested
✅
💡 Example Support Workflow
Support agent asks: "Where do I find the device log files?"
AI Agent replies with the exact SOP from your KB.
Agent marks the answer 👍 → added to training loop.
A conversation needs follow-up → Scoring Skill marks it.
Notification sent to the human support lead via SMS.
Conversation logged into Jira using the Webhook Skill.
Agent eventually supports external customers via Live Chat.
Last updated