KB Content: Work in progress
The OpenAI GPT Realtime implementation is currently tested in active development. This document will be expanded with further information. Until this note is removed, please do not implement the instructions below in a productive environment.
INC Work in Progress
This area of the Knowledge Base is still work in progress (WIP). Expect changes to the layout of this page as improvements are made.
INC Use Case Preview Feature
PREVIEW - This Use Case showcases features that may not yet be available to all customers. Note that functionality, scope, and design of preview features may change significantly in future. As we define the last technical details, requirements and steps may be subject to change.
In this Use Case, we're going to address the following topics in order to set up Virtual User in Nimbus for customer intent analysis. Topics covered are as follows:
- How to use Microsoft Copilot Studio to set up your first AI Agent acting as a Virtual User.
- How to configure Nimbus in order to use the Virtual User in your workflows.
- How to set up the workflow and make service calls for further adjustments.
INC Icon Legend Accordion
Show Icon Legend 💡 = A hint to signal learnings, improvements or useful informati...
Show Icon Legend
| 💡 = A hint to signal learnings, improvements or useful information in context. | 🔍 = Info points out essential notes or related page in context. |
| ☝ = Notifies you about fallacies and tricky parts that help avoid problems. | 🤔 = Asks and answers common questions and troubleshooting points. |
| ❌ = Warns you of actions with irreversible / data-destructive consequence. | ✅ = Intructs you to perform a certain (prerequired) action to complete a related step. |
Preconditions
INC AI and Virtual User Preconditions
PRECONDITIONS: AI Features, Bots and Virtual Users
Admin Consent: AI Features
✅ Nimbus Tenant Admin consent is required to enable the use of AI features. This is done in the Data Privacy Tenant Settings. AI use will require your Tenant data (e.g. caller information) to be processed by external services
💡Good to know - shared setup effort: Once consent is granted, both Admin Roles “Tenant Admins” and “OU Admins” are able to create various AI Configuration items in the Nimbus Admin Portal. This also includes:
- Access to Service Distribution Settings, e.g. to test workflows with your configured Virtual Users.
- Steer feature visibility via Companion Service Settings (e.g. when activities are supposed to happen unobtrusive in the background).
Virtual User Licensing
✅Nimbus License Management:
-
Contact Center Enterprise Routing service licenses are required to use AI-driven Virtual User functionality.
⮑With the license applied, corresponding Service Features (e.g. workflow activities, configuration tabs) are enabled. -
Individual Virtual User license requirement: Same as regular Nimbus users, each Virtual User requires a separate license. This license can be applied individually in the Virtual User configuration - or in bulk - via Admin > Licensing view.
⮑The license enables usage of the “Add Virtual User” Workflow Activity, without it the activity will be skipped over. For additional Virtual User licenses on your service, get in touch with your Customer Success representative.
Virtual User Bot Configuration
Virtual Users rely on Nimbus Configuration items:
✅ ALL: Virtual Users require Bots to be configured. Your bot choice also determines the underlying AI LLM into which the Nimbus Virtual User integrates. Your bot typ and profile should therefore be chosen with the intended use case in mind.
INC Language model comparison matrix
| Defined in Bots | Defined in Virtual Users | |
|
Type Nimbus managed or BYO |
Agent Type Underlying LLM |
Virtual User Profile Pre-built configurations to fulfill a certain use case |
| Nimbus AI Service | Azure OpenAI1 |
|
|
Custom AI Service
|
|
|
| M365 Copilot Direct Line 3.0 |
|
|
Notes
1 Due to Microsoft Azure AI foundry availability in limited regions, data processed by a Nimbus Virtual User “GPT”-Realtime" integration might temporality leave due to failover scenarios:
- Nimbus Cluster DE01, DE02, CH01, CH02, UK01, EU01, AU01 computation will be primarily performed in Sweden Central region.
- Nimbus Cluster US01 computation will be primarily performed in the US East region.
✅SPECIFIC AI services / Agent technologies (e.g. Microsoft Copilot) may require additional configuration items such as:
- Bot Response Templates, to map bot answers back into Nimbus workflow parameters.
- Speech Recognizers (Transcribers) to convert the Customer voice into text for the bot to process.
🔎For more details and specific configuration steps, refer to our AI Use Cases category and the specific Use Cases and pages linked above.
Azure Preconditions
Microsoft Services: In order to use AI-driven Bots in Nimbus, you will need access to Microsoft Cognitive Services. Azure Bot usage causes additional costs outside of your Nimbus subscription. Please note the licensing remarks below.
INC Azure Billing Cognitive Services
Disclaimer: Support for Azure Billing & Cognitive Services
☝ The usage of Microsoft Azure AI Services for Speech1 and AI2 will cause additional costs outside your Nimbus subscription, which are exclusively determined by Microsoft.
Please note:
- Nimbus Support does not cover pricing discussions or make recommendations on Microsoft Azure Services. → Please inform yourself via the references below on pricing models.
- Nimbus Support can not provide any technical help, insights or solutions on Azure Cognitive Service related technical problems.
1 Azure Speech Services: https://azure.microsoft.com/en-us/pricing/details/cognitive-services/speech-services/
2 Generative AI Services (Copilot Studio):
Use Case Overview
Details will be covered below in this Use Case. As a quick tl;dr, here is an overview of items you will need:
| What? | Where? | Why? |
|---|---|---|
| ✅ Copilot Bot Setup1 | https://copilotstudio.microsoft.com/ | To define the bot role, behavior, topics, triggers and to get the API authentication details |
| ✅Bot Configuration | Nimbus Admin > Configuration | To get the Bot API endpoint and Authentication details into Nimbus |
| ✅Bot Response Templates | Nimbus Admin > Configuration | To map the bot response JSON to Nimbus parameters for further processing within workflows. |
| ✅Virtual Users | Nimbus Admin > Configuration | To define the initial bot instructions as well as tying all other configuration items together. |
| ✅ “Add Virtual User Activity” (in Workflow) | Workflows > Conversation Handling Activities | To invite the Bot to the conversation and route the outcomes accordingly. |
1 Note that behaviors, UI designs, and configuration details are subject to frequent changes by Microsoft and may mismatch from our use case descriptions. Feel free to contact our support to request an update to our use case documentation.
Known Limitations
INC AI and Virtual User Limitations
General note on AI-driven interactions
AI driven replies are not deterministic and depend highly on what the Customer is saying. For example, if the Customer is saying “I have trouble with my internet” it is not necessarily given that the Bot will associate “Router, Modem” as your workflow routing exit, unless specifically handled in your Virtual User integration. In this specific example, AI instructions should also cover alternative wordings like “Router, Internet”, to be handled in topics accordingly.
🔎Refer to our Best Practices - Virtual Users in Nimbus for some considerations and risks when using Virtual Users instead of Human interaction.
General Virtual User limitations
The current Nimbus implementation with AI-driven bots underlies the following limitations:
- Supported Modalities: Virtual Users (Bots) are currently available for Audio/Video modality tasks.
- Virtual User Reporting: Sessions involving Virtual Users are not reflected as dedicated User Session. Virtual User session reporting is planned for a later point this year.
Microsoft Copilot Limitations
- Expect processing delays: Processing AI answers takes a few seconds for voice-to-text-transcription, followed by AI processing and then the same transcription back into a voiced response. Luware is trying to minimize the delay on Nimbus call infrastructure, but the dependency on external APIs will always incur this delay. The Customer will hear silence during this processing and no audio feedback / silence detection.
-
Ensure you have no ambiguity in your topics. For instance, the word “
Service” may be too generic if you want to transfer to different services. Rather usehealthcare|medical|emergencyas identifiers or use more complex Regular Expressions to identify replies.
Nimbus AI Services - Profile: Intent Analyzer
🔎By design (not a limitation or reportable issue):
- The Nimbus Intent analyzer Bot runs on pre-defined prompts. Any usage outside of the descriptions specified under Use Case - Setting up a Nimbus Virtual User for Intent Analysis is not supported.
-
Fallback handling: If a workflow uses exits with multiple parameters, e.g. to collect
customer nameandinvoice number, and only one parameter can be retrieved, the fallback exit is taken.
Nimbus / Custom AI Services - Profile: AI Workflow
🔎By design (not a limitation or reportable issue):
- Failed condition: If either a Web Request connection or MCP server connection fails, the “Failed” workflow activity exit will be taken immediately. Also applies for any other Nimbus-internal or technical error.
- Fallback condition: Any “Fallback” parameters will be updated with the response generated by the AI when a case cannot be handled.
☝️Current limitations, coming with a later update:
- For this profile there currently are no custom exits in the “Add Virtual User” Conversation Handling Activity. You only have Success, Fallback, Failed, Idle timeout.
- There is currently no support for custom Parameters in “Add Virtual User” Conversation Handling Activity.
Custom AI Services - Profile: Open Profile
☝️Current limitations, coming with a later update:
- The use of Extensions Tools (either Web Request or MCP Server) is currently mandatory in this profile. 💡This will be improved in a future update to allow for no tools to be used.
- For this profile there currently there are no custom exits in the “Add Virtual User” Conversation Handling Activity. You only have Success, Fallback, Failed, Idle timeout.
- There is currently no support for custom Parameters in “Add Virtual User” Conversation Handling Activity.