Use Case - Building a Nimbus Survey Agent in Copilot

Building a Copilot Agent for Nimbus which understands Customer feedback

In this Use Case, we're building a Survey Agent which understands Customer feedback and sends it to a SharePoint list together with key analytics such as topic, sentiment and detected suggestions for improvement.

Preconditions

To implement this Use Case you need:

  • For Nimbus: 
  • For Microsoft: 
    • A “Copilot Studio” Licence → Also see Azure Billing notes below.
    • Power Automate licensing needs, and all other Nimbus Power Automate Connector‍ preconditions must be met. 
      • A deployed AI model or Power Automate Premium license to use AI Builder.
      • Tenant Admin rights are required to implement all actions and connect to data sources required for your Services.
 

INC Azure Billing Cognitive Services

Disclaimer: Support for Azure Billing & Cognitive Services 

☝ The usage of Microsoft Azure AI Services for Speech1 and AI will cause additional costs outside your Nimbus subscription, which are exclusively determined by Microsoft.

Please note: 

  • Nimbus Support does not cover pricing discussions or make recommendations on Microsoft Azure Services. → Please inform yourself via the references below on pricing models.
  • Nimbus Support can not provide any technical help, insights or solutions on Azure Cognitive Service related technical problems.

1 Azure Speech Services: https://azure.microsoft.com/en-us/pricing/details/cognitive-services/speech-services/

2 Generative AI Services (Copilot Studio):

 

INC Icon Legend Accordion

Show Icon Legend 💡 = A hint to signal learnings, improvements or useful informati...

Show Icon Legend

💡 = A hint to signal learnings, improvements or useful information in context. 🔍 = Info points out essential notes or related page in context.
☝ = Notifies you about fallacies and tricky parts that help avoid problems. 🤔 = Asks and answers common questions and troubleshooting points.
❌ = Warns you of actions with irreversible / data-destructive consequence. ✅ = Intructs you to perform a certain (prerequired) action to complete a related step.
 
 

Designing the Copilot

✅Let's start by applying the best practices to our creation process by defining a scenario:

Scenario: 
The Copilot should conduct a service survey in a more natural way to gain better insight in customer experiences and automate form filling using Copilot. We want to replace Customer Input or Collect Information with DTMF tones to select options with a Virtual User for an enhanced customer experience when you need to collect information. The Copilot detects the satisfaction rating from the phrases rather than asking how many stars someone would give. It also automates the process of filling out survey forms. Natural language should encourage customers to express themselves and reflect about their experience.

Defining the Bot

Role The Copilot will greet the customer and process one message at a time.
Tasks
  • Greet
    • Be friendly and concise, you want customers to express their experience. Make them feel they are in a safe place. Explain that feedback is collected anonymously.
  • Listen & Understand
    • Process a customer feedback
      • Detect the satisfaction
      • Detect improvement suggestion
      • Classify it into a topic such as Agent Experience, Agent Friendliness, Service Speed, Accuracy, Data Privacy etc.
  • Automate the process of filling out the survey 
    • In our example we send the data to a SharePoint list
  • Exit
    • Once one single message is processed, send the exit signal to Nimbus to leave the conversation.
    • If a customer does not want to give feedback, exit
Access, checkpoints and thinking
  • The Copilot needs access to SharePoint list. 
  • As it only processes one message, it can terminate the conversation before sending the data to SharePoint.

Building the Copilot

✅The first step is to configure a new agent in Copilot. The agent will handle all the customer inputs via various topics, then parse the individual information into parameters.

Description Screenshot
  1. Head to Copilot Studio
  2. Creating a new (blank) Agent and describe it
  3. Disable Orchestration
  1. Add a tool to Copilot. The tool will use a Power Automate Flow to call an AI model and process the customer's input.

 

Tip: you can also replace this step by using the prompt tool in Copilot. It's up to you.

The Power Automate flow has 4 steps. which are detailed in the following.

The high level steps are:

  1. The trigger is called when the agent calls the flow using the customer's message.
  2. A HTTP request triggers your AI model to analyse the customer's message.
  3. The response JSON is parsed and…
  4. returned back to the Copilot agent.

Details on each step will be explained in the following.

  1. The flow trigger needs have a parameter defined which is the customer's input, type string
  2. Name it CustomerInput.
     

The HTTP element will request an AI model of choice - in this case ChatGPT. 

  • Fill in Request URI and Body as shown below
  • ☝Don't forget to add your api-key. 

Body:

{
 "messages": [
  {
   "role": "system",
   "content": "You are an AI assistant that extracts customer feedback form a message."
  },
  {
   "role": "user",
   "content": "Given the following message:\n\n@{triggerBody()?['text']}\n\nRespond with a valid JSON:\n{\n \"topic\": \"...\",\n \"sentiment\": \"...\",\n \"room_for_improvement\": \"...\"\n}"
  }
 ],
 "temperature": 0.2,
 "max_tokens": 200
}

 

Parse the output of the AI model:

body('HTTP')?['choices'][0]['message']['content']

Respond to the agent with the desired fields.

Once the Power Automate Flow is ready, the configuration of the tool is simple via the details:

  1. Add the flow as a new tool to Copilot
  2. Give it a name
  3. Leave everything else default in the configuration. 
    (See screenshot).
  1. Create a Custom Topic: Process JSON Input”.
  2. Pass along the TaskId for anonymous feedbacks. 
  3. Add the phone number and customer's display name or any other data for non-anonymous feedbacks.

JSON to process

{
"TaskId" : "$(CallId)",
"Phone": "+$(Caller.TelNumber)",
"Name": "$(Customer.DisplayName)"
}

To build the topic you can follow the instructions in Copilot - Initial Data Handover from Nimbus to Copilot 

 
At the end of the “Process JSON Input” topic, redirect to a new topic Gather Feedback”.
  1. Create a custom topic “Gather Feedback”.
    This topic is only redirected to by other topics.
  2. Add a Question to ask for feedback. Save response (Var1) as the input message for our AI Tool. 
  3. Store the outputs into global variables.
Then redirect to a new topic “Fill out the form
  1. Create the custom topic “Fill out the form”.
    This topic is only redirect to by other topics. 
  2. Send the exit message first, then create the list item in SharePoint using the values from the global variables.
  3. End this topic with “End conversation”.

You will also need a topic to greet the Customer. 

  1. Create a topic “Greeting” which handles a greeting 
  2. Add a redirect to the “Gather feedback” topic.

If your customer does not have any feedback, you need to handle this as well. 

  1. Add another topic “No Feedback Wanted” which handles the wish not to leave feedback.
  2. Leave a goodbye message and end the conversation.

 

Adding the Copilot to Nimbus

✅Finally, it is time to add your bot to Nimbus. The process consists of: 

  • Creating Authentication in Copilot Studio (Secret Keys)
  • A Nimbus Bot, to which you are “mapping” the API endpoint and authentication.
  • A Nimbus Bot Response Templates to map the bot JSON outputs to for further call routing.
  • A Virtual User which combines all of the above and can be added to your workflow.

Create Authentication

Copilot - Authentication

✅The following steps explain how to define your authentication in Copilot Studio so Nimbus can access the API later.

  1. Log into Copilot Studio.
  2. Security: Go to Agents > Settings > Security. 
    1. Select No authentication → See Design and Limitation Notes below.
    2. Go to Web Channel Security > Secrets and Tokens.
    3. 🧠Copy either Secret 1 or 2 for later within your Nimbus Bots configuration.
    4. Ensure “Require secured access” is enabled.
  3. Endpoint: Copilot Direct Line 3.0: https://directline.Botframework.com/v3/directline/conversations
    🧠Copy / Note the Endpoint URL for later use.

Configuring the Bot

✅The bot contains the API key and endpoint for your Copilot agent:

  1. Head to Nimbus Administration > Configuration > Bots and create a new Bot.
  2. Give your Bot a descriptive name
    💡This name is just for Nimbus UI purposes, and must not match Copilot. 
  3. Define the Organization Unit under which this Bot will be selectable. 
    ☝ Note that any service on this OU or higher can access it (which may result in additional Azure cost).
  4. Type is M365 Copilot Direct Line 3.0.
  5. API Key: 🧠 Paste the Secret from the Copilot Setup above.
  6. Endpoint: 🧠 https://directline.Botframework.com/v3/directline/conversations
  7. Save and Close.

Create a Response Template

✅The response template maps the outputs of your Copilot agent:

  1. In Nimbus, go to Admin > Configuration > Bot Response Templates 
  2. Add a new default response template for Copilot.

Create the Virtual User

✅The Virtual User is the element you add to your Nimbus workflow, which also marks the starting point of verbal interaction with the customer:

  1. In Nimbus, go to Admin > Configuration > Virtual Users.
  2. Add a new Virtual User with a Description and choose your bot accordingly.
  3. On the Initial Message to Bot the most important part is the System Parameter CallId. It corresponds to the Nimbus TaskID.

    💡The CallId is used to start the Copilot “Process JSON Input” Topic mentioned above, which will engage the Bot with a Customer. The next reply by the Customer is then handled exclusively Copilot itself.

     

Using the Copilot in Nimbus

✅The final step is to add your newly defined Virtual User to a workflow.

Use the Virtual User in your Workflow

  1. Within Nimbus go to Configuration > Workflows.
  2. Create a new Audio/Video Workflow or add your new “Add Virtual User” activity to an existing workflow. 
    1. Within the activity, you need to configure the Exit with the Regular Expression feedback to ensure that the Virtual User stops after processing the second input. 
      💡The wording “feedback” is adjusted in Copilot.

Testing your Virtual User

✅Test your Virtual User by verifying the results added to the SharePoint list.

Known Limitations

INC AI and Virtual User Limitations

AI-driven interactions

AI driven replies are not deterministic and depend highly on what the Customer is saying. For example, if the Customer is saying “I have trouble with my Skateboard” it is not necessarily given that the Bot will use the same word “Skateboard” as your exit unless specifically handled via Copilot. 

🔎Refer to our Best Practices - Virtual Users in Nimbus for some considerations and risks when using Virtual Users instead of Human interaction.

Microsoft Copilot Bot Limitations

  • Expect delays: Processing AI answers takes a few seconds for transcription of voice input, AI processing and then transcription back into a spoken reply. Luware is trying to minimize the delay on Nimbus call infrastructure, but the dependency on external APIs will always incur a delay. The Customer will hear silence during this processing and no audio feedback / silence detection.
  • Ensure you have no ambiguity in your topics. For instance, the word “Service” may be too generic if you want to transfer to different services. Rather use healthcare|medical|emergency as identifiers or use more complex Regular Expressions to identify replies.

Nimbus Technical Limitations

The current Nimbus implementation with AI-driven bots underlies the following limitations: 

  • Supported Modalities: Virtual Users (Bots) are currently available for Audio/Video modality tasks.
  • 3rd-Party API Support: Nimbus will be built flexibly to support “BYO” (bring your own) customer bot configurations and APIs. The first currently supported bot integration is M365 CoPilot Direct Line 3.0 APIs.
  • Authentication: The Copilot Direct Line 3.0 API Secret needs to be copy-pasted into the Nimbus Bots configuration. Further authentication types are planned to be supported in the near future.

Table of Contents