This ASP.Net web application provides a web service hosting the Semantic Kernel, enabling secure and modular access to its features for the Copilot Chat application without embedding kernel code and settings, while allowing user interfaces to be developed using frontend frameworks such as React and Angular.
Before you get started, make sure you have the following requirements in place:
./appsettings.json to configure your Azure OpenAI resource or OpenAI account.dotnet dev-certs https --trust
Select
Yeswhen asked if you want to install this certificate.
dotnet dev-certs https
To verify the certificate has been installed and trusted, run
dotnet run dev-certs https --check
To clean your system of the developer certificate, run
dotnet run dev-certs https --clean
eng.traineddata and add them to your ./data directory or the location specified in the Tesseract.FilePath location in ./appsettings.json.Copy to Output Directory value to Copy if newer.You can start the WebApi service using the command-line, Visual Studio Code, or Visual Studio.
cd semantic-kernel/samples/apps/copilot-chat-app/webapi
dotnet build
dotnet run
info: Microsoft.SemanticKernel.Kernel[0]
Health probe: https://localhost:40443/healthz
semantic-kernel/dotnet/SK-dotnet.sln).samples folder.CopilotChatWebApi and select Set as Startup Project.F5 or selecting the menu item Debug->Start Debugging.If you want to use SequentialPlanner (multi-step) instead ActionPlanner (single-step), we recommend using gpt-4 or gpt-3.5-turbo as the planner model. Using gpt-3.5-turbo will require with a relevancy filter.
To enable sequential planner,
"Type": "Sequential" under the Planner section.gpt-4 or gpt-3.5-turbo) under the AIService configuration section.
gpt-4, no other changes are required.gpt-3.5-turbo: change CopilotChatPlanner.cs to initialize SequentialPlanner with a RelevancyThreshold*.
using statement to top of file:
using Microsoft.SemanticKernel.Planning.Sequential;
CreatePlanAsync method should return the following line if this._plannerOptions?.Type == "Sequential" is true:
return new SequentialPlanner(this.Kernel, new SequentialPlannerConfig { RelevancyThreshold = 0.75 }).CreatePlanAsync(goal);
* The RelevancyThreshold is a number from 0 to 1 that represents how similar a goal is to a function’s name/description/inputs. You want to tune that value when using SequentialPlanner to help keep things scoped while not missing on on things that are relevant or including too many things that really aren’t. 0.75 is an arbitrary threshold and we recommend developers play around with this number to see what best fits their scenarios.
webapi - Copilot Chat should be now running locally with SequentialPlanner.By default, the service uses an in-memory volatile memory store that, when the service stops or restarts, forgets all memories. Qdrant is a persistent scalable vector search engine that can be deployed locally in a container or at-scale in the cloud.
To enable the Qdrant memory store, you must first deploy Qdrant locally and then configure the Copilot Chat API service to use it.
Before you get started, make sure you have the following additional requirements in place:
docker pull qdrant/qdrant
Change directory to this repo and create a ./data/qdrant directory to use as persistent storage.
Then start the Qdrant container on port 6333 using the ./data/qdrant folder as the persistent storage location.
cd /src/semantic-kernel
mkdir ./data/qdrant
docker run --name copilotchat -p 6333:6333 -v "$(pwd)/data/qdrant:/qdrant/storage" qdrant/qdrant
To stop the container, in another terminal window run
docker container stop copilotchat; docker container rm copilotchat;.
Enabling telemetry on CopilotChatApi allows you to capture data about requests to and from the API, allowing you to monitor the deployment and monitor how the application is being used.
To use Application Insights, first create an instance in your Azure subscription that you can use for this purpose.
On the resource overview page, in the top right use the copy button to copy the Connection String and paste this into the APPLICATIONINSIGHTS_CONNECTION_STRING setting as either a appsettings value, or add it as a secret.
In addition to this there are some custom events that can inform you how users are using the service such as SkillFunction.
To access these custom events the suggested method is to use Azure Data Explorer (ADX). To access data from Application Insights in ADX, create a new dashboard and add a new Data Source (use the ellipsis dropdown in the top right).
In the Cluster URI use the following link: https://ade.applicationinsights.io/subscriptions/<Your subscription Id>. The subscription id is shown on the resource page for your Applications Insights instance. You can then select the Database for the Application Insights resource.
For more info see Query data in Azure Monitor using Azure Data Explorer.
CopilotChat specific events are in a table called customEvents.
For example to see the most recent 100 skill function invocations:
customEvents
| where timestamp between (_startTime .. _endTime)
| where name == "SkillFunction"
| extend skill = tostring(customDimensions.skillName)
| extend function = tostring(customDimensions.functionName)
| extend success = tobool(customDimensions.success)
| extend userId = tostring(customDimensions.userId)
| extend environment = tostring(customDimensions.AspNetCoreEnvironment)
| extend skillFunction = strcat(skill, '/', function)
| project timestamp, skillFunction, success, userId, environment
| order by timestamp desc
| limit 100
Or to report the success rate of skill functions against environments, you can first add a parameter to the dashboard to filter the environment.
You can use this query to show the environments available by adding the Source as this Query:
customEvents
| where timestamp between (['_startTime'] .. ['_endTime']) // Time range filtering
| extend environment = tostring(customDimensions.AspNetCoreEnvironment)
| distinct environment
Name the variable _environment, select Multiple Selection and tick Add empty "Select all" value. Finally Select all as the Default value.
You can then query the success rate with this query:
customEvents
| where timestamp between (_startTime .. _endTime)
| where name == "SkillFunction"
| extend skill = tostring(customDimensions.skillName)
| extend function = tostring(customDimensions.functionName)
| extend success = tobool(customDimensions.success)
| extend environment = tostring(customDimensions.AspNetCoreEnvironment)
| extend skillFunction = strcat(skill, '/', function)
| summarize Total=count(), Success=countif(success) by skillFunction, environment
| project skillFunction, SuccessPercentage = 100.0 * Success/Total, environment
| order by SuccessPercentage asc
You may wish to use the Visual tab to turn on conditional formatting to highlight low success rates or render it as a chart.
Finally you could render this data over time with a query like this:
customEvents
| where timestamp between (_startTime .. _endTime)
| where name == "SkillFunction"
| extend skill = tostring(customDimensions.skillName)
| extend function = tostring(customDimensions.functionName)
| extend success = tobool(customDimensions.success)
| extend environment = tostring(customDimensions.AspNetCoreEnvironment)
| extend skillFunction = strcat(skill, '/', function)
| summarize Total=count(), Success=countif(success) by skillFunction, environment, bin(timestamp,1m)
| project skillFunction, SuccessPercentage = 100.0 * Success/Total, environment, timestamp
| order by timestamp asc
Then use a Time chart on the Visual tab.