Monitoring Dynamics 365 front to backend activities with Azure Application Insights
*Editor's note: The following post was written by Business Applications MVP Clément Olivier as part of our Technical Tuesday series. *
Azure application insights is a very powerful tool. You can access it for free from your azure subscription in order to track what's happening on your application. This allows you to do analysis over time by checking the different logs, events or even exceptions which occurred on your apps.
I kind of love this tool because it provides a central place for all your tracking, and allows you to create dashboards which give a nice overview of what's going on.
Since version 8.2, Dynamics 365 has become more integrated with Azure tools. With this version, you can send your CRM data into Azure Service Bus part 1 and part 2 in several clicks.
But here we will focus on the integration with Application Insights. There are 2 different integrations; client-side integration and server-side integration .
What we will see in this article:
- Client-side integration.
- Server-side integration for out of the box CRM actions.
- Server-side integration for Plugins exception handling.
Pre-Requisites:
Client-side integration can be done on any CRM version, with Online and On-Premise. Server-side integration needs a Dynamics 365 online instance running with the version 9.X for our use case.
Assumptions: You have an azure function and an application insights created, in order to perform the following steps.
1. Client-side integration
When you configure an application insights in Azure, a JavaScript code will be provided to allow you to track everything which is happening on your CRM Forms (JavaScript events, exceptions, errors while loading a page, etc.). You can find the how to on the docs.microsoft.com or here by Dilip Kumar.
This is a good opportunity to improve your code in order to speed up page loading or code logic for example. The drawback here is that you must manually add the Application Insights JavaScript code on each form you want to track. You can't add it to list view pages or any other pages which don't add custom JavaScript code.
2. Server-side integration for out of the box CRM actions
Here is a scheme of what we will do with "almost" no code:
[caption id="attachment_30825" align="alignleft" width="831"]
Figure 1: Server Side Integration for out of the box Crm events[/caption]
So why use an Azure Function here?
Using ILMerge to integrate the Application Insights assembly directly inside your Plugin assembly is unfortunately not supported by Microsoft.
"8/31/15 : This post has been edited to reflect that Dynamics CRM does not support ILMerge. It isn't blocked, but it isn't supported, as an option for referencing custom assemblies."
So let's move on, and focus on the process described above.
First step: configuring the CRM Side
As mentioned in the prerequisites, with the version 9.x, there is a new option with the plugin registration tool: Register New Web Hook
[caption id="attachment_30835" align="alignleft" width="350"]
Figure 2: Webhooks configuration in plugin registration tool.[/caption]
This will be the clue for the first step of our process. Once you have created your azure functions in the Azure portal, you will get a link like: https://YourAzureFunctionsNamespace.azurewebsites.net/api/FunctionName?code=somecharactershere. This link contains all necessary pieces of information to create our webhook from the Plugin Registration Tool.
Example:
- Name: MyWebhook (name of your webhook in your PRT)
- Endpoint URL: https://AzureFunctionNamespace.azurewebsites.net/api/FunctionName
- Authentication: WebhookKey
- Value: somecharactershere (everything which is after the parameter: ?code=)
After the creation of the webhook, you have the possibility to create steps (like for a regular plugin step) in order to send data to your azure functions whhich will process this data. /!\ The step should be running asynchronously to avoid freezing your CRM (we never know !)
Second step: configuring the Azure Functions side
By default, when you create an azure function with the httpTrigger, you will have something like:
using System.Net;
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
log.Info("C# HTTP trigger function processed a request.");
// parse query parameter
string name = req.GetQueryNameValuePairs()
.FirstOrDefault(q => string.Compare(q.Key, "name", true) == 0)
.Value;
if (name == null)
{
// Get request body
dynamic data = await req.Content.ReadAsAsync<object>();
name = data?.name;
}
return name == null
? req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a name on the query string or in the request body")
: req.CreateResponse(HttpStatusCode.OK, "Hello " + name);
}
We will perform few changes here to handle our CRM data properly.
a. Azure functions configuration
Making sure we integrate the Application Insights assembly. In order to do that, we will create a file called "project.json" in the functions :
[caption id="attachment_30845" align="alignleft" width="828"]
Figure 3 : Azure function project.json configuration[/caption]
- Open your Azure functions
- Click on the right pane to "View files"
- Click the Add button
- Call the file: "project.json"
- Copy the following:
{
"frameworks": {
"net46":{
"dependencies": {
"Microsoft.ApplicationInsights": "2.4.0" }
}
}
}
This will allow you to have access to the Microsoft.ApplicationInsights assembly within the Functions.
b. Azure Functions code modifications
We will now modify the code inside the functions in order to process the CRM data and basically send it to the Application Insights.
/*
* Created by: Clement Olivier
* Date: 05/14/2018
*/
using System.Net;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
try
{
log.Info("CRM2InsightsOOTB function processed a request.");
// Defining the telemetry details
TelemetryClient telemetryClient = new TelemetryClient();
telemetryClient.InstrumentationKey = "YourApplicationInsightsTelemetryKey";
telemetryClient.Context.Operation.Id = Guid.NewGuid().ToString();
// receiving the data from the CRM.
dynamic data = await req.Content.ReadAsAsync<object>();
var dataToString = Convert.ToString(data);
var evt = new EventTelemetry($"Data received from CRM");
var properties = new Dictionary<string, string>() {
{ "DataCRM", dataToString }
};
telemetryClient.TrackEvent(evt.Name, properties);
log.Info("Sent To Insights");
}
catch (Exception ex)
{
log.Error($"Error : {ex.Message}");
return req.CreateResponse(HttpStatusCode.BadRequest, ex.Message);
}
return req.CreateResponse(HttpStatusCode.OK, "Sent To Insights");
}
Note that we defined an extra property called "DataCRM" here which will contain the data coming from the CRM and passed as a parameter to the TrackEvent function of the Telemetry object.
Warning: Your Azure Functions must return a HttpStatusCode otherwise, or your plugin will fail.
The result will looks like this from your Azure functions:
[caption id="attachment_30855" align="alignleft" width="821"]
Figure 4: Results from Azure Function for out of the box Crm events [/caption]
The result will look like this from your Application Insights:
[caption id="attachment_30885" align="alignleft" width="815"]
Figure 5: Result from raw Plugin execution context in Azure Application Insights[/caption]
You can see on the screenshot above that we have the entire data stack coming from the PluginExecutionContext with the modified/created data on the CRM side.
c. Do what you want with this useful data!
Of course you can process this data to extract specific pieces of information and process it on Insights side to perform reports or other things.
3. Server-side integration for Plugins exception handling.
Here is what we want to achieve this time:
[caption id="attachment_30875" align="alignleft" width="821"]
Figure 6: CRM plugin class to Azure function and Azure Application Insights[/caption]
Let's say that this is the interesting part, at least from my point of view!
You might disagree because there is an entity dedicated to the plugins actions inside the CRM to have an overview of what's going on in your instance. That's true, however Application Insights is way easier to use to group/filter/analyze those exceptions. You can really customize the rendering based on your needs, and choose what will be sent to Application Insights to improve a proactive way the functionalities you are providing with your custom code.
For this part, we will use the same system as the point above with the OOTB functionality and will perform slight changes to handle the exceptions "only".
Scenario here: We want to catch all Plugin exceptions and send the following data to Application Insights:
- EntityName
- Plugin action
- Exception Message
- Exception StackTrace
(of course, you can add whatever you want more!)
First step: Plugin Assembly side
We will override the Plugin class which will implement the IPlugin interface. Inside the Execute function, we will add a catch which is specific to the Plugin errors between the try and catch (FaultException<OrganizationServiceFault> e)
catch (InvalidPluginExecutionException ex)
{
try
{
IServiceEndpointNotificationService cloudService = (IServiceEndpointNotificationService)serviceProvider.GetService(typeof(IServiceEndpointNotificationService));
if (cloudService == null)
throw new InvalidPluginExecutionException("Failed to retrieve the service bus service.");
localcontext.PluginExecutionContext.SharedVariables.Add("EntityName", localcontext.PluginExecutionContext.PrimaryEntityName);
localcontext.PluginExecutionContext.SharedVariables.Add("PluginAction", localcontext.PluginExecutionContext.MessageName);
localcontext.PluginExecutionContext.SharedVariables.Add("ExceptionMessage", ex.Message);
localcontext.PluginExecutionContext.SharedVariables.Add("ExceptionStackTrace", ex.StackTrace);
string response = cloudService.Execute(new EntityReference("serviceendpoint", new Guid("ServiceEndPointGUID")), localcontext.PluginExecutionContext);
}
catch {}
throw;
}
A bit of explanation here for the code above : We are using the Azure-aware plugin to send the PluginContext to Azure (in our case the Azure Functions), the docs is here (thanks Radu for the hint here).
To retrieve your ServiceEndpointGuid created from the PluginRegistrationTool earlier, you can go to your WebApi: https://tenant.crmX.dynamics.com/api/data/v9.0/serviceendpoints?$select=serviceendpointid,name and find the right ServiceEndPoint based on the name you provided and keep the serviceendpointid.
Another interesting point is that if you take a closer look on the code above I'm using the SharedVariables property to send my "custom" data because from the standard perspective, we don't have the exception pieces of information in the PluginExecutionContext which we are sending.
Second step: Azure Functions code
We have to customize a bit more to the code to handle the new stuff here. Regarding the configuration, we will just add the package "Newtonsoft.Json": "11.0.2" in the dependencies of the project.json in your Azure Functions.
Now, we will dive into the code:
/*
* Created by : Clement Olivier
* Date : 05/14/2018
*/
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
using Newtonsoft.Json.Linq;
using System.Net;
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, ExecutionContext context, TraceWriter log)
{
try
{
log.Info("C# ExceptionToInsightsAzure function processed a request.");
//Preparing the telemetry object
TelemetryClient telemetryClient = new TelemetryClient();
telemetryClient.InstrumentationKey = "APPINSIGHTS_INSTRUMENTATIONKEY";
telemetryClient.Context.Operation.Id = Guid.NewGuid().ToString();
dynamic data = await req.Content.ReadAsAsync<object>();
//We transform data to be able to query it
JObject dataToParse = JObject.Parse(Convert.ToString(data));
string inputParameter = Convert.ToString(dataToParse.SelectToken("InputParameters"));
var sharedVariablesData = dataToParse.SelectToken("SharedVariables");
// preparing the specific pieces of information we are looking for
string PluginAction = null;
string ExceptionMessage = null;
string EntityName = null;
string ExceptionStackTrace = null;
string sharedVariables = null;
Dictionary<string, string> sharedVariablesDic = new Dictionary<string, string>();
if (sharedVariablesData != null)
{
// Getting the keys info
foreach (var sharedVariable in sharedVariablesData){
sharedVariablesDic.Add(sharedVariable.SelectToken("key").ToString(), sharedVariable.SelectToken("value").ToString());
}
sharedVariables = Convert.ToString(sharedVariablesData);
PluginAction = sharedVariablesDic["PluginAction"];
log.Info($"PluginAction : {PluginAction}");
ExceptionMessage = sharedVariablesDic["ExceptionMessage"];
log.Info($"ExceptionMessage : {ExceptionMessage}");
ExceptionStackTrace = sharedVariablesDic["ExceptionStackTrace"];
log.Info($"ExceptionStackTrace : {ExceptionStackTrace}");
EntityName = sharedVariablesDic["EntityName"];
log.Info($"EntityName : {EntityName}");
}
var dataToString = Convert.ToString(data);
var evt = new EventTelemetry($"Exception in CRM plugin for entity {EntityName} ({PluginAction})");
var properties = new Dictionary<string, string>() {
{ "InputParameters", inputParameter },
{ "SharedVariables", sharedVariables },
{ "PluginAction", PluginAction },
{ "ExceptionMessage", ExceptionMessage },
{ "ExceptionStackTrace", ExceptionStackTrace },
{ "EntityName", EntityName }
};
telemetryClient.TrackEvent(evt.Name, properties);
}
catch (Exception ex)
{
log.Error($"Error : {ex.Message}");
return req.CreateResponse(HttpStatusCode.BadRequest, ex.Message);
}
return req.CreateResponse(HttpStatusCode.OK, "Sent To Insights");
}
You can see that we have a similar process here if we compare to the OOTB version. We simply need extra lines of code in order to retrieve the specific values from the ShareVariables node in the JSON received and use them.
The result in Application Insights will look like:
[caption id="attachment_30895" align="alignleft" width="500"]
Figure 7: Result from Plugin Exceptions in Azure Application Insights[/caption]
Once this is done, you can gather the data for your plugin exceptions and filter them by the custom properties we are sending from the Plugin Class. This will give you the possibility to spot the most common errors and fix them.
Next possible steps
We've seen how to improve your proactive CRM tracking using Application Insights for both the client side and the service side. What is described here is a sample which can be improved and completed based on your needs. And, create notifications and reports based on the data received.
Conclusion
Azure Application Insights is so powerful you can't miss it! Having a central place to track client or server side to improve your CRM instance can provide your end users with a useful tool.
I hope this helps. Happy CRM'ing!
notes:Full code for the plugin class is here, custom code starting line 197. Full code for the Azure Functions sending the exceptions is here.
Clément Olivier discovered Microsoft Dynamics CRM six years ago and loved it immediately. He likes to focus on the technical part of CRM and enjoys contributing to the community by writing articles and creating tools. Investigating and customizing is one of his favorite things to do especially with the CRM that offers infinite possibilities. You can reach Clément via Twitter @Clement0livier.