Alexa + Azure Functions + Microsoft Graph = a smarter assistant!

  • [アーティクル]

On this blog we have recently learned how to build an Alexa Skill using Microsoft tools and platforms, like C#, Visual Studio and Azure Functions. The skill we have built in the previous posts was available to every user: it didn't need to know who the user is asking the questions in order to return a proper feedback. However, there are some scenarios where this approach isn't good enoguh. Think, for example, to a skill to order food that can be delivered at home. The skill must know who is placing the order, to charge your credit card with the total amount; or to pick your home location as delivery address.

For these scenarios, Alexa provides a feature called account linking. When the user enables the skill, he will be requested to authenticate to a 3rd party service using an OAuth 2.0 flow. Once the linking is successful, every request to the skill will be authenticated against the service and Alexa will send you, in the JSON request, the access token which is required to perform further operations.

You can identify skills which require account linking thanks to the message displayed under the Enable button in the Alexa Store.

In this blog post we're going to see how to implement this flow by adding support to Azure Active Directory authentication and the Microsoft Graph to a skill. We'll welcome the user by saying his name and we'll be able to provide information about his OneDrive storage.

Let's start!

The account linking

To setup the account linking you need to select your skill on the Alexa Developer Console and choose Account linking from the menu on the left:

In the first part of the page you'll be able to customize the configuration of the account linking. In order to enable it, you have to turn on the first option Do you allow users to create an account or link to an existing account with you? . This will enable the message you have seen in the previous image in the Alexa Store.
The second option can be used if your skill offers some features which can be leveraged also without linking an account. For example, the food delivery skill could offer a command to know the restaurants around you which can be freely invoked; however, as soon as you try to order some food, you'll be asked to link your account.
In our scenario, we leave this option turned off. Our skill needs to use the Microsoft Graph, so we aren't able to provide any feature if the user hasn't login with his Microsoft Account.

Then make sure to check Auth Code Grant as authorization grant type, since it's the safest one.

The rest of the page is dedicated to configure the OAuth 2.0 flow, which will be triggered when the user enables the skill and authenticates with the 3rd party service. Let's take a step back and take a look at how the OAuth 2.0 flow works:

The process involves three steps:

  1. First, the client application (in our case, the skill) sends an authorization request to the specific endpoint exposed by the service. The user will see a popup or he will be redirected to a different page (in case of a web application), where he will be asked to login with his credentials. This page belongs to the owner of the service. This means that the client application (in our case, neither Amazon nor our skill) will never get access to the credentials of the user.
  2. If the authorization request is approved, the application can use the received grant to request an access token. The access token is a unique identifier of the user which, however, doesn't contain any information that can be used to identify his credentials.
  3. Once we have the access token, we can now finally access to all the protected resources exposed by the service. The access token will be included as authorization header of every request and, if it's valid, we will receive the resource we have asked.

If we take a look now at the section to configure the OAuth 2.0 flow, we will see how the various fields match the flow we have just highlighted:

  • Authorization URI: this is the endpoint exposed by the service to ask the permission to start using it. You send a request with all the details to identify your application and, if approved, you will receive back the authorization code.
  • Access Token URI: this is the endpoint to use as next step. After we have been authorized to use the service, we can use the authorization code to send a request to this URI to get back the access token.
  • Client ID is the unique identifier of our application.
  • Client secret is a password which has been provided by the owner of the service to authenticate our application.
  • Scope is a set of values which which identiy the features of the service we want to leverage.

In our scenario, we need to authenticate to the Microsoft Graph using a Microsoft Account or an Office 365 account. Where can we find all the information we need to fill the Account Linking page in the Amazon portal?
In order to leverage authentication with a 3d party service, we need to register our application against it. For this reason, all the 3rd party services which provide authentication (Microsoft, Google, Twitter, GitHub, etc.) offer a developer portal that can be used to register an application and to obtain all the credentials we need to implement the authentication flow.

The one offered by Microsoft to register an application which needs to support Azure AD authentication is available in the Azure Portal. As such, the first step is to login to https://portal.azure.com using the account which you want to use to register your application (Microsoft Account or Office 365).
Once you are logged into Azure, open the Azure Active Directory section and choose App registrations (Preview) :

Let's start to register our application. Click on the New registration button.

First give to your application a unique name. In my case, I chose MyAlexaSkill. Then you must specify which account types you want to support:

  • Accounts in the organizational directory only: if you're logged in with an Office 365 account, you will be able to create an application which is authorized to allow the login only for users from your organization. This approach is useful when you're building an internal app, which my be used only by people from your company.
  • Accounts in any organizational directory: this option will enable any Office 365 user to login with their account, regardless of the tenant they belong to.
  • Accounts in any organizational directory and personal Microsoft Account: this option will enable not only Office 365 users to login, but also users with a personal Microsoft Account.

In our scenario, let's enable the third option. We want to enable every user to use the skill to know how much space they have left on OneDrive, regardless if it's the personal space or the business one.

Ignore, for the moment, the Redirect URI section. We're going to fill it later. Press Register at the end of the section.

Now we can start to fill the various field in the Alexa Developer Portal. Let's see where to retrieve the various parameters:

  • Authorization URI and Access Token URI: we can find this information by clicking on the Endpoints button.

From there, we need to copy the two URIs under OAuth 2.0 authorization endpoint (v2) and OAuth 2.0 token endpoint (v2) .

  • Client Id can be retrieved in the main page under Application (client) ID:

  • Client Secret must be generated from the Azure portal. Click on Certificates & secrets in the left panel, then click on New client secret. Give it a description and choose an expiration date. In the context of a skill, you can safely choose Never. This way, you won't have to remember to update, from time to time, the client secret in the Alexa Developer Portal. Once the secret has been generated, copy it in the portal and store it also somewhere safe. The reason is that this is the only time you'll be able to see the client secret in clear. After that, it will be automatically masked and there won't be any way to see it again. You'll be forced to generate a new one, if you lose it.

  • Client Authentication Scheme: keep HTTP Basic.
  • Scope must match the various scopes that we have enabled in the Azure portal. You can see them by clicking on API permissions in the Azure app configuration. By default, the standard one is User.Read, which allows the application to read the basic information of the user profile, like his name or his mail address. If you want to add more scopes, you can click on Add a permission and explore all the other scopes that are offered by the Azure services or by your custom services. In this case, we're building a skill which integrates with the Microsoft Graph, so you'll find all the available permissions under Microsoft Graph:

Scopes are categorized as Delegated permissions and Application permissions. In our case, we want the skill to access to the API as the signed user, so we can focus on the first category. From there, you'll be able to explore all the available scopes and enable the ones you need. Pay attention that, for some of them, you may see the value Yes in the Admin consent required column. If you're building an enterprise skill which authenticates with an Office 365 account, the administrator of the tenant may block the usage of some scopes for security reasons. In such cases, you will need to reach him to ask to enable the proper permissions.

Once you have defined the scopes you need, you will need to copy them inside the Scope field of the Alexa Developer Portal. If you have multiple scopes, you can click on the + button to add new fields.

The last step requires the opposite process compared to what we have done so far. We'll need to take some info from the Alexa Developer Portal and copy them to the Azure one. In the OAuth 2.0 flow, in fact, once the authentication process is completed the service will forward the response to a specific endpoint exposed by our application. In this case, since it's an Alexa skill, the endpoints are provided directly by Amazon. We can find them at the end of the Account Linking configuration page:

We need to copy them in the Azure Portal, in the Authentication section:

Hit Save to complete the process, then move back to the Alexa Developer Portal and hit Save also there. We're done! Now account linking should have been properly enabled.

This is how the configuration of your Alexa skill should look like:

In order to test the linking, we need to open the Alexa web application, which you can find at https://alexa.amazon.com. Login with your account, move to the Skills section and click on the Your skills button on the top right.

Please note! I apologize if the following screenshots are in Italian, but it seems that there's no way to force Amazon to display the application in English if your account is based in Italy. However, they should be helpful to understand where to look for the various options.

You will find multiple categories, including one called Skills for developers. Here you will find all the skills you have created in the Alexa Developer Portal. Look for the one you're working on and notice how, below the name, you will see now a message informing that account linking is required.

Click on the skill and enable it. If you did everything correctly, another windows or tab of your browser will open up to display the Microsoft Account authentication page. Login with your Microsoft Account or Office 365 account and you should be all set! You should see a confirm message notifying you that your account is now linked to the skill.

Handling the authentication in the Azure Function

Now that we have completed the configuration on the portal, we need to start working on our backend, which is hosted by an Azure Function. We need, in fact, to retrieve the access token and use it to perform operations with the Microsoft Graph.
I won't explain from scratch how to build an Alexa Skill hosted by an Azure Function, since we already did it in the previous posts. Here we will focus only on the relevant snippets for handling the authentication process.

Thanks to Alexa.NET it's easy to retrieve the token, since it's stored in the Session.User.AccessToken property of the SkillRequest object, which is the one that maps the JSON coming from Alexa with all the details about the request:

 public static async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req, ILogger log)
{
    var json = await req.ReadAsStringAsync();
    var skillRequest = JsonConvert.DeserializeObject<SkillRequest>(json);

    var token = skillRequest.Session.User.AccessToken;
    
    ///handle the request and return the response
}

Now that we have the token, we can use it to perform operations against the Microsoft Graph. The Graph is nothing more than a set of REST endpoints, which you can call by sending HTTP commands. Based on the type of operation, it could be a GET, a POST, a PUT, etc. The easiest way to understand the opportunities offered by the Microsoft Graph is to use Graph Explorer, which is a sort of playground. You can login with your Microsoft Account or Office 365 account and then start making request to the APIs. The tool will offer you the opportunity to compose requests and to observe the JSON response, other than discovering all the available endpoints.

However, we are not required to perform manual HTTP requests in order to use the Graph. Microsoft, in fact, provides multiple SDKs which encapsulate all the logic to work with the Graph, similarly to how Alexa.NET abstracts all the JSON requests and responses and allows you to work with classes and objects. Of course, we have also a SDK for .NET, so we can easily integrate it in our Azure Function.

Right click on your project in Visual Studio, choose Manage NuGet packages and install the package called Microsoft.Graph.

The object that we can use to perform operations against the Graph is called GraphServiceClient, which is included in the Microsoft.Graph namespace. However, we can't use it as it is. We need to use an authenticated client, so we will need to supply the access token included in the request.

Here is a simple setup method that we can add to our Azure Function:

 public static GraphServiceClient GetAuthenticatedClientForUser(string token, ILogger logger)
{
   GraphServiceClient graphClient = null;

   // Create Microsoft Graph client.
   try
   {
       graphClient = new GraphServiceClient(
           "https://graph.microsoft.com/v1.0",
           new DelegateAuthenticationProvider(
               (requestMessage) =>
               {
                   requestMessage.Headers.Authorization = new AuthenticationHeaderValue("bearer", token);
                   return Task.CompletedTask;
               }));

       return graphClient;

   }
   catch (Exception exc)
   {
       logger.LogError(exc, "Could not create a graph client");
   }

   return graphClient;
}

We create a new instance of the GraphServiceClient object, passing as parameter:

  1. The endpoint we want to use for the Graph. We use the production one, which is https://graph.microsoft.com/v1.0 . There's also a preview one, which provides access to all the beta APIs, and it's available at https://graph.microsoft.com/beta .
  2. A DelegateAuthenticationProvider, which is the delegate used to handle the authentication. The delegate provides a reference to the HTTP request which is sent to the Microsoft Graph, that we need to customize by supplying the access token in the authorization header.

Now that we have an authenticated client, we can use it to perform operations with the Graph. For example, let's customize the welcome message of the skill by adding the name of the user. We do this in case the incoming request type is LaunchRequest:

 public static async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req, ILogger log)
{
   var json = await req.ReadAsStringAsync();
   var skillRequest = JsonConvert.DeserializeObject<SkillRequest>(json);

   // Verifies that the request is indeed coming from Alexa.
   var isValid = await skillRequest.ValidateRequestAsync(req, log);
   if (!isValid)
   {
       return new BadRequestResult();
   }

   var request = skillRequest.Request;
   SkillResponse response = null;

   var token = skillRequest.Session.User.AccessToken;
   var client = GetAuthenticatedClientForUser(token, log);

   if (request is LaunchRequest launchRequest)
   {

       var me = await client.Me.Request().GetAsync();

       var welcomeMessage = $"Welcome {me.DisplayName}";
       response = ResponseBuilder.Tell(welcomeMessage);
   }
   
    return new OkObjectResult(response);
}

The client simply maps all the endpoints that you can see in the Graph Explorer as objects. As such, in order to get the basic profile of the user (which is mapped with the endpoint https://graph.microsoft.com/v1.0/me/), we can use the Me property exposed by the client. To actually perform the request we need to call the Request() method, followed by the HTTP method we need to use. In this case, the me endpoint replies to a HTTP GET, so we call the GetAsync() method.
We get in return an object which maps all the properties that we can see in the response JSON in Graph Explorer. For our scenario, we retrieve the name using the DisplayName property and we add it to the welcome message.
The rest of the code is the same we've seen in the other posts: we use the ResponseBuilder class to create a response and then we send it back as response to the Alexa service which invoked our function.

Now we that we have seen how to handle the authentication in the backend, the sky is the limit 😃 For example, if we follow our original idea of integrating OneDrive with our Alexa skill, we can create a new intent in the Interaction Model with name Quota and then, in the function, handle it with the help of the Graph client:

 if (intentRequest.Intent.Name == "Quota")
{
    var drive = await graphClient.Me.Drive.Request().GetAsync();
    int free = (int)(drive.Quota.Remaining.Value / 1024 / 1024 / 1024);

    var quotaMessage = $"You have {free.ToString()} GB available";
    response = ResponseBuilder.Tell(quotaMessage);
}

The structure of the code is the same as the previous example. If we use Graph Explorer, we can see how the information about the OneDrive storage of the user is available through the https://graph.microsoft.com/v1.0/me/drive endpoint. This endpoint is mapped by the property Me.Drive, which returns a Drive object that contains all the properties of the storage. Thanks to the Quota property we can get the information we're looking for, which is the available space, and we use it to build a response for the user.

Wrapping up

In this post we have seen how to enable account linking for an Alexa Skill, allowing it to identify the user and to provide customized responses. In this specific example we have leveraged AAD authentication and the Microsoft Graph, but any service which supports OAuth 2.0 for the authentication flow is supported. You could integrate your Alexa skill, for example, with Twitter, GitHub, Facebook, etc.

Thanks to the OAuth 2.0 configuration provided in the Alexa Developer Portal, you don't have to handle the authentication flow on your own, by manually communicating with the various authentication and access token endpoints. You just have to provide to Amazon the configuration of your service and, automatically, you will receive the access token in the body of the request which is delivered to your skill.

You can find the full OneDrive sample on GitHub at https://github.com/Microsoft/Windows-AppConsult-samples-PWA/tree/master/OneDriveAlexaSkill. Of course, you will find only the code of the backend Azure Function. It's up to you to create the most appropriate interaction model and to setup an Azure AD application to support the authentication.

A huge thanks to my friend Marco Minerva, who is sharing with me lot of interesting activities around Alexa skills and who helped me in setting up Account Linking.

Happy coding!

Comments

  • Anonymous
    March 06, 2019
    Great post Matteo!For some reason, I followed all the steps (correctly I believe), but when I try to link the account, it takes me to the Office 365 sign in and after signing in I get the following message in the alexa portal: "We were unable to link SKILL NAME at this time."I did a small research and it appears that there's more people in this situation and some reference the incapacity to link with an O365 account. Did you use a personal one or O365?Error thread: https://social.msdn.microsoft.com/Forums/en-US/58918c68-182e-4b3c-9b70-fd6e8920e8cb/problems-linking-alexa-skill-to-microsoft-app-registration?forum=WindowsAzureAD
    • Anonymous
      March 07, 2019
      In my case I registered the AD app on my personal Office 365 account, but I've logged in with the personal one. I'll give it a try with the Office 365 and I'll let you know!