Azure WebJobs: ServiceBusTrigger

Article
1/17/2024

Introduction

In case we face a challenge to build a Web Job that listens or monitors a queue in the Microsoft Azure Service Bus within a certain namespace to process each message that is sent there by a message producer this article will provide you a walk-through how to accomplish that. The Web Job in this case acts as a message consumer of the messages on the queue. Below a high level diagram of a scenario that will be explained in this article and how to face the challenge.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Overview_zpsthzffacs.png
Picture 1. High Level Overview.

Steps to follow

The following steps demonstrate how to build, deploy and test the Web Job.

Step 1: Build Azure Web Job

We can build WebJobs inside Visual Studio by installing the WebJobs SDK. Once we have installed the SDK we have a template available to build a Web Job in C# or Visual Basic.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%201_zpssqourbww.png

Picture 2. Visual Studio Templates.

We can select this template, specify a name for the Web Job and click Ok. We will see that a program class will be created.

namespace ServiceBusMonitorWebJob
{
    // To learn more about Microsoft Azure WebJobs SDK, please see http://go.microsoft.com/fwlink/?LinkID=320976
    class Program
    {
        // Please set the following connection strings in app.config for this WebJob to run:
        // AzureWebJobsDashboard and AzureWebJobsStorage
        static void  Main()
        {
            var host = new  JobHost();
            // The following code ensures that the WebJob will be running continuously
            host.RunAndBlock();
        }
    }
}

And a Functions.cs.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
 
namespace WebJob1
{
    public class  Functions
    {
        // This function will get triggered/executed when a new message is written 
        // on an Azure Queue called queue.
        public static  void ProcessQueueMessage([QueueTrigger("queue")] string  message, TextWriter log)
        {
            log.WriteLine(message);
        }
    }
}

By default a method will be created for us to monitor or listen to an Azure Storage Queue, not Service Bus Queue! To have a method that will be triggered/executed when a message is written to an Azure Service Bus Queue we will need to have ServiceBusTrigger. This not available in the project and we will need to add the Microsoft.Azure.WebJobs.ServiceBus NuGet package.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%202_zpsne7lfzrj.png

Picture 3. Add NuGet Package Microsoft.Azure.WebJobs.ServiceBus.

Now a can change to ServiceBusTrigger in method ProcessQueueMessage and specify the queue I want to listen to.

public static  void ProcessQueueMessage([ServiceBusTrigger("inboundqueue")] BrokeredMessage message, TextWriter log)

Next change is changing the type of the message from string type to BrokeredMessage type. This type is not available in our class unless we add using statement for Microsoft.ServiceBus.Messaging. The package is already in the project, because it is part of the imported NuGet package. The TextWriter object can be used to write log statements that can be viewed in the AzureWebJob Dashboard.

When a message arrives on inboundqueue it will be picked up by Web Job and enter the ProcessQueueMessage method at runtime. Here I can extract the message body and send it for instance to Redis cache as a key value pair. To send it to Redis Cache I need to import another NuGet Package i.e. StackExchange.Redis (client library). Now the complete code for the functions class looks like below:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.ServiceBus.Messaging;
using StackExchange.Redis;
using System.Configuration;
 
namespace ServiceBusMonitorWebJob
{
    public class  Functions
    {
        private static  Lazy<ConnectionMultiplexer> lazyConnection = new Lazy<ConnectionMultiplexer>(() =>
        {
            return ConnectionMultiplexer.Connect(ConfigurationManager.AppSettings["RedisConnectionString"]);
        });
 
        public static  ConnectionMultiplexer Connection
        {
            get
            {
                return lazyConnection.Value;
            }
        }
 
        /// <summary>
        /// Method to Add Key/Value pair to Redis Cache
        /// </summary>
        /// <param name="key">Key as string</param>
        /// <param name="value">Value as string</param>
        static void  AddToRedis(string  key, string  value)
        {
            IDatabase cache = Connection.GetDatabase();
 
            cache.StringSet(key, value);
        }
 
        // This function will get triggered/executed when a new message is written 
        // on an Azure Queue called queue.
        public static  void ProcessQueueMessage([ServiceBusTrigger("inboundqueue")] BrokeredMessage message, TextWriter log)
        {
            //Log
            log.WriteLine("Message picked up from inboundqueue : " + message.MessageId);
 
            //Retrieve the message body regardless of the content as a stream
            Stream stream = message.GetBody<Stream>();
            StreamReader reader = new  StreamReader(stream);
            string s = reader.ReadToEnd();
 
            //Log
            log.WriteLine("Message Body : " + s);
 
            //Add to Redis Cache
            AddToRedis(message.MessageId, s);
        }
    }
}

Before the Web Job can be deployed to a WebApp a few configuration settings have to be done in the app.config. The connection strings for the AzureWebJobsDashboard and AzureWebJobsStorage need to be provided in the connectionStrings Section. These are required to view the log in Azure i.e. AzureWebJobsDashboard. The connection string that needs to be specified is the connection string to an Azure Storage account. The format is as follows:

DefaultEndpointsProtocol=https;AccountName=[Storage Account Name];AccountKey=[Access Key]

The other connection string that has to be provided is for the AzureWebJobsServiceBus. Format is:

Endpoint=sb://namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[Access Key]

Finally in the appSettings section the connection string for the Redis needs to be specified. Connection string has the format of:

namespace.redis.cache.windows.net,abortConnect=false,ssl=true,password=

Once configuration is done the app.config will look like:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <connectionStrings>
    <add name="AzureWebJobsDashboard" connectionString="DefaultEndpointsProtocol=https;AccountName=mijnnuon2;AccountKey=xxxxxxxxxxxxxxxxxxxxxxxx" />
    <add name="AzureWebJobsStorage" connectionString="DefaultEndpointsProtocol=https;AccountName=mijnnuon2;AccountKey=yyyyyyyyyyyyyy" />
    <add name="AzureWebJobsServiceBus" connectionString="Endpoint=sb://dev-mijnnuon-servicebus.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=zzzzzzzzzzzzzzzzzzzzz"/>
  </connectionStrings>
    <startup> 
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.WindowsAzure.Storage" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-4.2.1.0" newVersion="4.2.1.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-6.0.0.0" newVersion="6.0.0.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.ServiceBus" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-2.7.0.0" newVersion="2.7.0.0" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
  <system.serviceModel>
    <extensions>
      <!-- In this extension section we are introducing all known service bus extensions. User can remove the ones they don't need. -->
      <behaviorExtensions>
        <add name="connectionStatusBehavior" type="Microsoft.ServiceBus.Configuration.ConnectionStatusElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="transportClientEndpointBehavior" type="Microsoft.ServiceBus.Configuration.TransportClientEndpointBehaviorElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="serviceRegistrySettings" type="Microsoft.ServiceBus.Configuration.ServiceRegistrySettingsElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      </behaviorExtensions>
      <bindingElementExtensions>
        <add name="netMessagingTransport" type="Microsoft.ServiceBus.Messaging.Configuration.NetMessagingTransportExtensionElement, Microsoft.ServiceBus,  Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="tcpRelayTransport" type="Microsoft.ServiceBus.Configuration.TcpRelayTransportElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="httpRelayTransport" type="Microsoft.ServiceBus.Configuration.HttpRelayTransportElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="httpsRelayTransport" type="Microsoft.ServiceBus.Configuration.HttpsRelayTransportElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="onewayRelayTransport" type="Microsoft.ServiceBus.Configuration.RelayedOnewayTransportElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      </bindingElementExtensions>
      <bindingExtensions>
        <add name="basicHttpRelayBinding" type="Microsoft.ServiceBus.Configuration.BasicHttpRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="webHttpRelayBinding" type="Microsoft.ServiceBus.Configuration.WebHttpRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="ws2007HttpRelayBinding" type="Microsoft.ServiceBus.Configuration.WS2007HttpRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="netTcpRelayBinding" type="Microsoft.ServiceBus.Configuration.NetTcpRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="netOnewayRelayBinding" type="Microsoft.ServiceBus.Configuration.NetOnewayRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="netEventRelayBinding" type="Microsoft.ServiceBus.Configuration.NetEventRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="netMessagingBinding" type="Microsoft.ServiceBus.Messaging.Configuration.NetMessagingBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      </bindingExtensions>
    </extensions>
  </system.serviceModel>
  <appSettings>
    <add key="RedisConnectionString" value="mijnnuon.redis.cache.windows.net,abortConnect=false,ssl=true,password="/>
  </appSettings>
</configuration>

Step 2: Deploy Azure Web Job

Before the deployment (Publish to Azure) of the Web Job can be done a configuration setting in Web App has to be done to enable AzureWebJobDashboard.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%204_zpsxfzva9ab.png
Picture 4. Web App Settings, add the AzureWebJobsDashboard connection string.

This is an important step. We have to specify the name AzureWebJobsServiceBus, the connection string in format:

*DefaultEndpointsProtocol=https;AccountName=[Storage Account Name];AccountKey=[Access Key]
*
and choose type custom. In case we forget this than observing the Web Job logs will result in the following error:

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Code%206_zpsafnh3x9f.png

Picture 5. Error message when not configuring the connection string in the Web App.

Now the Web Job can be deployed via Visual Studio to a WebApp. Right click on the project and choose Publish as Azure WebJob...

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%203_zpsusneu2oy.png

Picture 6. Publish the Web Job to Azure in Visual Studio.

We will see a Publish Web dialog and here we import the publishing setting from WebApp. These settings can be downloaded from Azure Portal.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%206_zps3bjc5cci.png

Picture 7. Import the publishing settings of the Web App.

Next we can click OK and we will go to the next section of the dialog i.e. Connection.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%207_zpskzykcmp0.png

Picture 8. Validate connection to Web App.

Click Validate Connection to see if connection info is correct. When valid we can click Publish. Now the Web Job will be published to Web App. In the output window of Visual Studio we will see that deployment went successful.

Step 3: Test the Azure Web Job

In the Azure Portal we can see the Web Job in the Web App.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%208_zpsiowva9qt.png

Picture 9. Web Job inside the Web App.

When we click on the logs URL we will be redirected to the Microsoft Azure Web Jobs portal.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/WebJob%20details_zpsttz3frjo.png

Picture 10. Continuous WebJob details.

Nothing much has happened so far, only that the Job has started. In case I send a message to the queue using for instance ServiceBus Explorer, will see some action. Send a message via the ServiceBus Explorer to the queue.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%209_zps4hkspygp.png

Picture 11. Send Message to Service Bus Queue inboundqueue.

Refresh the AzureWeb Job Portal and a new entry is available.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%2010_zps1t4ck4pb.png

Picture 12. Invoke Function entry in Microsoft Azure WebJobs portal.

Once we click on the Functions.ProcessQueueMessage we examine the logs by click Toggle OutPut.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%2011_zpsuspjz3kl.png

Picture 13. Web Job log entries.

To explore what is in my REDIS cache we need to navigate to the service in the Azure Portal and open a console. Enter GET and messagid.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%2012_zpsw1wikuvc.png

http://i208.photobucket.com/albums/bb152/Steef-Jan/WebJobs/Azure%20WebJob%2013_zpsw4dlspg6.png

Picture 14. REDIS Cache Output in Console.

As we can see the message is now in the Cache.

Wrap up

It took me some time to get the into ServiceBusTrigger details. After some digging around I was able to get the ServiceBusTrigger working and see its behavior through the Azure Web Job Portal. The trigger is not limited to queues as it will also work for Service Bus Topic/Subscription. The signature of the method would look like:

public static  void ProcessQueueMessage([ServiceBusTrigger(Topic, Subscription)] BrokeredMessage message, TextWriter log)

Share via