Enforcing Ordered Delivery using Azure Logic Apps and Service Bus
Background
When consuming messages from an Azure service bus the order may not be guaranteed due to the brokered based messaging scheme where multiple consumers can consume messages from the bus. Sure we can force the Logic App to execute as a single instance but then we sacrifice performance and scalability. We can also use ReceiveAndDelete but then we loose the transactional nature of the bus. Ultimately to ensure a message is consumed in the correct order using the transactional nature of the bus we would add a sequence number to each message and use this to enforce the ordering.
Solution design
To achieve ordered delivery using Logic Apps, we would need to ensure all related messages are consumed by the same Logic App instance and for this, we use the session Id property on the service bus. Below is the full workflow process to force ordered delivery using Logic Apps and session Id’s on the service bus subscription.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb11.png?w=626&h=566
Building the solution
This scenario is based on a financial institution which requires all monetary transfers to be processed in an ordered fashion. The key is choosing a suitable session identifier and with this in mind, the account number was the most suitable candidate as we want a single consumer to process all the transactions for a particular account number.
Here we have created a subscription for a topic called AccountTransfers. Note the Enabled sessions is checked.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb1.png?w=318&h=528
Once the service bus has been configured, we can now dissect the workflow to see how we can achieve ordered delivery.
The workflow is initiated by a pooling Service Bus Connector. The properties of this connector are shown below. The key point here is to set the Session id to “Next Available”. This forces the Logic App to create a new instance for each unique session id value found on the service bus.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb2.png?w=582&h=323
The next action “ProcessSBMessage” is used to call another logic app which does the processing of the message found on the bus. Here I am just passing the raw base64 encoded message from the Service Bus Trigger action. Using the pattern “separation of concerns” moves the business logic away from the process of ensuring ordered delivery.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb12.png?w=579&h=285
Once the message has been sent to the chained Logic App and a response has been returned, we can complete the message from the bus with the following action.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb4.png?w=579&h=287
Next, we go into a loop until the exit condition has been satisfied. I am going to use a counter that is incremented if no messages are found on the service bus. If no more messages are found on the service bus after 30 seconds, the loop will exit.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb5.png?w=619&h=413
The loop inside starts with another service bus connector trigger which gets the messages from the topic subscription. Here we only want to retrieve one message at a time from the service bus using a peek-lock trigger and using the Session Id from the initial service bus trigger “When a message is received in a topic subscription”. We then check if a message is found in the output body using the expression “@not(equals(length(body(‘Get_messages_from_a_topic_subscription_(peek-lock)’)), 0))”
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb13.png?w=615&h=464
If a message is found, the “If True” branch is executed which again calls the same Logic App as before to process the message. Note the indexer to get to the context data as the service bus connector trigger above returns a collection.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb14.png?w=599&h=562
Once a successful response is received from the ProcessSBMessage Logic App, the message is completed and the LoopCounter variable is reset to zero. Note the lock token is from the service bus connector trigger within the loop and the Session Id is from the initial service bus connector which started the workflow. https://connectedcircuits.files.wordpress.com/2017/08/image_thumb15.png?w=603&h=477
Below is the code view for setting the lockToken and SessionId of the “Complete the message” action inside the loop. Take note of the indexer “[0]” before the LockToken element.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb16.png?w=620&h=114
If no messages are found on the service bus, the False branch is then executed. This simply has a delay action as not to pool too quickly and increments the LoopCounter.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb17.png?w=622&h=435
The last step is to close the session when the Until loop exists using the Session Id from the initial service bus connector trigger which started the workflow.
https://connectedcircuits.files.wordpress.com/2017/08/image_thumb10.png?w=628&h=291
Now we are ready to the send messages into the service bus. We should see a Logic App spin up for each unique session Id. Remember to set the session Id property on the service bus to some value before sending the message.