How To Call Logic App Synchronously From Azure Data Factory


By:   |   Updated: 2020-01-03   |   Comments   |   Related: More > Azure

Problem

I have a Logic App created in the Azure Portal. I have created a pipeline in Azure Data Factory where I need to call this Logic App so it can transfer some data to blob storage. However, when I call the Logic App using the Web Activity, it is executed asynchronously, meaning the task finishes immediately. I would like the task waiting for the completion of the Logic App. Is this possible in Azure Data Factory?

Solution

As a test case for this tip, we’re going to use a Logic App which sends an email:

Logic App email notification

The steps to create such a Logic App are described in the tip Azure Data Factory Pipeline Email Notification – Part 1. We’re going to expand this Logic App with a delay, so we can easily check if the task in the Azure Data Factory pipeline is executing synchronously (waiting for the Logic App to finish) or asynchronously (finishing immediately when the HTTP message is posted). Hover with your mouse over the arrow and click the plus icon to add a new action:

add new action

From the Schedule section, choose the Delay action:

add delay

Configure the action to wait 30 seconds:

wait for 30 seconds

The pipeline in Azure Data Factory (ADF) looks like this:

adf notification pipeline

You can find a step-by-step description on how to create this pipeline in the tip Send Notifications from an Azure Data Factory Pipeline – Part 2.

Synchronously Calling a Logic App

In ADF, there are two methods to start a Logic App from a pipeline: the Web Activity or the recently introduced Webhook Activity. In the tip mentioned above, the Web Activity was used to perform the HTTP request.

Using a Web Activity

To get the Web Activity to behave synchronously, we need to modify the Logic App. We need to add a HTTP response after we’ve send the email. This can be found under the Request connector:

add http response

The default response will have the status code 200, which means OK. Let’s also rename the action by clicking on the three dots in the upper right corner and choosing Rename.

200 response

We also want to fail the Logic app if the email couldn’t be sent, so let’s add another response. Click on the plus icon between the send mail action and the response and add a parallel branch.

add parallel branch

Add another response, but this time set the status code to 418 (which stands for I’m a teapot. You can find a list of status code on this website). You can choose any status code of the 400 and 500 range.

failure response

If this response is returned to a Web Activity, the ADF pipeline will fail. Rename the action, but also change the Configure run after setting:

configure run after

Here, we want to run this action only if the previous action has failed:

run after failure

We end up with the following Logic App configuration:

finished logic app with responses

Now we can run our pipeline! You can either trigger the pipeline or just debug it, but make sure to specify a valid email address in the parameters:

pipeline debug parameters

If you’re debugging, you can see the Web Activity is running synchronously:

synchrounous debugging

However, when it is finished it will display only 1 second duration:

pipeline

When you trigger the pipeline instead of a debug session, it will show the correct duration in the monitoring section:

correct duration trigger

If we change the Logic App to fail by sending the 418 response code, we can see the pipeline fails as well:

adf doesn

Using a Webhook

The Webhook activity was recently introduced as a new activity in the ADF pipeline. To use it, we need to slightly modify the initial Logic App. Instead of using a request response, we’ll use a HTTP POST message to a CallBackUri.

After the send mail action, add a new HTTP action:

add http action

The Method needs to be POST. For the URI, click on the Add dynamic content link or just click inside the text box.

configure http post

Switch to expression and add the following:

triggerBody().callBackUri
add callbackuri

The action now looks like this:

finished action with callbackuri

Now it’s time to create a pipeline with a Webhook activity. Create a new pipeline and add the Webhook Activity to the pipeline.

webhook activity

The configuration of the webhook is similar to the Web Activity:

webhook config
  • The URL is the trigger of the Logic App. You can copy paste it from the trigger when editing the Logic App.
  • The method is POST.
  • You need to specify application/json as the Content-Type in the headers.
  • The body specifies the JSON payload in the HTTP POST message. This contains pipeline parameters. Here’s the full body:
{"DataFactoryName":"@{pipeline().DataFactory}","PipelineName":"@{pipeline().Pipeline}","Subject":"@{pipeline().parameters.Subject}","ErrorMessage":"@{pipeline().parameters.ErrorMessage}","EmailTo":"@pipeline().parameters.EmailTo"}

Debugging the pipeline shows us the Webhook Activity is indeed executing synchronously:

send notification logic app

In contrast with the Web Activity, the Webhook shows the correct duration.

Conclusion

In this tip we’ve seen two different methods for executing a Logic App synchronously from Azure Data Factory: using the Web Activity and the Webhook Activity respectively. In both cases, the Logic App needs to be adapted slightly to accommodate this behavior. However, the configuration of the webhook requires only one additional action and the duration is displayed correctly in the ADF pipeline.

Keep in mind the methods explained in this tip are not exclusively for Logic Apps, but also for other services using webhooks, such as Azure Automation Runbooks for example.

Next Steps


Last Updated: 2020-01-03


get scripts

next tip button



About the author
MSSQLTips author Koen Verbeeck Koen Verbeeck is a BI professional, specializing in the Microsoft BI stack with a particular love for SSIS.

View all my tips
Related Resources





Comments For This Article





download


Recommended Reading

Adding Users to Azure SQL Databases

Getting Started with Azure Blueprints

Connect to On-premises Data in Azure Data Factory with the Self-hosted Integration Runtime - Part 1

Azure Data Factory vs SSIS vs Azure Databricks

Continuous database deployments with Azure DevOps





get free sql tips
agree to terms


Learn more about SQL Server tools