Retry SSIS Control Flow Tasks

Problem

In the Control Flow of a SQL Server Integration Service (SSIS) package there can be cases where some control flow tasks are dependent on outside resources.  For example, there can be control flows where it will copy files from one server to another or there can be cases where file compression is taking place. Since these are error prone activities, there can be cases where the entire package will fail. Therefore, there should be a method to retry the process in the event of a failure. However, the retry should take place only when there is a failure and only for limited number of tries otherwise the package could run forever.

Solution

There is no built-in feature in SSIS with this functionality, therefore you need to configure SSIS to achieve the above objectives.

Example SSIS Package Using Copying a File

Let’s see simple an implementation of a file copy using the File System Task in the SSIS control flow.

Let’s configure this for a simple file copy as shown in the below image.

When this is executed, it will execute as long as the destination path and the source file is available. In the case where any of these are not available, the task will fail which will cause the entire package to fail.

SSIS Package Retry Option

The requirement is that in the event the file is not available, it should automatically try again for a defined number of times with a specified wait interval between tries. This type of feature is available in SQL Server Agent Jobs as shown below. In the Advanced tab of the SQL Server Agent job step configuration, the user has the option of configuring the Retry attempts and the Retry intervals.

So, if you use this option in a SQL Server Agent job, the entire package will re-execute when a task fails. You can utilize this option, but you need to modularized your SSIS packages which could become difficult to manage.

So, what we are looking at is having a SSIS option to retry when is there is a failure.

The following SSIS implementation could be used which includes a For Loop Container, Expression (EXIT) and SQL Execution Task (WAIT).

To do this, we need two integer variables to be defined.  Variable I is the running variable while variable RetryCount is number of retries it should make.  The boolean variable UntilSuccess is used as a flag for successful execution.

With these variable definitions, in case there is a need to change the retry times, it is just a matter of changing the RetryCount variable value.

In the For Loop Editor, there are three properties to set: InitExpression, EvalExpession and AssignExpression.  The initial configurations are shown in the below image.

Success Path

For the Success path, when there is success, variable I will be set to the RetryCount and the loop will stop and exit.

Failure Path

The Failure path is little more complex than the success path as there are few more configurations.

The WAIT will be done using the T-SQL command WAITFOR DELAY ’00:00:05’. This delay is 5 seconds, however you can configure it suit to your needs.  You will need to have a SQL Server connection to execute this T-SQL statement.

If you need to configure the delay, you can configure this value as a variable and modify the above code.

Next is configuring the expression in the failure path flow. This configuration is only needed, if you are configuring your package to execute until it is successful. If not, you don’t need this path.

This expression is, if the UntilSuccess variable is true, variable I will be set to 0 if not it will be incremented by 1.

When it is set to 0, the For Each Loop will execute until the file copy is successful.

Next is an important configuration in the For Loop Container.  In the property window of the For Loop Container, the MaximumErrorCount default value is 1. This means this will fail after one error. This value needs to be set to 0 or it should be set to number of retries which means that controller will not fail.

That is all you need to do in order to configure the SSIS package to retry.

Execution of the SSIS Package

When the package executes, if the file copy fails, the package will take following execution where it takes the failure path and tries again.

When the package executes successfully after a few attempts, we can see the flow in the Progress window as shown below. The below shows the file copy is successful after a couple of failure attempts.

Next Steps

• Implement retry configuration with SSIS by following above steps.
• Check out these other SSIS tips

Dinesh Asanka

Dinesh Asanka holds a Bachelor of Science in Electrical Engineering, an MBA in Information Technology, a Master of Science in Artificial Intelligence, and an MPhil in Data Warehousing from the University of Moratuwa. With more than 30 years of industry experience, he has successfully bridged the gap between theoretical knowledge and practical application in the fields of data management and analytics. Currently serving as a Senior Lecturer at the University of Kelaniya, he teaches subjects including Data Science, Big Data Analytics, and Database Administration. His research interests focus on Educational Analytics, Health Analytics, Data Science, and Artificial Intelligence-driven solutions for decision-making and learning enhancement. In addition to his academic and research contributions, Dinesh is also an active columnist and presenter who regularly contributes to professional forums, conferences, workshops, and public discussions related to technology, analytics, and digital transformation.

• MSSQLTips Awards: Rookie of the Year Contender – 2018