Learn more about SQL Server tools

mssqltips logo
 

Tutorials          DBA          Dev          BI          Career          Categories          Webcasts          Scripts          Today's Tip          Join

Tutorials      DBA      Dev      BI      Categories      Webcasts

DBA    Dev    BI    Categories

 

SQL Server Integration Services SSIS Performance Tuning Techniques



By:
Overview

In the last chapter of this tutorial weíll look at a couple of performance optimizations you can implement in your SSIS packages. After all, you want to move data around as quickly as possible.

SSIS Control Flow Performance Optimizations

There is typically not much you can do in the control flow to improve performance. There are two key points however:

  • Think about if you want to perform tasks in SSIS or if you can do them somewhere else. For example, sorting data will be faster in SQL Server T-SQL code than in SSIS.
  • Perform tasks in parallel if possible, but donít overdo it. Going into parallel will surely improve performance, but this is heavily influenced by available memory and the number of processors. There is a certain overhead to parallelism. If thereís too much parallelism, the system will go slower instead of faster. Carefully test to find the optimum balance.

SSIS Data Flow Performance Optimizations

Most performance issues are related to the data flow. As with the control flow, think if SSIS or transformations in SQL will be faster. Try to visualize the data flow as a pipeline with data flowing through. You want to maximize the flow rate to get data to the destination as quickly as possible. There are some important properties you can set to influence the memory buffers.

buffer properties
  • DefaultBufferMaxRows: This is by default set to 10,000, which was made a good setting in 2005. Today we have more power in our machines, so you can set this to a higher number. Too large a buffer will take a while to fill and will make the destination sit idle, so you need to find a good optimum. 50,000 rows is a good start.
  • DefaultBufferSize: This is by default 10MB which is again quite small. You can bump this up to 50MB or even more.

The actual buffer size will be determined by which of the two properties is reached first. You can set AutoAdjustBufferSize to True to make sure that the specified number of rows in DefaultBufferMaxRows is always met.

SSIS Reading Data Performance Optimizations

Some guidelines:

  • Donít use the dropdown box to select the source table. Write a SQL statement and include filtering, grouping and sorting in the SQL code.
  • Only select columns you actually need.
  • Keep the data types of the columns small. The more rows you can fit in a single memory buffer, the better.

SSIS Transforming Data Performance Optimizations

Here we have some best practices as well:

  • Donít use blocking transformations (e.g. sort and aggregate component). They read all data in memory before even sending one single row to the output. Asynchronous transformations are to be avoided as well since they modify the memory buffer. You can find a good overview in this blog post.
  • Avoid the Slowly Changing Dimension Wizard. It uses the OLE DB Command, which executes SQL statements row-by-row, which is slow and results in excessive logging.
  • Donít use the OLE DB Command, as stated in the previous point.

SSIS Writing Data Performance Optimizations

Writing data is typically the slowest part of the process. Here are some tips to optimize the process:

  • The OLE DB Destination is the fastest adaptor for SQL Server at the moment. If you use the Fast Load option of course.
  • Make sure you use a table lock (which is enabled by default).
  • To speed up inserts, you can disable constraints and drop and recreate indexes.
  • Try to go for minimally-logged inserts, which means the destination needs to be in Simple recovery mode. Itís a broad topic, but you can find more info in the Data Loading Performance Guide.
  • Make sure the database files have an appropriate size. Time is lost when the database has to be constantly growing its files.
Additional Information

Last Update: 8/31/2017




More SQL Server Solutions











Post a comment or let the author know this tip helped.

All comments are reviewed, so stay on subject or we may delete your comment. Note: your email address is not published. Required fields are marked with an asterisk (*).

*Name    *Email    Email me updates 


Signup for our newsletter
 I agree by submitting my data to receive communications, account updates and/or special offers about SQL Server from MSSQLTips and/or its Sponsors. I have read the privacy statement and understand I may unsubscribe at any time.



    



Learn more about SQL Server tools