![]() |
|
|
By: Ray Barley | Read Comments (11) | Print Ray is a Principal Architect at RDA Corporation and a MSSQLTips.com BI Expert. Related Tips: 1 | 2 | 3 | More |
|
Problem
We have adopted XML configuration files as a standard development practice within our organization. We use them in ASP.NET web applications as well as Windows Forms applications. How can we use XML configuration files with our SSIS packages?
Solution
SSIS has built-in support for using XML files for package configuration. Just about any property setting in a package or task can be retrieved at runtime from an XML configuration file. A single XML configuration file can store as many configuration settings as you need, allowing you to use the same configuration file for multiple SSIS packages. However, every property that is specified in the XML configuration file must exist in the SSIS package else an error is raised when opened in BIDS (or Visual Studio). A simple work around for this behavior is to use multiple XML configuration files, each containing a group of properties that exist in every package that uses the XML configuration file.
Let's walk thru an example to demonstrate the steps required to setup an XML package configuration. Assume that we want to populate an Excel file with sales information from the AdventureWorks database. Ultimately we would like to execute this SSIS package from a SQL Agent job which will run on a scheduled basis. In addition we would like the ability to run the package for any date range on demand.
For our example we will specify the following configuration settings in our XML package configuration:
Now we are ready to setup an SSIS package to use XML package configuration. To begin, open Business Intelligence Developer Studio (BIDS) or Visual Studio and create a new Integration Services project. Add a new SSIS package to the project then perform the following steps:
| 1. Add the Begin Date and End Date variables to the package; we will set these from the XML package configuration and use them as parameters in our SQL command (right click on the Control Flow and select Variables): |
| 2. Add an OLE DB Connection Manager for AdventureWorks (right click in the Connection Managers area and add a new OLE DB connection): |
| 3. Add an Excel Connection Manager for our output file (right click in the Connection Managers area, select New Connection then Excel): |
| 4. Add a Data Flow Task to the Control Flow (drag/drop Data Flow Task from the Toolbox onto the Control Flow): |
| 5. Add an OLE DB Source and an Excel Destination to the Data Flow Task (right click the Data Flow Task then select edit; drag/drop source and destination from the Toolbox onto the Data Flow Task): |
| 6. Configure the OLE DB Source (right click then select Edit): |
| 7. Set the parameters for the SQL command text (click the Parameters button in the OLE DB Source Editor): |
| 8. Configure the Excel Destination (right click and select Edit) |
| 9. Specify the Excel create table statement (click New on the Excel Destination Editor; override the LineTotal column type as shown below): |
| 10. Enable Package Configuration (right click on the Control Flow, select Package Configurations, then check Enable package configurations): |
| 11. Add the XML Package Configuration (click Add on the Package Configurations Organizer) : |
| 12. Select the properties to include in the XML Package Configuration file (drill in to the Objects and click the check boxes for the AdventureWorks ConnectionString, Excel Connection Manager ConnectionString, v_BeginDate and v_EndDate variable's Value): |
| 13. Complete the XML Package Configuration: |
| 14. Review the XML Package Configuration file created (navigate to the file name as noted in Step 13 above and open it with Internet Explorer): |
Note the following in the XML package configuration file (as shown in step 14 above) :
At this point the XML package configuration example is complete; you can edit the XML file as appropriate and run the SSIS package to see the results. However, the XML file name and path is hard-coded into the SSIS package. It would be nice to make this a variable that we can specify at runtime. Two options come to mind:
Step 11 above is where the XML file name and path was specified. We selected the radio button "Specify configuration settings directly" then entered the full path to the file. Instead we could specify the full path to the file in an environment variable, select the radio button "Configuration location is stored in an environment variable", and specify the environment variable. For example:
To specify the XML file name and path on the command line when executing the SSIS package, navigate to the SSIS package file using Windows Explorer and double click the file to launch the Execute Package Utility. Select Configurations and add the configuration file as shown below then click the Execute button to run the package:
Clicking on Command Line will show the actual DTEXEC command line:
Caveats:
Remember that any environment variable that you add while BIDS (or Visual Studio) is open will not show up in the environment variable list (e.g. Select Configuration Type dialog above) until you close the application and reopen it.
When you are working with an SSIS package in BIDS (or Visual Studio) , the package configuration is read when you open the package. Any changes made to the configuration will not be reflected until you close and reopen the SSIS package.
When using an Excel destination the Excel file has to already exist. In addition the default behavior is to append rows to the worksheet. It's usually a good idea to make a backup copy of the Excel file that gets created when configuring the Excel destination (see steps 8 and 9 above). You can then copy/rename the backup to the original Excel file name to start out with an empty worksheet before rerunning your package.
Next Steps
| Share: | Share | Tweet |
|
![]() |
![]() |
Free SQL Server Learning |
| Friday, June 26, 2009 - 5:49:20 AM - Luminai | Read The Tip |
|
Hi! Am I to understand that the only way to specify a path for an xml configuration file for an SSIS package after deployment, is to use an environment variable? This seems silly and strange since there is acutally a page in the ExecutePackageUtility where it seems possible to specify which configuration files to use... Or am I missing something? |
|
| Monday, June 29, 2009 - 6:22:55 PM - raybarley | Read The Tip |
|
If you execute the package using DTEXECUI, there is a Configurations page where you can specify the XML configuration file(s). If you execute the package using DTEXEC, you can specify like this: /CONFIGFILE "C:\demo\SSIS-XMLConfig\SSIS-XMLConfig.dtsConfig". |
|
| Monday, June 29, 2009 - 11:24:39 PM - Luminai | Read The Tip |
|
I used the UI, and did specify my updated config file, but it did not use this file anyway... I just learned about the deployment utility option when building my package. Could it be that since I did not have a deployment version of the package, it ignored the settings I provided?
Edit: I just looked at my first post, realised I did say 'after deployment', but by this I actually was referring to my copying the necessary files to the destination server and trying to use them there. Not deploying as in building with the deployment utility activated. |
|
| Tuesday, June 30, 2009 - 4:26:42 AM - raybarley | Read The Tip |
|
When you execute the package with DTEXECUI select Reporting in the list box on the left, then click the Verbose checkbox. This gives quite a bit of output when you run the package. Maybe it will uncover something. |
|
| Wednesday, July 01, 2009 - 3:31:45 AM - Luminai | Read The Tip |
|
Seems that I now am passed the configuration issue. Selecting to create a deployment manifest, and then using that on the server, allowed me to specify where settings where supposed to be read from, as well as supplying the server settings. Next problem is that apparently we have not got the oledb engine for automating excel installed on servers, so now I have to battle the operations team to get that sorted :P. Should anyone else be in need of a smiliar solution, this is what I did: 1. Created a xml-configuration with the values I wanted (I choose one config file for each SSIS package). 2. Activated the option Create deployment utility, which is in the projet property page under Deployment Utility. 3. Built the project 4. Copied manifest, dtsx files and dependencies (config and excel files in my case) to the server 5. Double-clicked the manifest to run it, followed the guide to install and configure my packages. Voilį! All done! |
|
| Tuesday, August 11, 2009 - 2:12:37 AM - alextvino | Read The Tip |
|
Hi All
I am having my configuration in sql server SSIS Configurations table how can i add this to SQL Server Integration Services (SSIS) Packages Regards Alex |
|
| Tuesday, August 11, 2009 - 3:19:33 AM - raybarley | Read The Tip |
|
Take a look at this tip: http://www.mssqltips.com/tip.asp?tip=1405 To add to existing packages, open the package and go through the same steps as described in the tip. When specifying that you want a particular property value to be set from the SQL Server configuration you will get a prompt whether to reuse the existing configuration - answer yes. |
|
| Tuesday, August 11, 2009 - 4:16:51 AM - alextvino | Read The Tip |
|
Hi raybarley i have alredy done this my actual problem is i have created the package , added sql config and then i have imported the package to integration service msdb folder.here i have to add configuration to run the package but it accepts only xml config .tell me how to add sql config to imported package in msdb
Regards ALEX |
|
| Tuesday, August 11, 2009 - 4:59:50 AM - raybarley | Read The Tip |
|
Sorry I don't understand your question. I typically deploy packages to SQL Server which means they wind up being stored in the msdb database, in particular in the table [msdb].[dbo].[sysdtspackages90]. You do not need any other configuration to make this work although you could. When you use the SQL Server table for configuration, you have to make sure that when you deploy your packages (wherever you deploy them) when you run the package it can find the SQL Server database where you have the configuration table. There is a connection manager in your package that specifies the connection string to the SQL Server database that has the configuration table. In the tip I pointed you to in my earlier reply, I use the environment variable configuration to specify the connection string to the SQL Server database where the configuration table is. So all I have to do to deploy is make sure the environment variable gets set properly and anything that's currently running (like SQL Server agent) gets restarted so it can see the environment variable that's been added. This is a bit of a pain but you only have to do it once. |
|
| Tuesday, August 11, 2009 - 6:52:51 AM - alextvino | Read The Tip |
|
let me check this thanks for your lightning replay |
|
| Saturday, May 19, 2012 - 1:47:13 AM - Debarchan | Read The Tip |
|
Thanks for the post! Also, there is an utility now available in CodePlex which can modify SSIS Package Configuration file paths without BIDS: |
|
|
privacy | disclaimer | copyright | advertise | about authors | contribute | feedback | giveaways | user groups Some names and products listed are the registered trademarks of their respective owners. Edgewood Solutions LLC | MSSharePointTips.com | MSSQLTips.com |