Strip double quotes from an import file in Integration Services SSIS

Problem

When loading data using SQL Server Integration Services (SSIS) to import data from a CSV file, every single one of the columns in the CSV file has double quotes around the data. When using the Data Flow Task to import the data I have double quotes around all of the imported data. How can I import the data and remove the double quotes?

Solution

This is a pretty simple solution, but the fix may not be as apparent as you would think. Let’s take a look at our example.

Here is the sample CSV file as it looks in a text editor. You can see that all of the columns have double quotes around the data even where there is no data. The file is comma delimited, so this should give us enough information to import the data column by column.

sample csv data

To create the package we use a Data Flow Task and then use the Flat File Source as our data flow source.

ssis control flow
ssis data flow

When setting up the Flat File Connection for the data source we enter the information below, basically just selecting our source file.

ssis connection manager

If we do a quick preview on the dataset we can see that every column has the double quotes even the columns where there is no data. If you open the text file in Excel the double quotes are automatically stripped, so what needs to be done in SSIS to accomplish this.

ssis flat file connection

On this screen you can see the highlighted area and the entry that is made for the “Text qualifier”. Here we enter in the double quote mark ” and this will allow SSIS to strip the double quotes from all columns.

ssis text qualifier

If we do another preview we can see that the double quotes are now gone and we can move on to the next part of our SSIS package development.

ssis data preview from connection manager

As mentioned above, this is a simple fix to solve this problem. If you are faced with this issue, hopefully this gives you a quick answer to get your development moving forward. This same technique can be used to strip any other text qualifier data from your files.

Next Steps

2 Comments

  1. Hi Chintan, for your example you might want to just load all columns as strings and then after you get the data loaded you can strip the ” at the beginning and the end and then convert the data into the appropriate data types.

    Another thing you could do preprocess the file before using SSIS maybe using PowerShell or some other tool. Replace any occurrence where ” is the first character or the last character in the row and then replace “,” with just ,

    This should strip out all of the double quotes.

    -Greg

  2. Hello,

    This solution was helpful but my csv file have text qualifier as ” and some of the columns have “
    Below is the example :

    “Worker”,”Y”,”Mr”,”Peter (“Pietro”)”,”Marciano”,”1973-09-19

    The column Peter (“Pietro”) have ” in it.

    How to load such a data via SSIS ?

Leave a Reply

Your email address will not be published. Required fields are marked *