solving sql server problems for millions of dbas and developers since 2006



SQL Server DBA Tips SQL Server Developer Tips SQL Server Business Intelligence Tips SQL Server Career Tips SQL Server Tip Categories SQL Server Tutorials SQL Server Webcasts SQL Server Whitepapers SQL Server Tools SQL Server Questions and Answers MSSQLTips Authors About MSSQLTips SQL Server User Groups MSSLQTips Giveaways MSSQLTips Advertising Options

MSSQLTips Facebook Page MSSQLTips LinkedIn Page MSSQLTips RSS Feed MSSQLTips Twitter Page MSSQLTips Google+ Page





SQL Server Cursor Example

By: | Read Comments (20) | Print

Jeremy is the CTO @ Edgewood Solutions, co-founder of MSSQLTips.com and SQL Server MVP since 2009.



Related Tips: 1 | 2 | 3 | More

Problem

In my T-SQL code I always use set based operations.  I have been told these types of operations are what SQL Server is designed to process and it should be quicker than serial processing.  I know cursors exist but I am not sure how to use them.  Can you provide some cursor examples?  Can you give any guidance on when to use cursors?  I assume Microsoft created them for a reason so they must have a place where they can be used in an efficient manner.

Solution

In some circles cursors are never used, in others they are a last resort and in other groups they are used regularly.  In each of these camps they have different reasons for their stand on cursor usage.  Regardless of your stand on cursors they probably have a place in particular circumstances and not in others.  So it boils down to your understanding of the coding technique then your understanding of the problem at hand to make a decision on whether or not cursor based processing is appropriate or not.  To get started let's do the following:

  • Look at an example cursor
  • Break down the components of the cursor
  • Provide additional cursor examples
  • Analyze the pros and cons of cursor usage

Example SQL Server Cursor

Here is an example cursor from tip Simple script to backup all SQL Server databases where backups are issued in a serial manner:

DECLARE @name VARCHAR(50) -- database name  
DECLARE @path VARCHAR(256) -- path for backup files  
DECLARE @fileName VARCHAR(256) -- filename for backup  
DECLARE @fileDate VARCHAR(20) -- used for file name

SET @path = 'C:\Backup\'  

SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112)

DECLARE db_cursor CURSOR FOR  
SELECT
name
FROM MASTER.dbo.sysdatabases
WHERE name NOT IN ('master','model','msdb','tempdb')  

OPEN db_cursor  
FETCH NEXT FROM db_cursor INTO @name  

WHILE @@FETCH_STATUS = 0  
BEGIN  
       SET
@fileName = @path + @name + '_' + @fileDate + '.BAK'  
      
BACKUP DATABASE @name TO DISK = @fileName  

      
FETCH NEXT FROM db_cursor INTO @name  
END  

CLOSE
db_cursor  
DEALLOCATE db_cursor

SQL Server Cursor Components

Based on the example above, cursors include these components:

  • DECLARE statements - Declare variables used in the code block
  • SET\SELECT statements - Initialize the variables to a specific value
  • DECLARE CURSOR statement - Populate the cursor with values that will be evaluated
    • NOTE - There are an equal number of variables in the DECLARE <cursor_name> CURSOR FOR statement as there are in the SELECT statement.  This could be 1 or many variables and associated columns.
  • OPEN statement - Open the cursor to begin data processing
  • FETCH NEXT statements - Assign the specific values from the cursor to the variables
    • NOTE - This logic is used for the initial population before the WHILE statement and then again during each loop in the process as a portion of the WHILE statement
  • WHILE statement - Condition to begin and continue data processing
  • BEGIN...END statement - Start and end of the code block
    • NOTE - Based on the data processing multiple BEGIN...END statements can be used
  • Data processing - In this example, this logic is to backup a database to a specific path and file name, but this could be just about any DML or administrative logic
  • CLOSE statement - Releases the current data and associated locks, but permits the cursor to be re-opened
  • DEALLOCATE statement - Destroys the cursor

Additional SQL Server Cursor Examples

In the example above backups are issued via a cursor, check out these other tips that leverage cursor based logic:

SQL Server Cursor Analysis

The analysis below is intended to serve as insight into various scenarios where cursor based logic may or may not be beneficial:

  • Online Transaction Processing (OLTP) - In most OLTP environments, SET based logic makes the most sense for short transactions.  Our team has run into a third party application that uses cursors for all of its processing, which has caused issues, but this has been a rare occurrence.  Typically, SET based logic is more than feasible and cursors are rarely needed.
  • Reporting - Based on the design of the reports and the underlying design, cursors are typically not needed.  However, our team has run into reporting requirements where referential integrity does not exist on the underlying database and it is necessary to use a cursor to correctly calculate the reporting values.  We have had the same experience when needing to aggregate data for downstream processes, a cursor based approach was quick to develop and performed in an acceptable manner to meet the need.
  • Serialized processing - If you have a need to complete a process in serialized manner, cursors are a viable option.
  • Administrative tasks - Many administrative tasks need to be executed in a serial manner, which fits nicely into cursor based logic, but other system based objects exist to fulfill the need.  In some of those circumstances, cursors are used to complete the process.
  • Large data sets - With large data sets you could run into any one or more of the following:
    • Cursor based logic may not scale to meet the processing needs.
    • With large set based operations on servers with a minimal amount of memory, the data may be paged or monopolize the SQL Server which is time consuming can cause contention and memory issues.  As such, a cursor based approach may meet the need.
    • Some tools inherently cache the data to a file under the covers, so processing the data in memory may or may not actually be the case.
    • If the data can be processed in a staging SQL Server database the impacts to the production environment are only when the final data is processed.  All of the resources on the staging server can be used for the ETL processes then the final data can be imported.
    • SSIS supports batching sets of data which may resolve the overall need to break-up a large data set into more manageable sizes and perform better than a row by row approach with a cursor.
    • Depending on how the cursor or SSIS logic is coded, it may be possible to restart at the point of failure based on a checkpoint or marking each row with the cursor.  However, with a set based approach that may not be the case until an entire set of data is completed.  As such, troubleshooting the row with the problem may be more difficult.

SQL Server Cursor Alternatives

Below outlines alternatives to cursor based logic which could meet the same needs:

Next Steps

  • When you are faced with a data processing decision determine where you stand with SQL Server cursor usage.  They may or may not have a place in your application or operational processes.  There are many ways to complete a task, so using a cursor could be a reasonable alternative or not.  You be the judge.
  • If you run into issues with another coding technique and need to get something done quickly, using a cursor may be a viable alternative.  It may take longer to process the data, but the coding time might be much less.  If you have a one time process or nightly processing, this could do the trick.
  • If cursors are shunned in your environment, be sure to select another viable alternative.  Just be sure the process will not cause other issues.  As an example, if a cursor is used and millions of rows are processed will this potentially flush all of the data from cache and cause further contention?  Or with a large data set will the data be paged to disk or written to a temporary directory?
  • As you evaluate a cursor based approach versus other alternatives make a fair comparison of the techniques in terms of time, contention and resources needed.  Hopefully these factors will drive you to the proper technique.


Related Tips: 1 | 2 | 3 | More | Become a paid author


Last Update: 10/23/2011

Share: Share 






Comments and Feedback:

Tuesday, October 07, 2008 - 7:27:41 AM - aprato Read The Tip

One thing that should be added... if you're going to use a cursor and the results do not have to be sensitive to change and you don't need to move back and forth within the set, then look at using the FAST_FORWARD option which is a low overhead, read only, forward only type with no tempdb overhead.  It runs through the resultset sequentially.


Thursday, October 09, 2008 - 9:45:15 AM - timothyrcullen Read The Tip

I think another point to mention is to limit the amount of information obtained for use with the cursor.  I see a number of people who select everything from a table even though they only need two fields. 


Thursday, October 09, 2008 - 5:36:31 PM - aprato Read The Tip

 Ahhhh, yes. Good one, Tim.  I can't tell you how many times I've seen SELECT * used when declaring a cursor.


Wednesday, January 27, 2010 - 9:37:15 AM - emcquaid Read The Tip

We are using a script very similar to this.  It uses exec msdb.dbo.sp_send_dbmail to email our clients appointment confirmations.  

A sql view has been written that calls the data set correctly.   Some clients have two appts the same day for different reasons and at different times. Most emails are sent correctly, with the correct data. 

But these records for clients that have two appts are sending only the data for one of the appts, and then sending it twice.  Any ideas?  Is there a flush command that can solve this?

Cheers,

 

Eric


Wednesday, January 27, 2010 - 8:28:35 PM - admin Read The Tip

Eric,

My first inclination would be that the query you use to populate the cursor is not getting the correct data.  If you just run the SELECT query in the 'Declare Cursor' portion of the code, are you getting the correct data? 

If not, this code needs to be amended.

If so, can you provide some sample data and the code?

Thank you,
The MSSQLTips Team


Wednesday, February 09, 2011 - 4:47:35 PM - Douglas Brum Read The Tip

Jeremy,

 

thanks for the tips. It helped a lot.

 

Cheers,

Douglas Brum


Monday, April 16, 2012 - 11:06:31 AM - Tony Read The Tip

I have an EMPLOYEE table which consists of empID, empLastName, empFirstName, empAddress1... columns.

How do I reference a specific column in a cursor using SELECT * from EMPLOYEE?

Thank you very much.

 

Example code:

DECLARE emp_cur CURSOR FOR
SELECT * FROM Employee
ORDER BY empID;

OPEN empl_cur
FETCH NEXT FROM emp_cur
WHILE @@FETCH_STATUS = 0
   BEGIN
      -- PRINT empID, empFirstName, empLastName here
   END
CLOSE emp_cur;
DEALLOCATE emp_cur;


Monday, April 16, 2012 - 12:40:15 PM - Tim Cullen Read The Tip

Unless you absolutely have to use every column in the table for the cursor activity I would avoid SELECT * statements.  You should be able to reference the column by name.  Did you get an error when you attempted to execute the code above?


Monday, April 16, 2012 - 12:48:56 PM - Tim Cullen Read The Tip

Tony:

 

Here is an example of using a non-cursor loop to get the information:

SET NOCOUNT ON;

DECLARE @Transactions TABLE

(EntryID INT IDENTITY(1,1) PRIMARY KEY, TransactionTypeID INT, TransactionType VARCHAR(150))

 

DECLARE @Start INT = 1, @End INT, @TransactionTypeID INT, @TransactionType VARCHAR(150)

INSERT INTO @Transactions (TransactionTypeID, TransactionType)

SELECT TransactionTypeID, TransactionType FROM dimTransactionType WITH (NOLOCK)

SELECT @End = @@ROWCOUNT

WHILE @Start <= @End

BEGIN

SELECT @TransactionTypeID = TransactionTypeID, @TransactionType = TransactionType 

FROM @Transactions WHERE EntryID = @Start

PRINT @TransactionTypeID

PRINT @TransactionType

SELECT @Start += 1

CONTINUE

END

SET NOCOUNT OFF;


Friday, April 20, 2012 - 11:53:22 AM - Jeremy Kadlec Read The Tip

Tony,

I apologize for my delayed response.  I think this might be the code you are looking for.

DECLARE @empID INT
DECLARE @empFirstName varchar(50)
DECLARE @empLastName varchar(50)

DECLARE emp_cur CURSOR FOR
SELECT empID, empFirstName, empLastName
FROM Employee
ORDER BY empID;

OPEN emp_cur
FETCH NEXT FROM emp_cur INTO @empID, @empFirstName, @empLastName

WHILE @@FETCH_STATUS = 0
   BEGIN
       
 -- PRINT empID, empFirstName, empLastName here
 
 FETCH NEXT FROM emp_cur INTO @empID, @empFirstName, @empLastName

   END
CLOSE emp_cur;
DEALLOCATE emp_cur;

 

Thank you,
Jeremy Kadlec


Friday, April 20, 2012 - 12:20:38 PM - Jeremy Kadlec Read The Tip

Tim,

Thank you for the quick response and alternatives.

Thank you,
Jeremy Kadlec


Friday, April 20, 2012 - 1:17:52 PM - Tony Read The Tip

Thank you Tim and Jeremy for your responses. The code I submitted was only an example. I'd really prefer to use SELECT * statement as I have to use and process more than 100 columns in the table. I really don't want to list them all multiple times (DECLARE, SELECT, FETCH NEXT (twice)).

When I referred to empID as emp_cur.empID, I got the error "SQL Server Database Error: The name "emp_cur.empID" is not permitted in this context. Valid expressions are constants, constant expressions, and (in some contexts) variables. Column names are not permitted.

Thanks again for your help.

Tony


Friday, April 20, 2012 - 2:20:20 PM - Jeremy Kadlec Read The Tip

Tony,

To take a step back, what are you ultimately trying to do?  Issue a simple UPDATE, INSERT or DELETE statement?  ETL process?  Data import or export?

If I understand your first question correctly, you would need to reference each column when you create a variable, populate the cursor, iterate the row set if you are processing each column in some sort of manner. 

If I understand your second question correctly, you would need to reference the variable @empID.

Thank you,
Jeremy Kadlec


Monday, April 30, 2012 - 6:53:56 AM - Deepak Read The Tip

 i want to use cursor in this table ..want to fetch value from id through cursor..please help me immediately..

Stu_Id

Stu_Name Stu_Class
1 Komal 10
2 Ajay 10
3 Rakesh 10
4 Bhanu 10
5 Santosh 10
6 Tanuj 10

 

 i want to use cursor in this table ..want to fetch value from id through cursor..please help me immediately..


Tuesday, May 01, 2012 - 9:07:01 AM - Armando Prato Read The Tip

WHY do you need a cursor, Deepak?   What post processing do you need to do on the data that can't be done using a SET operation?

Cursors should never be your first choice for data access.  


Tuesday, May 01, 2012 - 1:22:39 PM - Deepak Read The Tip

yes sir you are right..But i want to understand the concept of cursor with a simple and easy example...

 

 

please help me sir with better eaxample through which i can understand the concept easily...as i am the beginner in SQL Server...


Tuesday, May 01, 2012 - 4:16:43 PM - Tony Read The Tip

I'm not sure if you've received my post as I don't see it above. Here it is again and I appologize if I've posted it twice.

I have to write a process that reads Employee records, update other tables and insert into another table for other processes. Before updates and inserts can be done, I'd have to do a lot of data manipulations. I guess my question should be:

If I MUST use select * from employee cursor, how would I go about referencing a column (i.e. employee.empID) from the cursor without having to declare vairables and fetch into them?

Below is the equivalent in ORALCE:

for emp_cur in (select * from employee order by empID) loop

   str_empFormalName := emp_cur.empLastName || ', ' || emp_cur.empFirstName || ', ' || emp_cur.empMiddleInitial;

   update tempEmployee1
      set empFormalName = str_empFormalName;
    where empID = emp_cur.empID;

    insert into tempEmployee2 (empID, empLastName, empFirstName, empMiddleInitial, empFormalName)
       values (emp_cur.empID, emp_cur.empLastName, emp_cur.empFirstName, emp_cur.empMiddleInitial);

end loop;

As you can see from above, I've named my cursor emp_cur and used select * from employee. When I want to reference a column, I qualify the column name with the cursor name (i.e. emp_cur.empID), and I do not have to use variables.

Thanks,

Tony

 


Monday, May 07, 2012 - 9:57:52 AM - Jeremy Kadlec Read The Tip

Tony,

Did you see these two posts?

  • Monday, April 16, 2012 - 12:48:56 PM - Tim Cullen
  • Friday, April 20, 2012 - 11:53:22 AM - Jeremy Kadlec

Did they help point you in the right direction?

If the issue is creating variables for each column, I would try to use Tim's code as an example.

HTH.

Thank you,
Jeremy Kadlec

 


Monday, May 07, 2012 - 9:59:28 AM - Jeremy Kadlec Read The Tip

Deepak,

Did these two sections in this tip make sense to you:

  • Example SQL Server Cursor
  • SQL Server Cursor Components

Please let me know.

Thank you,
Jeremy Kadlec


Monday, May 07, 2012 - 10:40:41 AM - Tony Read The Tip

Thank you both Tim and Jeremy. I really would like to use SELECT * in the cursor and avoid using variables just like my example in Oracle. But if I have to use variables then...

Thanks again for your time.

Tony



Post a Comment or Question

Keep it clean and stay on the subject or we may delete your comment.
Your email address is not published. Required fields are marked with an asterisk (*)

*Name   *Email   Notify for updates
Comments
*Enter Code refresh code


 
New SQL Monitor v3.0

New SQL Monitor v3.0


Sponsor Information
Try the free performance monitoring tool from Idera!

Time-strapped database professionals use SQL Monitor to look after their servers. Try it online.

SQL Servers keeping you up at night? Contact the Edgewood SQL Server DBAs for a Health Check. Prices start at $995.

Find and Fix SQL issues with Foglight Performance Analysis. Get a free copy.

Solving SQL Server problems for millions of DBAs and Devs since 2006. Join now.

The SQL Server Security THREAT - It’s Closer Than You Think


Copyright (c) 2006-2012 Edgewood Solutions, LLC All rights reserved
privacy | disclaimer | copyright | advertise | about
authors | contribute | feedback | giveaways | user groups
Some names and products listed are the registered trademarks of their respective owners.


Edgewood Solutions LLC | MSSharePointTips.com | MSSQLTips.com