solving sql server problems for millions of dbas and developers since 2006 join MSSQLTips for free SQL Server tips













































   Got a SQL tip?
            We want to know!

Recursive Queries using Common Table Expressions (CTE) in SQL Server

MSSQLTips author Edwin Sarmiento By:   |   Read Comments (13)   |   Related Tips: More > Scripts

Problem
In SQL Server 2000, you need to implement recursive queries to retrieve data which is presented in a hierarchical format. We normally resort to implementing views, cursors or derived tables and perform queries against them. The problem arises when the hierarchy level increases as SQL Server is limited to 32 levels of recursion. We need a better way to implement recursive queries in SQL Server 2005. How do we do it?

Solution
Common Table Expression (CTE) was introduced in SQL Server 2005 and can be thought of as a temporary result set that is defined within the execution scope of a single SELECT, INSERT, UPDATE, DELETE, or CREATE VIEW statement. You can think of CTE as an improved version of derived tables that more closely resemble a non-persistent type of view. Look at CTEs as your derived tables in SQL Server 2000. A CTE can be used in many of the same ways you use a derived table. CTEs can also contain references to themselves. This allows the developer to write complex queries simpler. CTEs can also be used in place of views. The use of CTEs provides two main advantages. One is that queries with derived table definitions become more simple and readable. While traditional T-SQL constructs that are used to work with derived tables normally requires a separate definition for the derived data such as a temporary table or a table-valued function, using CTEs make it easier to see the definition of the derived table with the code that uses it. The other thing is that CTEs significantly reduces the amount of code required for a query that traverses recursive hierarchies.

To understand what a CTE is all about, let's first take a look at the syntax to create it in SQL Server 2005.

Syntax
In general form a recursive CTE has the following syntax:

WITH cte_alias (column_aliases
AS 
(
cte_query_definition   --initialization
UNION ALL
cte_query_definition2 --recursive execution

SELECT FROM cte_alias 

You provide the CTE with an alias and an optional list of aliases for its result columns following the keyword WITH which usually defines the derived table based on the query definition; write the body of the CTE; and refer to it from the outer query.

To put this in the right perspective, let's come up with a simple example which uses recursion. We'll look at the Employees table in the Northwind database and see that a particular employee reports to another employee. One question we can come up with is, "Who reports to whom?" The Employees table is designed in such a way that the ReportsTo column is a foreign key field that refers to the primary key field EmployeeID. Thus, we can create a query to answer our question. A sample query using CTE will look something like this.

WITH Managers AS
(
--initialization
SELECT EmployeeIDLastNameReportsTo 
FROM Employees
WHERE ReportsTo IS NULL
UNION ALL
--recursive execution
SELECT e.employeeID,e.LastNamee.ReportsTo
FROM Employees e INNER JOIN Managers m 
ON e.ReportsTo m.employeeID
)
SELECT FROM Managers 

Code Walkthrough

  1. The recursive CTE, Managers, defines an initialization query and a recursive execution query
  2. The initialization query returns the base result and is the highest level in the hierarchy.  This is identified by the ReportsTo value of NULL, which means that the particular Employee does not report to anybody.  Depending on how the table is designed, the value can be anything as long as it represents the highest level in the hierarchy
  3. The recursive execution query is then joined to the initialization query using the UNION ALL keyword.  The result set is based on the direct subordinate as returned by the initialization query, which then appears as the next level in the hierarchy. Recursion occurs because of the query referencing the CTE itself based on the Employee in the Managers CTE as input. The join then returns the employees who have their managers as the previous record returned by the recursive query.  The recursive query is repeated until it returns an empty result set.
  4. The final result set is returned by querying the Managers CTE 

The sample query contains the elements that a recursive CTE must contain. What's more is that the code is a lot more readable. This enables the developers to write complex queries with ease.

You can also use a query hint to stop a statement after a defined number of loops. This can stop a CTE from going into an infinite loop on a poorly coded statement. You do this by including the MAXRECURSION keyword in the SELECT query referring to the CTE. To use it in the previous example

SELECT FROM Managers OPTION (MAXRECURSION 4

To create a similar yet non-recursive query that produces the same result in SQL Server 2000, you might come up with something similar to this code:

DECLARE @rowsAdded INT

--table variable to hold accumulated results
DECLARE @managers TABLE --initialize @managers who do not have managers
   
(EmpID INTMgrID INTprocessed INT DEFAULT(0))

INSERT @managers 
SELECT EmployeeIDReportsTo
FROM Employees 
WHERE ReportsTo IS NULL

SET @rowsAdded=@@rowcount 

--do this while new employees are added in the previous iteration 
WHILE @rowsAdded 
BEGIN

   
--mark employee records going to be found in this iteration with --processed=1 
   
UPDATE @managers SET processed=WHERE processed=0

   
--insert employees who report to employees not yet processed 
   
INSERT @managers 
   
SELECT EmployeeIDReportsTo
   
FROM Employees e 
   
INNER JOIN @managers ON e.ReportsTo r.EmpID 
   
WHERE ReportsTo <> EmployeeID AND r.processed 
          
   
SET @rowsAdded @@rowcount 

   
--mark employee records found in this iteration as processed 
   
UPDATE @managers SET processed=WHERE processed=

END 

SELECT 
FROM @managers 

Next Steps

Given the example above, hierarchical data structures, organizational charts and other parent-child table relationship reports can easily benefit from the use of recursive CTEs. Common Table Expression is just one of those T-SQL enhancements available for SQL Server 2005. CTEs bring us the chance to create much more complex queries while retaining a much simpler syntax. They also can lessen the administrative burden of creating and testing views for situations where the view will not be reused.

  • Learn more about using recursive CTEs from the MSDN Library
  • You can get the Northwind database used in the sample here


Last Update: 6/16/2008


About the author
MSSQLTips author Edwin Sarmiento
Edwin Sarmiento works as a SQL Server DBA for The Pythian Group in Ottawa and is a SQL Server MVP.

View all my tips


print tip Print  
Become a paid author





join MSSQLTips for free SQL Server tips     



Learn more about SQL Server tools
Post a comment or let the author know this tip helped you.

       All comments are reviewed, so stay on subject or we may delete your comment.

*Name   *Email Notify for updates



       Note: your email address is not published. Required fields are marked with an asterisk (*)


Get free SQL tips:

*Enter Code refresh code     



Saturday, February 15, 2014 - 8:33:14 AM - Mohit Read The Tip

I need to write much more complex queries of 1000 lines in CTE but i am unable to understand the scenario how to fetech all the data after end of my temp tables.Could someone help me on that?

Dta is all about of HR:

My data is one temp table for all the Hires data,2) temp table for all the Terminatins, 3) temp table for start data ,4) temp table end data,

I need to find tranfer in and Transfer out among the hire and end data.

How can I find out  transfer In and Transfer Out?

 

Please Help!!!!!!!!!!!!! 

 


Friday, October 11, 2013 - 9:26:32 AM - upen Read The Tip

Yes,

this was really simple and informative.

 

Thanks,


Tuesday, July 02, 2013 - 8:16:37 PM - Kristy Read The Tip

"Select EmployeeID, LastName, ReportsTo from Employees" what is the difference between this and 

"WITH Managers AS --initialization ... "

query above?

I get the same result running on SQL 2012 database against NorthWnd database.


Thursday, March 21, 2013 - 5:21:41 PM - Tallion Read The Tip
An example I was able to understand and put into action. I have one table (Inventory) with three columns (ID, Desc, ParentID) where the parentID links back to the ID. I store an item in house A (id=1, PID = null), Room B (ID=2, PID=1) box C (ID=3, PID=2) (house A > Room B > box C). So when i query the table i have the box C ID of 3. this allowed me to 'walk' up the chain to get room B and house A WITH InvItems AS ( SELECT a.ID ,a.Description , A.ParentLocationID from InventoryLocations A where a.ID = 6 UNION ALL SELECT b.ID,b.Description, b.ParentLocationID from InventoryLocations b INNER JOIN InvItems I ON b.ID = I.ParentLocationID ) SELECT * FROM InvItems returns ID Desc ParentD 3 box C 2 2 Room B 1 1 House A Null Thanks, Tallion

Tuesday, March 12, 2013 - 12:47:48 AM - Rashid Abbas Read The Tip

WITH Managers AS 

  --initialization 

  SELECT Employee_Id,Full_Name,SBT_Name, Manager_Id,1 LevelNumber

  From HRM.dbo.HR_Employee

  WHERE Manager_Id IS NULL 

  UNION ALL 

  --recursive execution 

  SELECT e.Employee_Id,e.Full_Name,e.SBT_Name, e.Manager_Id,m.LevelNumber+1 LevelNumber 

  FROM HRM.dbo.HR_Employee e

  INNER JOIN Managers m  ON e.Manager_Id = m.Employee_Id 

SELECT * FROM Managers OPTION (MAXRECURSION 3)--Number should by three or more


Monday, February 18, 2013 - 7:33:13 AM - Ian Read The Tip

Nice, very clear, I think the guy saying you could do it another way is missing the point. thanks


Monday, July 30, 2012 - 1:31:20 AM - Nikhilesh Patidar Read The Tip

 

Thank you  for provide solution


Friday, May 18, 2012 - 12:10:53 AM - Abhi Read The Tip

Suppose I have a table REL(ParentID int, ChildID int)

The values in the table are

11, 1

11,2

22,1

22,4

33,5

44,4

Given the input as 11, I wnat the output as

11

22

44

Logic: Starting with 11, the childIDs are 1,2.

Now ParentId having child id = 1,2 are 22.

Now ParentId 22 has childid as 1 and 4.

Again ParentID 44 has childis as 4.

So the output should be

11

22

44

 How can I acheive this? Please help.


Thursday, July 21, 2011 - 9:12:52 AM - Raj (SQL lover) Read The Tip

One of the best example for CTE in SLQ Server 2008.

Good Job.

Raj

 


Wednesday, March 16, 2011 - 1:57:13 PM - Jim Davis Read The Tip

Excellent example, it saves a lot of time over other methods to do the same thing. Much simpler than examples from MS. Personally, I really appreciate it when someone, like this writer, makes such an effort to explain how to do something like this. This will save everyone alot of time and effort.

Keep up the good work.

Jim Davis


Friday, January 07, 2011 - 8:19:46 PM - Randy Edge Read The Tip

Thanks for the code.  You saved me a day or two of messing around.

As usual Microsofts example was overly complicated. It had joins to other tables which just make it harder to see what what going on.You did an excellent job.


Friday, July 18, 2008 - 5:42:16 AM - bass_player Read The Tip

Hi Pro,

CURSORS and SELF JOINS can be used in the given example. It's just that CTEs would make it a lot simpler and easier to achieve the required task


Friday, July 18, 2008 - 2:35:46 AM - Pro Read The Tip

Though the post is nice but the example that was taken here can be easily handles using a self join on the employer table and using curser

Can anyone please explain me why should we use CTE in place of CURSORS or SELF JOIN for the given example




 
Sponsor Information







Copyright (c) 2006-2014 Edgewood Solutions, LLC All rights reserved
privacy | disclaimer | copyright | advertise | about
authors | contribute | feedback | giveaways | free t-shirt | user groups | community | events | first timer?
Some names and products listed are the registered trademarks of their respective owners.