![]() |
|
|
By: Greg Robidoux | Read Comments (24) | Print Greg is the President of Edgewood Solutions and a co-founder of MSSQLTips.com. Related Tips: More |
|
Problem
Every once in awhile a table gets created without a primary key and duplicate records get entered. The problem gets even worse when you have two identical rows in the table and there is no way to distinguish between the two rows. So how do you delete the duplicate record?
Solution
One option that SQL Server gives you is the ability to set ROWCOUNT which limits the numbers of records affected by a command. The default value is 0 which means all records, but this value can be set prior to running a command. So let's create a table and add 4 records with one duplicate record.
Create a table called duplicateTest and add 4 records.
CREATE TABLE dbo.duplicateTest
(
[ID] [int] ,
[FirstName] [varchar](25),
[LastName] [varchar](25)
) ON [PRIMARY]
INSERT INTO dbo.duplicateTest VALUES(1, 'Bob','Smith')
INSERT INTO dbo.duplicateTest VALUES(2, 'Dave','Jones')
INSERT INTO dbo.duplicateTest VALUES(3, 'Karen','White')
INSERT INTO dbo.duplicateTest VALUES(1, 'Bob','Smith')
If we select all data we get the following:
SELECT * FROM dbo.duplicateTest
| ID | FirstName | LastName |
| 1 | Bob | Smith |
| 2 | Dave | Jones |
| 3 | Karen | White |
| 1 | Bob | Smith |
If we try to select the record for Bob Smith will all of the available values such as the following query:
SELECT * FROM dbo.duplicateTest WHERE ID = 1 AND FirstName = 'Bob' AND LastName = 'Smith'
We still get 2 rows of data:
| ID | FirstName | LastName |
| 1 | Bob | Smith |
| 1 | Bob | Smith |
So to delete the duplicate record with SQL Server 2000 and 2005 we can use the SET ROWCOUNT command to limit the number of rows affected by a query. By setting it to 1 we can just delete one of these rows in the table. Note: the select commands are just used to show the data prior and after the delete occurs.
SELECT * FROM dbo.duplicateTest
SET ROWCOUNT 1
DELETE FROM dbo.duplicateTest WHERE ID = 1
SET ROWCOUNT 0
SELECT * FROM dbo.duplicateTest

With SQL Server 2005 we can also use the TOP command when we issue the delete, such as the following. Note: the select commands are just used to show the data prior and after the delete occurs.
SELECT * FROM dbo.duplicateTest
DELETE TOP(1) FROM dbo.duplicateTest WHERE ID = 1
SELECT * FROM dbo.duplicateTest

So as you can see with SQL Server 2005 there are two options to allow you to delete duplicate identical rows of data in your tables.
Here is one note from Microsoft about using SET ROWCOUNT:
| Using SET ROWCOUNT will not affect DELETE, INSERT, and UPDATE statements in the next release of SQL Server. Avoid using SET ROWCOUNT together with DELETE, INSERT, and UPDATE statements in new development work, and plan to modify applications that currently use it. Also, for DELETE, INSERT, and UPDATE statements that currently use SET ROWCOUNT, we recommend that you rewrite them to use the TOP syntax. |
Next Steps
Change History
| Release | History |
| May 24, 2007 | Here is an updated way to delete a variable set of duplicate rows using the TOP command. The data below represents a sample data set where the same value appears multiple times, but there is no primary key to only delete all but one record. EmpID Aufaq Aufaq Aufaq Aufaq Aufaq Basharat Basharat Basharat Basharat John John John John John John If you use the following command this will get a count of how many rows there are and delete this minus one record. Thanks goes out to Basharat Bhat for this update to this tip. Basharat Bhat |
| Share: | Share | Tweet |
|
![]() |
![]() |
Free SQL Server Learning |
| Tuesday, March 18, 2008 - 2:29:33 AM - Bals | Read The Tip |
|
The corelated query in TOP clause is not working .. Basharat Bhat has given only for single dup entry |
|
| Friday, April 04, 2008 - 7:38:20 AM - admin | Read The Tip |
|
Bals, Can you please post the code you are having an issue with? Thank you, |
|
| Tuesday, May 06, 2008 - 1:16:11 PM - Gadi35 | Read The Tip |
|
There is another very good way to do this, in SQL Server 2005, if you do not have the key set up, but want to effectively use a column or more as your primary key. ;with DelDup as (select row_number() over (partition by --alternate-- ;with DelDup as (select row_number() over (partition by You can easily adjust this to use one column or as many as you wish, as long as you update the column(s) after the partition by and after the order by. |
|
| Friday, May 09, 2008 - 2:29:02 PM - admin | Read The Tip |
|
Thank you both for the feedback. Thank you, |
|
| Thursday, May 15, 2008 - 7:31:44 AM - Prakash M | Read The Tip |
|
Hi Friend Pls test the following one, dis s a very simple method Delete Top (Select Count(*)-1 From EmpTest Where EmpId=A.EmpId)
|
|
| Sunday, May 18, 2008 - 11:57:57 PM - balakumar.sk | Read The Tip |
|
It didnt work Prakash, gave me the follwoing errorMsg 156, Level 15, State 1, Line 2 Incorrect syntax near the keyword 'As'."
|
|
| Friday, May 29, 2009 - 2:22:18 AM - kanagarajum | Read The Tip |
|
Hai, can u try this code?
DELETE FROM TestTable WHERE
|
|
| Wednesday, February 22, 2012 - 12:30:20 AM - karthick | Read The Tip |
|
hi, im facing one problem in ur example you knows the duplicate data's key value so you can delete easily. What my question in there is 1000's records with few duplicate (in composit key one is different) then how to extract the that particular duplicate records and delete it
|
|
| Friday, March 02, 2012 - 1:49:40 AM - manolitobsj | Read The Tip |
|
This post have been posted for more than two year but it helped me alot. Thanks. |
|
| Friday, March 09, 2012 - 6:24:41 AM - gopi | Read The Tip |
|
There are duplicate records in Fact and Facterror table(both table have same records in them some 100 records).i need to delete the records from facterror table and insert it into Fact table. |
|
| Friday, March 09, 2012 - 8:24:37 AM - Greg Robidoux | Read The Tip |
|
Gopi, not sure what you are asking. If the records already exist in both the FACT table and FACTERROR then can you just delete the records from the FACTERROR table? |
|
| Wednesday, March 14, 2012 - 1:42:54 PM - erni | Read The Tip |
|
how to delete duplicate using phpmyadmin?is working when insert data,data can not read..that he publish 1234567 are duplicate primary key and when insert 3456789,he also publish 1234567? how i can i do?plz |
|
| Wednesday, March 14, 2012 - 1:48:47 PM - Greg Robidoux | Read The Tip |
|
Hi Erni, not exactly sure what you need to do. If you are trying to eliminate duplicate primary key values in a table, make sure the table has a PRIMARY KEY setup so there is not a chance to get duplicate records. The front end application would need to handle the error message from SQL Server, but this would ensure you don't have duplicates. |
|
| Tuesday, April 10, 2012 - 6:50:11 AM - kisan | Read The Tip | |||||||||
|
I want To delete Secound Record how i can delete:
|
||||||||||
| Tuesday, April 10, 2012 - 7:31:14 AM - Greg Robidoux | Read The Tip |
|
Hi Kisan, if the records are identical it doesn't really matter which record you are deleting also you have no control over what record SQL Server will present to you. If there are other columns in the table that have different values then you can add that to the where clause if not you won't have any real control. You can also look the post above from Gadi35. |
|
| Wednesday, April 11, 2012 - 8:13:02 AM - kisan | Read The Tip |
|
I got your answer if i delete record it will delete both beczz their is no. primary key use it... if i m taking any column so i cant delete but thier is redudency it will delete both record......
so telll me sir what synatx should i use in delete query to deleted secound record.. |
|
| Wednesday, April 11, 2012 - 8:49:12 AM - Greg Robidoux | Read The Tip |
|
Kisan - if you run this query this will delete only one of these records. Since they are identical it doesn't matter which one you delete. DELETE TOP(1) FROM dbo.duplicateTest WHERE ID = 1
|
|
| Wednesday, April 11, 2012 - 12:14:13 PM - Gadi35 | Read The Tip |
|
Greg, that last code will cancel anything that exists and only leave the duplicates. |
|
| Wednesday, April 11, 2012 - 12:55:19 PM - Greg Robidoux | Read The Tip |
|
Gadi35 - yes you are correct that it will only delete one record. I was referring to the example that "kisan" listed that just had two records, so this would delete one of those two records, but I agree if there were more than two duplicates only one record would be deleted.
|
|
| Wednesday, April 11, 2012 - 12:56:01 PM - kisan | Read The Tip |
|
thanku... sir.......
this i try this query it will work fine....... thanku muchhhhhh.. |
|
| Thursday, April 12, 2012 - 3:00:31 AM - kisan | Read The Tip |
|
sir I have One More Question Suppose i create auto id like create table demo ( id int identity(1,1) primary key ) if i inserted 5 records and i delte record Number 5 so agin if i inserted record so it will give id no:6 but i want id 5 so how will be do sir.... |
|
| Thursday, April 12, 2012 - 8:59:47 AM - Greg Robidoux | Read The Tip |
|
Kisan - if you delete records from a table that uses an identity column you are going to have gaps in the numbering. If there were only 5 records in the table 1,2,3,4,5 and you delete record number 5 the next identity value will still be 6. You could use the RESEED option to fix this and make the next number 5 again. Take a look at this tip: http://www.mssqltips.com/sqlservertip/1123/managing-and-maintaining-sql-server-identity-values/ But if you have records with values 1,2,3,4,5 and you delete record number 3 you will still have a row without a number 3 value. To get around this you could use SET IDENTITY INSERT. Take a look at this tip: http://www.mssqltips.com/sqlservertip/1061/using-identity-insert-to-keep-sql-server-table-keys-in-sync/
|
|
| Friday, April 27, 2012 - 8:45:00 AM - Jeff | Read The Tip |
|
Seems like you could also use select top 1 |
|
| Friday, April 27, 2012 - 4:57:45 PM - varshini | Read The Tip |
|
Nice Job
|
|
|
privacy | disclaimer | copyright | advertise | about authors | contribute | feedback | giveaways | user groups Some names and products listed are the registered trademarks of their respective owners. Edgewood Solutions LLC | MSSharePointTips.com | MSSQLTips.com |