Problem
One difficulty in our modern world is all the choices. Your favorite streaming service offers hundreds of shows to choose from. Approaches to problem-solving in SQL Server are the same, with numerous options. Do you need help deciding on a technique for tracking history in your tables? You’ve seen a few options but don’t know which one to choose. Stay tuned for a comparison of two popular methods.
Solution
In this article, I want to look at two different methods for accomplishing the same goal, tracking table history. First, we’ll look at why you generally want to track a table’s history. Continuing, I’ll present two typical methods you see, triggers and temporal. Does one perform better than the other? What are some drawbacks of each approach? These are a couple of questions I’ll answer. By the end of this article, you can make a better decision regarding implementing a history-tracking method.
Tracking Table History
Why do you need to track history in a table? I can think of two primary reasons. You can likely come up with others. First, your organization wants to know what’s changing and who made the change. Whenever someone deletes or changes data, the first question is: Can we tell who did this? Getting to the bottom of this is helpful when you seek to stop someone from accidentally doing it again. Now, if the person does the same harmful behavior, there may be consequences. Having the history is nice because we can likely bring the data back to life or provide it to the end user to re-add.
The second reason dovetails off the first: legal or compliance regulations may require tracking data modification. For example, you could have a SOC 2 compliance standard calling for monitoring all data modifications. This reason could tie in with needing historical data for reporting.
Tools for Tracking
SQL Server offers multiple tools for tracking modifications in tables. A few are Change Data Capture (CDC) and Change Tracking. The latter doesn’t track the before and after values like CDC. Another option might be to incorporate the logic into whatever method you’re using to modify the data in the first place, either with stored procedures or entity framework. This approach takes forethought in planning when building the application, which we all know doesn’t always happen. Additionally, you can use one of the many third-party applications. However, this article explores triggers and temporal tables.
Temporal Tables
When discussing temporal tables, people sometimes hear temporary tables. But they are nothing alike. What’s a system-versioned temporal table? Microsoft defines them as a user table with a full history of data changes, allowing easy point-in-time analysis. Microsoft introduced them in SQL Server 2017, and they provided a great alternative to triggers or CDC for history tracking.
It’s a breeze to enable temporal, which is one reason I like them. The code would look something like what’s below. A primary key must be defined on the temporal table, or SQL returns an ugly error message.
source: https://www.mssqltips.com<br /><br />
CREATE TABLE dbo.MySpecialTable<br />(<br /> Id INT NOT NULL,<br /> Column1 NVARCHAR(250) NULL,<br /> Column2 NVARCHAR(250) NULL,<br /> SysStartTime DATETIME2 GENERATED ALWAYS AS ROW START NOT NULL,<br /> SysEndTime DATETIME2 GENERATED ALWAYS AS ROW END NOT NULL,<br /> PERIOD FOR SYSTEM_TIME(SysStartTime, SysEndTime),<br /> CONSTRAINT PK_EmployeeTemporal_Employee_Id<br /> PRIMARY KEY CLUSTERED (Id),<br />)<br />WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.MySepcialTable_History));<br />GO
When you add a new column to the non-temporal table, SQL automatically adds it to the temporal table. A downside of the temporal table is the lack of customization. For example, if you allow users to modify data without checking if something changes, SQL still inserts a row into the history table. You can argue that you should fix the underlining code causing duplicate records; I agree.
Also, if a product implements a retention policy where client data removal occurs every 20-30 days, it’s a huge pain to turn off the versioning and back on. Imagine performing this operation on 100+ tables.
Using Triggers
If you’ve been in the database game for a while, you have an opinion on triggers. Some people hate them, but others can’t get enough. One area where you’ll see people generally agree is when it comes to auditing. Microsoft defines a trigger as a special stored procedure that automatically runs when an event occurs in the database server. This article focuses on DML triggers that execute after an UPDATE or DELETE.
Simple triggers are easy to create. The code below creates a trigger with similar functionality to the above temporal table.
--source: https://www.mssqltips.com<br /><br />
CREATE OR ALTER TRIGGER dbo.TR_MySpecialTable_Insert_History<br />ON dbo.MySpecialTable<br />AFTER UPDATE, DELETE<br />AS<br />BEGIN<br /> INSERT INTO dbo.EmployeeTrigger_History<br /> (<br /> Id,<br /> Column1,<br /> Column2,<br /> SysStartTime,<br /> SysEndTime<br /> )<br /> SELECT Id,<br /> Column1,<br /> Column2,<br /> SysStartTime,<br /> GETUTCDATE()<br /> FROM DELETED;<br />END;<br />GO
One thing I don’t like about triggers is that you need to maintain them. For example, if you add a new column to the non-history table, you need to add it to the history and update the triggers logic to account for it. If your development team creates T-SQL regularly, this might be a non-issue. Additionally, it’s easy to manage if you only track history on 5-20 tables.
Triggers are also a gateway to incorporating excessive business logic. When business folks learn about triggers, they become the duct tape of databases. We don’t need to fix the application code; add a trigger to do it. However, for our comparison in the article, we’ll only focus on keeping track of history.
Building the Dataset
Let’s create decent-size tables for testing with one million rows. The code below creates four tables and populates the non-history ones. One nice thing about temporal is that the history table uses page compression. With that in mind, we’ll enable page compression on our TriggerHistory table. Additionally, on the EmployeeHistory_Trigger table, I’ll add a clustered index on the SysStartTime and SysEndTime columns.
--source: https://www.mssqltips.com<br /><br />
USE [master];<br />GO<br /><br />IF DATABASEPROPERTYEX( 'HistoryDemo',<br /> 'Version'<br /> ) IS NOT NULL<br />BEGIN<br /> ALTER DATABASE HistoryDemo SET SINGLE_USER WITH ROLLBACK IMMEDIATE;<br /> DROP DATABASE HistoryDemo;<br />END;<br />GO<br /><br />CREATE DATABASE HistoryDemo;<br />GO<br /><br />ALTER DATABASE HistoryDemo SET RECOVERY SIMPLE;<br />GO<br /><br />USE HistoryDemo;<br />GO<br /><br />DECLARE @UpperBound INT = 1000000;<br />;WITH cteN (Number)<br />AS (<br /> SELECT ROW_NUMBER() OVER (ORDER BY s1.[object_id])<br /> FROM sys.all_columns AS s1<br /> CROSS JOIN sys.all_columns AS s2<br /> )<br />SELECT [Number]<br />INTO dbo.Numbers<br />FROM cteN<br />WHERE [Number] <= @UpperBound;<br /><br />CREATE UNIQUE CLUSTERED INDEX CIX_Number<br />ON dbo.Numbers ([Number])<br />WITH (FILLFACTOR = 100);<br /><br />CREATE TABLE dbo.EmployeeTemporal<br />(<br /> Id INT IDENTITY(1, 1) NOT NULL,<br /> EmployeeNumber CHAR(20) NOT NULL,<br /> FirstName NVARCHAR(256) NOT NULL,<br /> LastName NVARCHAR(256) NOT NULL,<br /> MiddleIntial CHAR(1) NULL,<br /> YearlySalary DECIMAL(36, 2) NOT NULL,<br /> DepartmentCode CHAR(5) NOT NULL,<br /> StartDate DATE NOT NULL,<br /> IsActive BIT NOT NULL<br /> DEFAULT 1,<br /> IsDeleted BIT NOT NULL<br /> DEFAULT 0,<br /> SysStartTime DATETIME2 GENERATED ALWAYS AS ROW START NOT NULL,<br /> SysEndTime DATETIME2 GENERATED ALWAYS AS ROW END NOT NULL,<br /> PERIOD FOR SYSTEM_TIME(SysStartTime, SysEndTime),<br /> CONSTRAINT PK_EmployeeTemporal_Employee_Id<br /> PRIMARY KEY CLUSTERED (Id)<br />)<br />WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.EmployeeTemporal_History));<br /><br />CREATE TABLE dbo.EmployeeTrigger<br />(<br /> Id INT IDENTITY(1, 1) NOT NULL,<br /> EmployeeNumber CHAR(20) NOT NULL,<br /> FirstName NVARCHAR(256) NOT NULL,<br /> LastName NVARCHAR(256) NOT NULL,<br /> MiddleIntial CHAR(1) NULL,<br /> YearlySalary DECIMAL(36, 2) NOT NULL,<br /> DepartmentCode CHAR(5) NOT NULL,<br /> StartDate DATE NOT NULL,<br /> IsActive BIT NOT NULL<br /> DEFAULT 1,<br /> IsDeleted BIT NOT NULL<br /> DEFAULT 0,<br /> SysStartTime DATETIME2 NOT NULL<br /> DEFAULT GETUTCDATE(),<br /> SysEndTime DATETIME2 NOT NULL<br /> DEFAULT CONVERT( DATETIME2,<br /> '9999-12-31 23:59:59.9999999'<br /> )<br /> CONSTRAINT PK_EmployeeTrigger_Employee_Id<br /> PRIMARY KEY CLUSTERED (Id)<br />);<br />GO<br /><br />CREATE TABLE dbo.EmployeeTrigger_History<br />(<br /> Id INT NOT NULL,<br /> EmployeeNumber CHAR(20) NOT NULL,<br /> FirstName NVARCHAR(256) NOT NULL,<br /> LastName NVARCHAR(256) NOT NULL,<br /> MiddleIntial CHAR(1) NULL,<br /> YearlySalary DECIMAL(36, 2) NOT NULL,<br /> DepartmentCode CHAR(5) NOT NULL,<br /> StartDate DATE NOT NULL,<br /> IsActive BIT NOT NULL,<br /> IsDeleted BIT NOT NULL,<br /> SysStartTime DATETIME2 NOT NULL,<br /> SysEndTime DATETIME2 NOT NULL<br />)<br />WITH (DATA_COMPRESSION = PAGE);<br />GO<br /><br />CREATE CLUSTERED INDEX ix_EmployeeTrigger_History<br />ON dbo.EmployeeTrigger_History (<br /> SysEndTime ASC,<br /> SysStartTime ASC<br /> );<br /><br />INSERT INTO dbo.EmployeeTemporal<br />(<br /> EmployeeNumber,<br /> FirstName,<br /> LastName,<br /> MiddleIntial,<br /> YearlySalary,<br /> DepartmentCode,<br /> StartDate,<br /> IsActive<br />)<br />SELECT TOP (1000000)<br /> CONCAT('E00', n.Number) AS EmployeeNumber,<br /> SUBSTRING('ABCDEFGHIJKLMNOPQRSTUVWXYZ', (ABS(CHECKSUM(NEWID())) % 26) + 1, 6) AS FirstName,<br /> SUBSTRING('ABCDEFGHIJKLMNOPQRSTUVWXYZ', (ABS(CHECKSUM(NEWID())) % 26) + 1, 7) AS LastName,<br /> SUBSTRING('ABCDEFGHIJKLMNOPQRSTUVWXYZ', (ABS(CHECKSUM(NEWID())) % 26) + 1, 1) AS MiddleInitial,<br /> ABS(CHECKSUM(NEWID()) % 100000) + 45000 AS YearlySalary,<br /> CONCAT('M000', ABS(CHECKSUM(NEWID()) % 3) + 1) AS DepartmentCode,<br /> DATEADD(DAY, RAND(CHECKSUM(NEWID())) * (1 + DATEDIFF(DAY, '2020-01-01', '03-31-2023')), '2020-01-01') AS StartDate,<br /> CASE<br /> WHEN (n.Number % 1000) = 0 THEN<br /> 0<br /> ELSE<br /> 1<br /> END AS IsActive<br />FROM dbo.Numbers n;<br />GO<br /><br /><br />INSERT INTO dbo.EmployeeTrigger<br />(<br /> EmployeeNumber,<br /> FirstName,<br /> LastName,<br /> MiddleIntial,<br /> YearlySalary,<br /> DepartmentCode,<br /> StartDate,<br /> IsActive,<br /> IsDeleted<br />)<br />SELECT <br /> EmployeeNumber,<br /> FirstName,<br /> LastName,<br /> MiddleIntial,<br /> YearlySalary,<br /> DepartmentCode,<br /> StartDate,<br /> IsActive,<br /> IsDeleted FROM dbo.EmployeeTemporal;<br />GO<br /><br />CHECKPOINT;<br />GO
The screenshot below illustrates what our tables look like.
With our tables and data, it’s now time to create a trigger. I want a trigger to insert rows into the history anytime someone updates or deletes rows, mimicking the temporal behavior. You could also create two triggers, one for the update and another for the delete. For the demo, we’ll keep things simple.
--source: https://www.mssqltips.com<br /><br />
CREATE OR ALTER TRIGGER dbo.TR_Employee_Insert_History<br />ON dbo.EmployeeTrigger<br />AFTER UPDATE, DELETE<br />AS<br />BEGIN<br /> INSERT INTO dbo.EmployeeTrigger_History<br /> (<br /> Id,<br /> EmployeeNumber,<br /> FirstName,<br /> LastName,<br /> MiddleIntial,<br /> YearlySalary,<br /> DepartmentCode,<br /> StartDate,<br /> IsActive,<br /> IsDeleted,<br /> SysStartTime,<br /> SysEndTime<br /> )<br /> SELECT Id,<br /> EmployeeNumber,<br /> FirstName,<br /> LastName,<br /> MiddleIntial,<br /> YearlySalary,<br /> DepartmentCode,<br /> StartDate,<br /> IsActive,<br /> IsDeleted,<br /> SysStartTime,<br /> GETUTCDATE()<br /> FROM DELETED;<br />END;<br />GO
Configure the Performance Test
Without reading further, which method do you think performs better when updating or deleting data? Before writing this, I picked temporal outperforming triggers by a mile.
To test this, I’ll first update 10 thousand rows in each non-history table using SQLQueryStress.
--source: https://www.mssqltips.com<br /><br />
UPDATE dbo.EmployeeTrigger<br />SET YearlySalary = ABS(CHECKSUM(NEWID()) % 100000) + 45000,<br /> IsDeleted = 1,<br /> DepartmentCode = 'M0004',<br /> SysStartTime = GETUTCDATE()<br />WHERE Id = @trigger;<br /><br />UPDATE dbo.EmployeeTemporal<br />SET YearlySalary = ABS(CHECKSUM(NEWID()) % 100000) + 45000,<br /> IsDeleted = 1,<br /> DepartmentCode = 'M0004'<br />WHERE Id = @trigger;
I’ll define the parameter substitution query below. This returns 10 thousand random numbers between one and a million.
--source: https://www.mssqltips.com<br /><br />
SELECT TOP (10000)<br /> Number AS Id<br />FROM dbo.Numbers<br />ORDER BY NEWID();
Running the Test
Now it’s time to execute each test and review the results. I executed each of these a few dozen times. Below are the typical results.
After testing, temporal tables slightly outperformed triggers. The results are close enough that it’s a tie in my book. You could modify the trigger to improve the performance. The performance of the delete operation followed the same pattern, with temporal narrowly outperforming triggers.
| Method | Logical Reads | CPU Time | Elapsed Time |
|---|---|---|---|
| Temporal | 5.60 | .0002 | 1.03 |
| Triggers | 5.66 | .0003 | 1.06 |
Making a Choice
Based on the performance test, it doesn’t make a big difference which method you choose. However, as mentioned above, multiple factors go into deciding. For example, does your team hate working with T-SQL? After typing that, it hurts to admit such people exist. Well, triggers might not be the best option. Do you need to track history on hundreds of tables? Again, creating and maintaining triggers add overhead down the road. Maybe you have under 30 tables and don’t expect to add any and want the ability to add some customization to the history tracking process. You might consider going with triggers over enabling temporal.
We make decisions based on our past experiences. It’s understandable if triggers rubbed you the wrong way, ultimately avoiding them. Let me know in the comments below what method you use to track history in a table.
Key Takeaways
- Both triggers and temporal are valid approaches to tracking table history.
- Triggers allow adding custom logic, which temporal tables lack.
- Temporal tables are easier to add at scale compared to creating triggers.
- Do you have a strict data retention policy in place? Temporal tables add extra complexity because of the need to disable them when deleting data.
Next Steps
- Are you interested in learning more about SQL Server triggers? Aubrey Love wrote an article to help, An Introduction to SQL Triggers.
- Do you want to explore enabling temporal tables in your environment? Koen Verbeeck wrote an informative article titled, Introduction to SQL Server Temporal Tables.
- Ameena Lalani wrote a detailed article SQL Server Temporal Tables vs Change Data Capture vs Change Tracking – part 1, comparing the three methods.
