Automate Database Restore for SQL Server

By:   |   Comments (30)   |   Related: > Restore


Problem

I frequently get requests to refresh a test / dev SQL Server database and need a way to test database backups from the latest full backup of a production database. It’s easy enough to do this with SQL Server Management Studio, but I would rather have an easy to configure script that’s less prone to making a mistake that I could run on demand and / or easily automate rather than do this as a manual process.

Solution

We previously saw how to accomplish this in this tip: Automating a SQL Server Database Refresh, with a little PowerShell, a little T-SQL and SQL Server Agent. In the interest of keeping the configuration simpler and only changed in one script the following is a revised way to do it with just PowerShell and optionally SQL Server Agent.

The following versions were used writing this tip:

  • SQL Server 2017 CU19 Developer Edition
  • SQL Server Management Objects (SMO) installed SQL Server Management Studio 18.4
  • PowerShell 5.1.17763.1007

Configure PowerShell Script

To get started, copy the following PowerShell into the editor of your choice and configuring the variables for your purposes.

# restores a full database backup to another database from source's latest full backup file in specified directory
 
# begin script configuration here
$TargetSqlServerInstance = "JGAVIN-L\SQL2017"                                                                        # target server instance 
$TargetDb = "RefreshTest"                                                                                            # target database 
$BackupDir = "\\jgavin-l\SQL2017_Backup\RefreshProd"                                                                 # directory / share where backups are stored
$SourceLogicalDataFileName = "RefreshProd"                                                                           # logical data file name of source db 
$SourceLogicalLogFileName =  "RefreshProd_log"                                                                       # logical log file name of source db
$TargetLogicalDataFileName = "RefreshTest"                                                                           # logical name you want to change logical data file on target db to
$TargetLogicalLogFileName =  "RefreshTest_log"                                                                       # logical name you want to change logical log file on target db to
$TargetPhysicalDataFileName = "C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017\MSSQL\DATA\RefreshTest.mdf"     # full path\file of target db physical data file 
$TargetPhysicalLogFileName =  "C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017\MSSQL\DATA\RefreshTest_log.mdf" # full path\file of target db physical log file 
$CompatLevel = 140                                                                                                   # compatibility level to set target database to (2019=150, 2017=140, 2016=130, 2014=120, 2012=110, 2008/2008R2=100, 2005=90, 2000=80, 7=70) 
# end script configuration here
 
# import sqlserver module
Import-Module sqlserver
 
# latest full backup file name is dynamically determined and appended to backup directory
$LatestFullBackupFile = Get-ChildItem -Path $BackupDir -Filter *.bak | Sort-Object LastAccessTime -Descending | Select-Object -First 1 
$FileToRestore = $BackupDir + '\' + $LatestFullBackupFile
 
# kill any connections in target database
$KillConnectionsSql=
"
USE master
GO
ALTER DATABASE $TargetDb SET SINGLE_USER WITH ROLLBACK IMMEDIATE
GO
USE master
GO
ALTER DATABASE $TargetDb SET MULTI_USER
GO
USE master
GO
"
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query $KillConnectionsSql 
 
# import sqlserver module
Import-Module sqlserver
 
# restore
$RelocateData = New-Object Microsoft.SqlServer.Management.Smo.RelocateFile("$SourceLogicalDataFileName", "$TargetPhysicalDataFileName")
$RelocateLog  = New-Object Microsoft.SqlServer.Management.Smo.RelocateFile("$SourceLogicalLogFileName",  "$TargetPhysicalLogFileName")
Restore-SqlDatabase -ServerInstance $TargetSqlServerInstance -Database $TargetDb -BackupFile $FileToRestore -RelocateFile @($RelocateData,$RelocateLog) -ReplaceDatabase
# end restore
 
# set db owner to sa
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Database $TargetDb -Query "EXEC sp_changedbowner sa"
 
# set compatibility level
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE $($TargetDb) SET COMPATIBILITY_LEVEL =$($CompatLevel)"  
 
# set recovery model to simple
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE $($TargetDb) SET RECOVERY SIMPLE WITH NO_WAIT"  
 
# rename logical files
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE $TargetDb MODIFY FILE (NAME='$SourceLogicalDataFileName', NEWNAME='$TargetLogicalDataFileName')"
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE $TargetDb MODIFY FILE (NAME='$SourceLogicalLogFileName', NEWNAME='$TargetLogicalLogFileName')"
 
# dbcccheckdb
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "DBCC checkdb ($TargetDb) --WITH NO_INFOMSGS"
 
# display sp_helpdb
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "EXEC sp_helpdb $($TargetDb)" 

The following table lists each configurable variable and its description:

Variable Description
$TargetSqlServerInstance Name of SQL Server with database we’re restoring to in the form: MachineName or MachineName\InstanceName
$TargetDb Name of target database we’re restoring to
$BackupDir Path to local drive or UNC share with backup files in the form: \\ServerName\ShareName or DriveLetter:\DirectoryName
$SourceLogicalDataFileName Logical data file name of source database
$SourceLogicalLogFileName Logical log file name of source database
$TargetLogicalDataFileName Logical data file name of target database
$TargetLogicalLogFileName Logical data file name of target database
$TargetPhysicalDataFileName Fully qualified path to physical data file of target database
$SourceLogicalLogFileName Fully qualified path to physical log file of target database
$CompatLevel Compatibility Level we want for target database

To obtain the logical and physical file names for the source and target databases:

From Object Explorer in SQL Server Management Studio:

  1. Right click on source database
  2. Properties
Obtain source database logical file names
  1. Files
Source database logical file names

Repeat for the target database.

  1. Right click on source database
  2. Properties
Obtain target database logical and physical file names
  1. Files
Target database logical and physical file names

Our source database full and transaction log backups are saved in the C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017\MSSQL\Backup\RefreshProd directory. Looking at the time and date stamps, we see that RefreshProd_full_backup_2.bak is the latest full backup, and that’s the one we want to refresh the test database with. The fully qualified name is determined by the string in $BackupDir appended with the dynamically generated string in $LatestFullBackupFile.

Source database backup files

Execute Script

After configuring the variables, I’ve saved the PowerShell as C:\DbaScripts\RefreshTestDb.ps1. Execute it and we see the output of sp_helpdb on our newly refreshed database.

RestoreTestDb execution

The SQL Server errorlog shows us:

  1. C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017\MSSQL\Backup\RefreshProd, RefreshProd_full_backup_2.bak was restored
  2. Restore completed
  3. Compatibility Level was set
  4. Database recovery model set to SIMPLE
  5. DBCC CHECKDB ran without errors
errorlog

Automate Script in a SQL Server Agent Job

Now, we’ll automate it in a SQL Server Agent Job.

  1. Expand SQL Server Agent dropdown
  2. Right click Jobs
  3. New Job…
New SQL Agent Job
  1. Name Job
  2. Set Job owner
  3. Enter Job description
  4. Steps
New Job
  1. New
New Step
  1. Name Step
  2. Choose Operating System (CmdExec) in Type dropdown
  3. Enter PowerShell executable name with a -File=FullyQualifiedScriptName
  4. OK
New Job Step
  1. Schedules
  2. New
New Job Schedule
  1. Name Job Schedule
  2. Choose Frequency
  3. OK
Configure New Job Schedule
  1. OK
Create Job
  1. Expand Jobs
  2. Right click on new Job
  3. Start Job at Step…
Execute Job

Look for Success in Status column.

Job Status
  1. Right click on Job
  2. View History
View Job History
Job History
Job Step History
Next Steps

Here are links to tips with a wealth of information on the following related topics:



sql server categories

sql server webinars

subscribe to mssqltips

sql server tutorials

sql server white papers

next tip



About the author
MSSQLTips author Joe Gavin Joe Gavin is from the Greater Boston area. He started working with SQL Server and Sybase in 1998 in the financial services industry and has been a SQL Server Database Administrator for a dairy cooperative since 2011. He graduated from Northeastern University in Boston with a Bachelor of Science in Engineering Technology (BSET) degree in Computer Technology. Joe has spoken at the Boston and Providence SQL Saturday events.

This author pledges the content of this article is based on professional experience and not AI generated.

View all my tips



Comments For This Article




Tuesday, April 9, 2024 - 2:26:40 PM - Joe Gavin Back To Top (92163)
Ivan, I'm guessing the account you're running this doesn't have right to the backup directory. If you step through the script manually, after this line '$LatestFullBackupFile = Get-ChildItem -Path $BackupDir -Filter *.bak | Sort-Object LastAccessTime -Descending | Select-Object -First 1', $LatestFullBackupFile is likely empty.

Tuesday, April 9, 2024 - 7:47:44 AM - Ivan Dave Fajardo Agustino Back To Top (92160)
Hi, Just want to ask about my error while doing this script. Actually, this script worked previously on the sql servers, however, when i changed the target SQL servers, i received an error message indicating that Error: 18204, Severity 16, State:1
BackupDiskFile:OpenMedia: BackupDevice "path of backups" failed to open. Operating System Error 123(the filename, , the directory name, or volume label syntax is incorrect)

Kindly help me with this one. Thank you

Monday, June 5, 2023 - 1:33:51 PM - Joe Gavin Back To Top (91251)
Thanks Abel. Stay tuned, there may be another tip coming that will include restoring full plus differential backups.

Friday, May 26, 2023 - 3:52:50 PM - Abel Back To Top (91228)
Hello,

Thanks for the information, greately appreciated, just have question, what would you need to add if I would want to restore a full and a diferential.

Thanks again.

Tuesday, January 10, 2023 - 12:25:32 PM - Joe Gavin Back To Top (90818)
Excellent Mando. Glad the fix was easy.

Tuesday, January 10, 2023 - 9:14:37 AM - Mando Back To Top (90816)
That was it, Joe! Thanks so much. Now trying to get this to work on databases with multiple data files...

Monday, January 9, 2023 - 2:38:12 PM - Joe Gavin Back To Top (90810)
Mando, I'm suspecting you may have omitted the -ReplaceDatabase.

Friday, January 6, 2023 - 10:07:34 AM - Mando Back To Top (90800)
Glenn's issue is his db name has a '-' in it.

I'm working through using this script myself and currently getting this error:
"
Restore-SqlDatabase : System.Data.SqlClient.SqlError: The file 'F:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\MSSQL\DATA\[DBNAME].mdf' cannot be overwritten. It is being used
by database '[DBNAME]'.
"

Any ideas?

Tuesday, January 18, 2022 - 11:45:41 AM - Joe Gavin Back To Top (89671)
C. Uerel, the database will be overwritten. No need to drop drop and recreate it.

Tuesday, January 18, 2022 - 10:08:45 AM - C. Uerel Back To Top (89669)
if a database was restored from an earlier backup and thus already exists on the SQL server, does this procedure overwrite that db with the newest bak file or do I have to delete that db first by an additional script?

Friday, December 17, 2021 - 11:26:51 AM - Joe Gavin Back To Top (89600)
Glenn, I can't really tell where the issue is. I did copy and paste the PowerShell from the tip again, configured it and tested it successfully. Guessing you may have inadvertently have an an additional character or 2 that's breaking things. I'd suggest copying and pasting it again.

Thursday, December 16, 2021 - 10:21:31 AM - Glenn Back To Top (89597)
Hello,

I'm trying to run the script but we are getting the below error, anyone has a idea?

Invoke-Sqlcmd : Incorrect syntax near 'GO'.
Incorrect syntax near '-'.
Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an
xmlnamespaces clause or a change tracking context clause, the previous statement must be
terminated with a semicolon.
Incorrect syntax near 'IMMEDIATE'.
Incorrect syntax near '-'.
'MULTI_USER' is not a recognized SET option.
Msg 102, Level 15, State 1, Procedure , Line 1.
+ Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query $KillCo ...

Monday, June 28, 2021 - 10:46:08 AM - Joe Gavin Back To Top (88914)
Good question Jack. That's a little out of scope for these requirements. I put this together based on a common request I get to restore test/dev from the previous nights' full backup. Don't know off the top of my head how I'd go about with this script. The dbatools Restore-DbaDatabase cmdlet is probably the best place to start.

Monday, June 28, 2021 - 7:14:51 AM - jack h Back To Top (88911)
how would you include all the subsequent log files so we always restore the latest copy from production?

tyia!

Thursday, April 29, 2021 - 10:01:07 PM - Joey Back To Top (88629)
Thanks Joe. you are very helpful.

Tuesday, April 27, 2021 - 4:33:33 PM - Joe Gavin Back To Top (88617)
I have Joey. The reason is with SQL Server (and Sybase for that matter), logins to the SQL Server and users in a database are 2 separate but related entities tied together with a number called a sid. If you're restoring the backup to the same SQL Server or another one where the logins were added in the same order, the sids will match, and you're all set. Here are a couple of tips that will give you some more detail:

https://www.mssqltips.com/sqlservertip/1063/sql-server-database-restores-mapping-users-to-logins/

https://www.mssqltips.com/sqlservertip/1590/understanding-and-dealing-with-orphaned-users-in-a-sql-server-database/


Tuesday, April 27, 2021 - 12:19:18 AM - Joey Back To Top (88607)
Thanks Joe. Do you come across the user permission issue or login problem after the full backup restore? even the dev database already has the same user logins created as the production database.

Friday, April 23, 2021 - 1:43:25 PM - Joe Gavin Back To Top (88598)
You are 100% correct Joey. I frequently get requests to restore a dev database from the previous night's full backup of production so that was my basic requirement for this tip.

Friday, April 23, 2021 - 12:48:01 AM - Joey Back To Top (88591)
seems like your script only restore the bak file, it does not restore the subsequent (transaction logs files *.trn). Please correct me if I am wrong.

Wednesday, February 10, 2021 - 9:35:24 AM - Ima Silva Back To Top (88207)
Joe and Greg, no worries and thank you, This script helps me a a lot and saved me loads of time, exactly I was after , keep up the good work!

Wednesday, February 10, 2021 - 9:26:35 AM - Greg Robidoux Back To Top (88206)
The code has been updated.

-Greg

Wednesday, February 10, 2021 - 9:12:29 AM - Joe Gavin Back To Top (88205)
Ima, sorry you had problems with this. Thanks for pointing it out.

Greg,

I confirmed the following lines had soem spaces removed:


ALTER DATABASE$TargetDb SET SINGLE_USER WITH ROLLBACK IMMEDIATE

ALTER DATABASE$TargetDb SET MULTI_USER

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE$($TargetDb) SET COMPATIBILITY_LEVEL =$($CompatLevel)"

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE$($TargetDb) SET RECOVERY SIMPLE WITH NO_WAIT"

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE$TargetDb MODIFY FILE (NAME='$SourceLogicalDataFileName', NEWNAME='$TargetLogicalDataFileName')"

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE$TargetDb MODIFY FILE (NAME='$SourceLogicalLogFileName', NEWNAME='$TargetLogicalLogFileName')"

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "EXEC sp_helpdb$($TargetDb)"


Here are the edited lines:


ALTER DATABASE $TargetDb SET SINGLE_USER WITH ROLLBACK IMMEDIATE

ALTER DATABASE $TargetDb SET MULTI_USER

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE $($TargetDb) SET COMPATIBILITY_LEVEL = $($CompatLevel)"

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE $($TargetDb) SET RECOVERY SIMPLE WITH NO_WAIT"

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE $TargetDb MODIFY FILE (NAME='$SourceLogicalDataFileName', NEWNAME='$TargetLogicalDataFileName')"

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "ALTER DATABASE $TargetDb MODIFY FILE (NAME='$SourceLogicalLogFileName', NEWNAME='$TargetLogicalLogFileName')"

Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query "EXEC sp_helpdb $($TargetDb)"

I can send you the whole scripts with edits if you like.

Tuesday, February 9, 2021 - 5:04:24 PM - Greg Robidoux Back To Top (88201)
Joe, let me know if the script needs to be updated. It is possible the formatting got messed up when it was converted.

Thanks
Greg

Tuesday, February 9, 2021 - 4:52:05 PM - Ima Silva Back To Top (88200)
Great article, thanks very much!

In case if it helps anyone I wasted couple hours due to errors like below

Logical file xxx is not part of database yyy..
Unkown object type 'xxx' used in a create, drop..

Turned out I had to add spaces in multiple places in the script

DATABASE$($TargetDb) -->DATABASE $($TargetDb)
DATABASE$TargetDb --> DATABASE $TargetDb
sp_helpdb$($TargetDb) --> sp_helpdb $($TargetDb)


Monday, January 4, 2021 - 10:29:44 AM - Joe Gavin Back To Top (87996)
Thanks Mike. Glad you like it.

I'm presuming we're talking about multiple full backup files in one directory and you want to test each file. In that case I'd do something like this to build a list of backup files in order then restore each of them within a ForEach.


# build list of backup files in order from oldest to newest
$BackupFiles = Get-ChildItem -Path $BackupDir -Filter *.bak | Sort-Object LastAccessTime

ForEach($FileToRestore in $BackupFiles)
{
# kill any connections in target database
$KillConnectionsSql=
"
USE master
GO
ALTER DATABASE $TargetDb SET SINGLE_USER WITH ROLLBACK IMMEDIATE
GO
USE master
GO
ALTER DATABASE $TargetDb SET MULTI_USER
GO
USE master
GO
"
Invoke-Sqlcmd -ServerInstance $TargetSqlServerInstance -Query $KillConnectionsSql

# restore
$RelocateData = New-Object Microsoft.SqlServer.Management.Smo.RelocateFile("$SourceLogicalDataFileName", "$TargetPhysicalDataFileName")
$RelocateLog = New-Object Microsoft.SqlServer.Management.Smo.RelocateFile("$SourceLogicalLogFileName", "$TargetPhysicalLogFileName")
Restore-SqlDatabase -ServerInstance $TargetSqlServerInstance -Database $TargetDb -BackupFile $BackupDir\$FileToRestore -RelocateFile @($RelocateData,$RelocateLog) -ReplaceDatabase
# end restore
}

Thursday, December 31, 2020 - 2:38:02 PM - Mike Back To Top (87991)
Great article. How would we change this to include multiple bak files? I currently back up a few databases to small bak files to accommodate size restrictions and online storage. My largest is currently 16 seperate files numbered accordingly (.bak1, .bak2, etc)

Monday, August 10, 2020 - 9:17:50 AM - Joe Gavin Back To Top (86265)
Thanks Simon. Definitely considered dbatools. They're fantastic. We decided a requirement for this tip was to be be purely native with nothing additional installed.

Sunday, August 9, 2020 - 9:24:14 AM - Simon Back To Top (86260)
Hi, nice article. Have you considered using the dbatools.io Powershell suite ?

Monday, July 6, 2020 - 9:58:35 AM - Joe Gavin Back To Top (86091)

Thanks Robert.


Monday, July 6, 2020 - 8:39:30 AM - robert rogers Back To Top (86090)

I'm not a big fan of PowerShell when it comes to SQL Server...but this great post. Thanks for write up.















get free sql tips
agree to terms