Scheduling with Azure Automation to Move Data


In this video we look at Azure Automation (aka runbooks) which is a way to schedule tasks on Azure to do things like run PowerShell scripts or other things you want to set on a automate.

Recommended Reading

Video Transcript

Hi everyone, welcome to this video series about moving data around in Azure. My name is Koen Verbeeck, and I work as a Senior BI Consultant in Belgium. If you have any questions about this video, which will be about Azure Automation, you can post them in the comments, or you can contact me on LinkedIn, Twitter, or on my blog.

All right, so, the focus of today's video is about Azure Automation and it'll be a very short video, because it's not an actual Azure Data services, but I included it in the series because you can write code in it and you can schedule it. So, in theory, you can do a lot with it.

So Azure Automation, which used to be called Azure Runbooks, is a way to run pieces of code on a schedule. So you can, for example, run a Python Script or a PowerShell Script. But one of its disadvantages is it doesn't support Visual Studio integrations, which means there is no testing, no easy testing. There's no source control and stuff like that. And so, in theory, you can use it to move data around. You can, for example, write a Python Script that imports data from somewhere and arrives at somewhere else. But the problem is, it's all a bit of a black box. It's hard to see which libraries are installed, it's hard to see which drivers are installed, so it's really a hassle to get something to work in it if you can't see what tools you can use. So it's very hard to work with.

There are other alternatives in Azure that are much easier to get data around. So try to avoid this one for just moving data around. However, it's really good at automating administrative tasks. For example, you can use Azure Automation, each morning, you check, okay, do I need to scale up my data warehouse? And at the evening, after it has run, do I need to scale it down? Are there any virtual machines running and can I shut them down so I can save? 'Cause these are the types of tasks when you use Azure Automation for but not really to move data around, even though it's theoretically possible.

All right, that was it. I told you it's gonna be a short video. If you have any more questions, just post them in the comments. And I will see you at the next video. Thank you for watching.

Last Update: 4/27/2020

Comments For This Article


get free sql tips
agree to terms