Problem
I have been hearing a lot about DevOps. Can you tell me what this is and how to implement?
Solution
In this tip we will explain what DevOps is and the relationship between DevOps and SQL Server.
What is DevOps?
DevOps is a new philosophy to integrate Developers and Operators (IT) operations. The main idea is to coordinate between developers and operators to automate tasks and generate releases of software products more often and more efficiently.
The idea is to improve the collaboration between these two teams to generate and optimize integration, testing and releases.
DevOps emphasizes automation and monitoring to release the software in shorter cycles.
DevOps tries to solve a frequent problem when developers have a new release and the IT team is a bottleneck to release software updates to servers of the company.
When was the term DevOps created?
The concept started to be popular in 2009, but was created in 2008 and the term was coined in 2009.
Who is the creator of the DevOps culture?
Patrick Debois and Andrew Shafer started this new concept at the Agile Toronto Conference.
What is the relationship between DevOps and SQL Server?
DevOps is a philosophy to integrate developers and operators. If you are a developer in T-SQL or if you have an application that uses a SQL Server database, you may need to coordinate with the IT team to create software releases. Or if you are an IT member, you may need to coordinate with developers. If you need to optimize your development and operations, then DevOps is for you.
Why should I consider DevOps?
If the deliveries depend on the operation department (IT) and the interaction between developers and IT is a bottleneck in your company, you should learn how DevOps can improve the time to release new versions of your software.
DevOps will help you to do the following:
- Code versioning
- Track code changes
- Automate software building
- Automate releases and versioning
- Automate infrastructure configuration
- Automate monitoring tools for the software
How do I get started with DevOps?
I recommend starting with this video as an introduction:
Once you watch this video, I recommend reading some books related to DevOps that are mentioned below.
Are there some books related to DevOps?
There are some nice books about DevOps.
One of my favorite books is listed below. This book was written by Patrick Debois. One of the founders of DevOps and also other authors like Gene Kim, John Willis and Jez Humble.
Another interesting book is this one:
There are other books like these:
- The DevOps Adoption Playbook: A Guide to Adopting DevOps in a Multi-Speed IT Enterprise
- DevOps: 2 Manuscripts – DevOps Handbook and DevOps Adoption
- Effective DevOps: Building a Culture of Collaboration, Affinity, and Tooling at Scale
- DevOps: A Software Architect's Perspective (SEI Series in Software Engineering)
- Leading the Transformation: Applying Agile and DevOps Principles at Scale
How do I move from my current methodology to DevOps?
If you are using Agile, DevOps is compatible and we can say that is part of Agile.
Some of the questions you may ask are:
- Are the software release operations automated?
- Are the monitoring operations automated?
- Are the infrastructure configuration operations automated?
If you think it’s possible to automate the operation and it will bring a benefit to software operations, you should create a plan to gradually automate some operations. Do not migrate all your operations at once. Gradually automate some operations and when you think that the operation is stable, move to the next one.
Who is responsible for DevOps in the organization?
Usually, the same developers are responsible to automate the operations with the help of IT. You can hire an external consultant to improve the learning curve and automate your operations faster. You can hire an external DevOps engineer who can help your team to automate IT and developer operations. Depending on your requirements you can hire a DevOps engineer permanently or for some months.
What are the DevOps Engineering skills?
Usually these guys are good with scripting, using open-source tools and great knowledge on OSs like Linux and databases.
They are also good at automation using shells like C shell, PowerShell, bash, etc. They are also usually good with Perl, Python, Groovy and Java. Depending on your needs the languages required may be very different.
What does a DevOps team look like?
It depends on what you have in your company. If your current team does not have good automation skills, you need to plan to train them in DevOps and process automation. You may also need to hire more external resources temporarily or permanently to help the current team until all the development, quality control and monitoring processes are automated.
A good DevOps team should have both IT and, development knowledge. The IT and developers should have more common tasks and meetings together to coordinate. Both teams should have common goals and more activities together.
How are changes made in Dev, Test and Prod with DevOps?
In development you should have tools to track changes and versions to deploy automatically.
In testing you should have automation test cases to test functionality and performance to make sure the new release did not produce regression bugs.
In production you should be able to deploy automatically usually using Docker and monitor that the new version is stable and report to all the members of the team when it is deployed including the changes and a report of the test cases results.
How does DevOps help the DBA, Developer, IT Department, users, company, etc.?
All the teams will learn new skills to perform the tasks more efficiently using automation tools in different stages of the software development. Another advantage is that the DevOps culture will help coordination between developers and IT members and DBAs.
If quality assurance and monitoring is automated, the releases will have less errors and if the deployment is automated, the IT department will have less tasks and deliver releases in less time. The company will have the software in less time and the users will have less errors. Everybody wins with DevOps.
What are the Pros and Cons of DevOps?
The pros are that the automation should improve the development time and decrease resources needed. The software development will be more efficient.
In some companies, software changes a lot and automating the process is not possible because the software is changed constantly and the old test cases and scripts to deploy are no longer valid. If your software drastically changes a lot and you believe the automation effort is high, but it will need to be developed again in new releases in few months, you need to analyze if DevOps is good for your company.
When would you not want to use DevOps?
If you are in a development company, you will always want to integrate Developers and Operations to improve speed. However, depending on your type of software, some things cannot be automated or at least cannot be automated easily.
For example, in a company, to install the software manually requires 8 hours, but to create an automatic easy to install deployment requires 3 months. That company only has 2-3 customers per year, then it is not cost-effective to automate the setup of the software.
In another case, we want to automate some test cases for the UI, but the Development leader told us that they will change the UI from a Web to a Desktop application. If you think you are going to radically change some modules in the future, do not automate the process (or automate, but with simple test cases that do not require several months to develop).
In other words, analyze your pros and cons before implementing in different software development stages.
Other companies have specific roles and processes and it is not possible to work with DevOps. In some companies, the functions are very specific and the developers are not allowed to do IT operations and the roles cannot be modified because some standards that are not compatible with the DevOps philosophy. If you want to implement DevOps, in many cases you will need to convince the Chief Technology Officer and the shareholders to implement it. If you do not have permissions from the primary stakeholders of the company, the DevOps implementation may fail.
Does the development process speed up or slow down?
The development process will speed up. Especially in monitoring automatically, detecting errors automatically, and deploying automatically.
However, people do not usually like changes. DevOps requires a lot of extra work, training at the beginning and the development will be slower at the beginning because it will require extra time to train your team and to develop all the scripts, code, install and learn new tools. Once you have implemented the automation of the Development operations it will be faster, but before the implementation it will require a lot of work depending on the job required to have great DevOps practices.
Are there more or less mistakes with DevOps?
At the beginning there will be a lot of problems with DevOps, but after some time, when the team has more experience, there will be less mistakes because everything will be automated.
What are the benefits of DevOps?
The main characteristic is to improve the relationship between developers and operations to release software deliveries faster using automation tools.
Which tools can I use the improve the DevOps operations in SQL Server?
In SQL Server you can use Visual Studio Team Services, SSIS, PowerShell, T-SQL, SQL Server Data Tools, mssql-scripter, bcp, MSBuild, sqlcmd or other tools of your preference.
SSIS let’s you create tasks visually. PowerShell is a powerful shell that can help you automate tasks. Sqlcmd is used to execute T-SQL commands or scripts.
What is CI/CD and what is the relationship with DevOps?
Continuous Integration (CI) and Continuous Deployment (CD) are terms related to integrate the code and deploy easily. If you have simple tools to send your code to deployment, you are reducing the steps to develop software and reducing the effort and coordination time required to synchronize between developers and operators. In SQL Server you can use SQL Server Data Tools and Visual Studio Team Services.
For more information about CI/CD with Microsoft tools, refer to this link:
I heard that Docker can help in the DevOps philosophy. How can Docker help in DevOps?
Docker is a software designed to create portable applications in containers. In a Virtual Machines, we create environments with our database, applications and drivers. A Virtual Machines requires a guest OS and consumes several resources. Docker optimizes the resources required to create portable databases and applications in containers. It requires less resources and it is faster to install. The number of containers supported by a single machine is higher than the number of virtual machines supported.
If you have a portable application, it is easier to deploy your application in production.
For more information about Docker in SQL Server 7 and DevOps, refer to this link:
Which Microsoft certifications are related to DevOps?
Microsoft offers the MPP for DevOps. Microsoft will release a new certification in 2018 the Exam 70-538, Implementing Microsoft Azure DevOps Solutions.
Is DevOps part of the Agile Software Development?
Yes, DevOps was born in an Agile conference. The term was born in a conference named Agile infrastructure in Toronto in 2008. Agile was mainly oriented to developers and software testers, stakeholders, product owners, team leads, but there was not much written about IT and integration with development.
What types of companies use DevOps?
Basically, any development company can apply DevOps in the company. If your company is related to infrastructure, installation, administration, but does not include software development, you may want to read more about some DevOps ideas to automate installation and configuration processes.
What is your experience with DevOps?
In my experience, it is very easy and natural to incorporate the DevOps culture with companies that are familiar with Agile. On the other hand, for government entities, it was harder to implement because they had some inflexible standards that were not compatible with the DevOps philosophy.
Next Steps
For more information about DevOps, refer to these links:
