DevOps Process Glossary

In this model of instructing the automation robot, instead of specifying steps to perform a job, we provide our desired end state of any element to be managed by the automation robot. For example: instead of telling the robot to pick up a red paint and paint the wall with it, we will say “I wish that you ensure this wall becomes red in color”. With that kind of instruction a robot can evaluate current color of the wall, decide if it is different from red, and then calculate what steps it needs to take to convert the wall to red color, and after calculation, executes the decided steps, after executing steps, it will revalidate if now the wall has indeed become red or not, and then confirm back to us that the wall is red. The one major takeaway from this approach is to understand what would have happened if the wall was already red. In that case, the robot would have found the wall to be already red and it would have just informed us that our wish is already fulfilled as it is red already. And then it would have continued to check every now and then, because we have asked it to ensure that it remains red. But it would have never tried to repaint the wall. Contrast that with the other instruction approach where we just ask it to pick up red color and paint the wall. It would not even bother if the wall was already red. It would still pick up the color and start painting even if it was already red. Also after painting once, it would never continue to check, because it has performed the instruction already and there is nothing in instruction for it to continue checking (no mention of ensure in instruction).

Similar to that, in IT automation there are procedural languages like ruby, python where you need to specify steps and the programming language executes the steps as it is. And then there are programming languages like puppet where you dont need to specify all steps but have to express your desire. Such as “I wish to have apache software installed on all my 1000 servers” or “I wish to ensure that all my 1000 servers will have ntp service enabled”. A tool like puppet can read such desired state instructions and ensure that all of your 1000 servers follow the same. In servers where ntp is already running, it will skip them. And on the ones, ntp is not running, it will start them. Had it been ruby/python, you had to specify all steps for all servers, and then these programs would have forced start of ntp even on servers where ntp would have been running already as the ruby/python coding languages do not have the inbuilt infrastructure management intelligence that can be seen in tools like puppet.

Idempotency refers to the behavior of a program, which even if we run multiple times, does not change the outcome.

For example: Puppet follows desired state definition. If any target environment is already compliant to the desired state, it does not force run the commands anymore on the same, and rather skips that environment. This behavior is known as idempotency. Example: If the instruction to puppet is to ensure that ntp service is running across all servers, then it will skip the servers where ntp is already running.

Think about difference between these two commands on linux shell:

echo “hello” > test.txt
echo “hello” >> test2.txt
If you run both commands 10 times, you will end up with a test2.txt that has the word “hello” written 10 times in it. Whereas test.txt will have “hello” written only once. So across the 10 executions, every time test2.txt got different values – 1 line of hello, 2 lines of hello, 3 lines of hello …. 10 lines of hello. But test.txt outcome always remained same – 1 line of hello – no matter how many times you run the first command. Hence, first command is idempotent, and second command is non idempotent.

When an infrastructure environment (compute, storage, network etc.) is completely controlled by code, you can also in the reverse way say that the code defines the infrastructure environment. Because once you have written an automation that launches a virtual machine for you, you can say that the virtual machine is actually defined by the code. In this model the objective is to create a code set that defines the setup of all components of your infrastructure.

So lets say, in your datacenter you have implemented VMWare Virtualization. And then you have written a code that once launched can create all the VMs and network routes on VMWare. So now you have a code that defines your entire infrastructure on VMWare. Think of a scenario, when for some reason you need to rebuild that same set of VMs and Networks on another VMWare setup in another datacenter. So all you need is another VMWare installation in another datacenter as your only pre-requisite. Once you get that, you can run your existing code on the new VMWare setup and voila! your entire infrastructure setup on VMWare is recreated through execution of your code ‘automatically’. This is what is called as managing infrastructure as code.

Another use of infrastructure as code is infrastructure versioning. Lets say you run an application on 10 servers using an infrastructure code which we will call as v1. Then you decided to reduce the number of VMs to 3. So you modified the infrastructure code to delete 7 vms and keep only 3 – we will call this version of code as v2. Then you launched v2 and it reduced number of servers. After a while you realized that it has been a bad decision as the application is not able to perform properly on 3 servers. So you want to go back to situation where you had 10 servers. In the infrastructure as code management practice, this would just mean rolling back to previous version of code and launching the same to regain the same previous setup of 10 servers. Hence the infrastructure designs and architectures become version controlled.

When we implement a devops pipeline using a CI/CD tool such as jenkins, we typically integrate it such a way with the source management systems such as git, so that any new code checked into git triggers the pipeline workflow on the jenkins. And therefore for every new code chevked in, we have code integrations happening continuously, builds getting created continuously. We call these as continuous integration and continuous build processes. We similarly also have continuous testing process which again means that for every new code check in, we create the build, run the application and perform all tests on that application as necessary. But before you can do the application testing, you need to have the application up and running on some machines.

Which brings us to creating a new test server whenever such testing is required. Hence for continuous testing, we will have continuous provisioning of servers.

Now provisioning a server is not enough for running an application. For example, if you try to run a java application directly on just a fresh installation of ubuntu server, it wont work. Because a java application needs something more than the operating system to run. This middle level of software layer required by application software layer on top of the operating system layer is said to be the middleware. Different applications require different middlewares. For example java applications require JRE but .NET applications require .NET runtime environment.

Setting up of all these runtime environments, middleware, databases – are called as configuration of the operating system. When you automate that with tools and programming languages like puppet, we call that as configuration automation. When such configuration automation happens as part of continuous provisioning process as part of a CI/CD pipeline, it is referred to as Continuous Configuration Automation.