The ROI of DevOps: 5 Key Metrics

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

By Zubin Irani

“DevOps” is short for “Development and Operations,” but there is nothing concise about this term. DevOps has become so loaded, complex, and contentious that I can’t discuss the return on investment (ROI) without defining it first.

Think about DevOps this way: there are martial arts, there is jujutsu, and there is judo, an offshoot of jujutsu that you saw in the Rio Olympics. Software development is martial arts, Agile is jujutsu, and DevOps is judo.

Six traits distinguish DevOps and produce the ROI we seek to measure:

1. Continuous Integration (CI): A process in which developers and testers validate new code.

2. Continuous Delivery (CD): The process of creating releasable artifacts.

3. Dynamic Cloud infrastructure: The virtualization of infrastructure that optimizes computing resources.

4. Test Automation: The scripts that carry out both functional and interface testing.

5. Security Automation: The scripts that perform security checks, increasingly known as DevSecOps

6. Monitoring: Constant measurement of the environment to proactively address issues

Development, operations, and quality assurance (QA) work collaboratively to make DevOps possible. They do DevOps not because it’s hip and mystifying, but rather to meet business objectives. Let’s be frank: companies spend a boatload on development expecting to make a ship-ton of money.

To compete in a crowded app store or enterprise market, tech companies deliver novel products rapidly, update them continuously, and maintain the quality rigorously. By doing so, the company achieves returns on investment that the old “Waterfall model” or basic Agile methods can’t deliver. DevOps organizations should measure five categories of ROI:

1. Faster time-to-market

Some Agile teams within DevOps organizations release software multiple times a day, while others do so less frequently. Market conditions and strategy dictate the pace. But sometimes, companies need to race from idea to production before their competitors do. The ROI of releasing first can be worth billions of dollars. In those races, I bet my money on the company that practices DevOps.

Measuring time-to-market is straightforward. Compare your cycle times under the Waterfall model versus DevOps. Your average cycle is a good proxy for what your team could do in a race-to-market situation. As the ancient Greek poet Archilochus put it, “We don’t rise to the level of our expectations, we fall to the level of our training.”

2. Fewer employees and higher productivity

You can look at DevOps and talent in a glass-half-empty or glass-half-full way. Pessimists might say that DevOps destroys jobs because it enables companies to release more software with fewer people. Optimists might say that DevOps saves companies from hiring unneeded staff.

Both views are valid, and whether you cut people or never add them, DevOps saves money. You will see the most dramatic differences on your ops team. With DevOps, the same number of people can manage 100 servers or 1,000. Automation makes the differences in scale unimportant.

3. No Downtime

Flawed code and overburdened infrastructure lead to downtime, which can jeopardize thousands of dollars per second. Of course, downtime is only a cost if it happens. So, most companies wait until they’ve lost several million dollars to downtime before investing in DevOps. I can’t blame them. How many people do you know who own a backup generator for power outages? Unless you’ve suffered three nights without electricity, you won’t join the generator club.

Continuous Integration and Continuous Delivery (CI/CD), paired with automated testing and security checks, prevent downtime by producing exceptionally stable code. Moreover, a dynamic cloud-based infrastructure that can scale up on demand and automatically account for system failures mitigates the risks of fluctuations in traffic. By automating the assignment and reassignment of servers and containers, DevOps eliminates bottlenecks that otherwise crash systems.

Just as importantly, DevOps fights downtime by utilizing “microservices,” one of the most important evolutions in system architecture in years and a key tool of many a DevOps team. Software platforms used to be one monolithic system. If the log-in service stopped working or became overwhelmed, the rest of the platform went down. Now, DevOps organizations create tens and even hundreds of individual microservices, all linked by APIs. If one microservice fails, the rest continue to function as long as the platform is designed correctly. Thus, microservices can reduce downtime and eliminate the pain of outages.

4. Reduced infrastructure costs

Microservices are not only more reliable but significantly less expensive. You can build your application to apply extra horsepower only to where you need it. In addition, using containers, you can squeeze out every last bit of efficiency to maximize resource utilization per instance. Combined with a dynamic cloud infrastructure, you can set your collection of instances and containers to expand and contract automatically as your workflows ebb and flow.

A perfect example is Amazon.com. During the holidays, they have to handle a huge influx of traffic, so additional server instances are created on demand. They may discover that Tickle Me Elmo becomes the hot toy everyone wants, so they can add more computing power to that one area of their system. Amazon only spends money on infrastructure where and when they need it.

Again, to measure the ROI of DevOps, look at change over time. What did you spend on virtual machines in the dark ages (i.e. the Waterfall years)? What do you spend now? Take into account the size of your user base and extent of the services you offer.

5. Improved quality and performance of applications

Quality and performance are elusive traits with quantitative implications. They affect three types of metrics: service tickets, usage patterns, and demand signals. Depending on your business, you will measure these differently.

Let’s say you make enterprise apps for internal clients. How many service tickets do you generate in the first 48 hours after a release? How do end users interact with new features? Keep in mind that if users don’t interact with your new feature, it won’t generate any tickets. How many existing users download the newest release and how quickly?

Conversely, if you make enterprise apps for external clients, look at customer service activity, usage patterns, and sales activity. For consumer apps, you might look at App Store ratings, usage patterns, downloads, and app store conversion rates.

For the Time-Challenged

The ROI from implementing DevOps is complicated. Because DevOps improves so many of the cost and revenue aspects of operations, companies struggle to calculate all the upside. If you’re a time-strapped CTO or CIO though, the benefits are clear. Success is more stable, secure code that is released more often.

At any level of IT, teams will reap rewards from improved speed, quality, and consistency of DevOps methods. There is nothing loaded, complex, or contentious about that.

About the Author:

Zubin Irani is co-founder and CEO of cPrime, a full-service consultancy that implements Agile transformations and delivers Agile solutions for more than 50 Fortune 100 firms and many of Silicon Valley’s biggest employers.