AWS CloudFormation: Using Infrastructure as Code

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

AWS CloudFormation is a leader in Infrastructure as Code (IAC), which greatly eases the process of provisioning IT resources in the cloud.

First, some background:

Cloud Storage and Backup Benefits

Protecting your company’s data is critical. Cloud storage with automated backup is scalable, flexible and provides peace of mind. Cobalt Iron’s enterprise-grade backup and recovery solution is known for its hands-free automation and reliability, at a lower cost. Cloud backup that just works.

SCHEDULE FREE CONSULT/DEMO

Between the shortage of skilled talent, the move to the cloud and good old human error, computing has increasingly relied on automation to perform tasks otherwise requiring human intervention.

Cloud service providers like Amazon Web Services and Microsoft Azure make it possible to spin up a virtual server in under 20 minutes. But it’s one thing to spin up an EC2 instance. It’s another thing with the complex smorgasbord of services these providers offer, getting them all interconnected and working properly.

This is where Infrastructure as Code (IAC) comes in. IAC is a type of IT infrastructure provisioning process where systems are automatically built, managed and provisioned through code execution rather than scripting or a manual process.

By using code to automate the process of setting up and configuring a virtual machine or container, you have a fast and repeatable method for replicating the process. That’s why Infrastructure as Code is sometimes referred to as programmable infrastructure. You can rebuild the exact same virtual environment over and over again for developer scenarios, for example.

IAC isn’t all that different from scripting in terms of automating an IT process, it’s just that scripts are not as flexible as programming languages. IAC gives you the versatility of code in a script-like environment.

IAC is used in an environment that runs DevOps processes and use Agile development methodologies. It’s after that, you need the language of choice, and for most, it is either Chef or Puppet. Both languages are made for IAC. Chef is designed for fast collaboration in a DevOps environment and Puppet is used primarily for automating the process of building an infrastructure.

“Infrastructure as Code enables both visibility and reproducibility for an organization,” said Todd Hildebrant, senior manager with consulting giant PricewaterhouseCoopers. “It allows that the same processes and repositories used for managing code and changes to be used to manage the entire infrastructure. This allows the actual environment that the code executes within to be considered as part of the life cycle of the application.”

Plus, infrastructure-declaring languages or frameworks like Chef and Puppet, along with Ansible and Fugue, can work as easily as on-premises as on AWS. So in addition to running on AWS, customers using CloudFormation templates can also use them internally on their own data center to standardize as much or as little of their existing architectural patterns as they wish.

Chef is both the name of a company and the product. The language is written in Ruby and is designed entirely to be a configuration management tool. Chef is used to streamline the task of configuring and maintaining a company’s servers. Puppet operates under the same principal as Chef, in that it uses a declarative language to describe system resources and their configuration, except it is written in C++.

Enter AWS CloudFormation

Amazon was the first cloud vendor to offer IAC through CloudFormation, which it launched in 2011. CloudFormation provides a common language for you to describe and provision all the infrastructure resources in your cloud environment using a simple text file to model and provision all the resources needed for your applications across all regions and accounts.

CloudFormation has two parts: templates and stacks. A template is a JavaScript Object Notation (JSON) text file, which defines what AWS resources or non-AWS resources are required to run the application. For example, the template may declare that the application requires an Amazon Elastic Compute Cloud (EC2) instance with Simple Storage Service (S3) and MongoDB database.

Recently, Amazon has added the option of YAML (Yet Another Markup Language or YAML Ain’t a Markup Language, depending on your sense of humor) for CloudFormation. The basic opinion of the two is that JSON is easier for computers to parse and generate, while YAML is easier to read. It comes down to preference, but at least developers have the option of the two.

Since YAML is a superset of JSON, any existing JSON can easily be converted to YAML with the right tools. So existing JSON-based IAC templates can be converted right over. However, there is no YAML-to-JSON conversion path, so once you convert them, they won’t be convertable back to JSON.

When the template is submitted to AWS, CloudFormation creates the necessary resources in the customer’s account and builds an instance based on the requirements of the template, putting dependencies and data flows in the right order automatically.

The running instance is called a stack. A stack is a collection of AWS resources that you can manage as a single unit. Customers can make changes to the stack after it’s been deployed using CloudFormation as well. When a stack is deleted, all related resources are deleted automatically as well.

You can build your own templates or choose from a collection of dozens of templates provided by Amazon for free. You only pay for the resources you use, not the template.

“The strength of Amazon’s offering is that it has such tight integration with all the AWS services,” said Hildebrant. “Almost anything that can be done in AWS can also be scripted using CloudFormation. This enables customers to use a template to experiment with creating an application, to use templates that are compliant with best practices, or to record their existing architecture as a template. Moving a complete application stack to a new region now only requires changing a few variables in order to have exact reproducibility.”

In addition to all of the templates from Amazon, there are also many third-party products designed to work with CloudFoundry, from software vendors like New Relic and CloudReach to free, open source tools such as Stacker, Stack Master and cfn-flow.

CloudFoundry is not without its flaws. It still has some issues despite its relative maturity. For starters, there is a long startup ramp when you first begin making templates. This is because you need to provide a lot of structure from the very first step. After you’ve made a few templates you can copy and paste segments of code from older templates.

Second, it has a long feedback cycle. Templates need to be validated with an external API which is said to be very time consuming, and errors only show up during the runtime, which comes late in the development cycle. And the nature of IAC code is highly repetitive and YAML and JSON don’t support small templates. So you have to repeat yourself, meaning copy and paste, a lot.

Also, Amazon does not suggest or enforce any standards for writing and structuring CloudFoundry templates, so it’s something of an anything-goes environment. That means every developer can have their own standards. So sharing or using other people’s templates as the potential for trouble.

Still, CloudFoundry is a simple way, once you wrangle it into usefulness, to manage your AWS instances and services, and more importantly, shut them all down when you are done.