How AWS CloudFormation Works (and How to Create a Virtual Private Cloud with it)

Here’s an in-depth walkthrough of how CloudFormation works and an analysis of a template that creates a VPC

This is the fifth article in our Infrastructure as Code blog series. Check out our other posts here:

• Infrastructure as Code: 5 Reasons Why You Should Implement IaC Now

• 6 best practices to get the most out of IaC

• 15 Infrastructure as Code tools you can use to automate your deployments

• What is AWS CloudFormation and how can it help your IaC efforts?

• How to incorporate S3, EC2, and IAM in a CloudFormation template

• How to create a Redshift stack with AWS CloudFormation

In our last article, we provided an overview of AWS CloudFormation and how it can help you manage your deployments. In this post, we’re going to dig deeper into CloudFormation, provide a template that we wrote that creates a virtual private cloud (VPC), and dissect how it works.

This is the first of three templates that we’ll break down, with each getting progressively more complex. In the next two posts, we’ll highlight how we built the CloudFormation templates for our product SFTP Gateway and a Redshift stack. To be notified when future posts go live, click here.

Let’s go!

The anatomy of a CloudFormation template

Here’s a super simple CloudFormation template that creates an S3 bucket:

AWSTemplateFormatVersion: 2010-09-09
Description: Bucket Stack
Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: my-bucket

You can create CloudFormation templates using JSON or YAML. We prefer the latter, so all the templates included in this article are in YAML. Regardless of which you choose, templates can consist of the primary sections of information highlighted below, with the Resources section being the only one that is mandatory.

Format Version (optional)

This is the AWS CloudFormation template version that your template conforms to and identifies the capabilities of the template. This is not the same as the API or WSDL version.

At the moment, the latest and only valid template version is 2010-09-09.

While there is only one valid version at the moment and this field is optional, it’s a good idea to include it in your templates for future reference.

Description (optional)

This section is a text string that provides the reader with a short description (0 to 1024 bytes in length) of the template. This section must come right after the Format Version section.

Be descriptive but concise!

Metadata (optional)

This section isn’t used all that often, but here you can include additional information about your template. This can include information for third-party tools that you may use to generate and modify these templates and other general data.

Please note that the Metadata section is not to be confused with the Metadata attribute that falls under the Resources section. This Metadata section should include information about the template overall, while the Metadata attribute should include data about a specific resource.

Parameters (optional)

You can use the Parameters section to input custom values to your template when you create or update a stack so you can make these templates portable for use in other projects.

For instance, you can create parameters that specify the EC2 instance type to use, an S3 bucket name, an IP address range, and other properties that may be important to your stack.

The Resources and Output sections often refer to Parameters, and these references must be included within the same template.

Mappings (optional)

The Mappings section allows you to create key-value dictionaries to specify conditional parameter values.

Examples of this include deploying different AMIs for each AWS region, or mapping different security groups to Dev, Test, QA, and Prod environments that otherwise share the same infrastructure stack.

To retrieve values in a map, you use the “FN::FindInMap” intrinsic function in the Resources and Outputs sections.

Conditions (optional)

Conditions allow you to use logic statements (just like an “if then” statement) to declare what should happen under certain situations.

We mentioned an example above about Dev, Test, QA, and Prod environments. In this case, you can use conditions to specify the type of EC2 instance to deploy in each of these environments. If the environment is Prod, you can set the EC2 instance to be m4.large. If the environment is Test, you can set it to be t2.micro to save money.

Transform (optional)

The Transform section allows you to simplify your CloudFormation template by condensing multiple lines of resource declaration code and reusing template components.

There are two types of transforms that CloudFormation supports:

1. “AWS::Include” refers to template snippets that reside outside of the main CloudFormation template you’re working with. Thus, you can make multi-line resource declarations in YAML or JSON files stored elsewhere and refer to them with a single line of code in your primary CloudFormation template.
2. “AWS::Serverless” specifies the version of the AWS Serverless Application Model (SAM) to use and how to process it.

You can declare multiple transforms in a template and CloudFormation executes them in the order specified. You can also use template snippets across multiple CloudFormation templates.

Resources (required)

The Resources section is the only section that is required in a CloudFormation template.

In this section, you declare the AWS resources, such as EC2 instances, S3 buckets, Redshift clusters, and others, that you want deployed in your stack. You also specify the properties, such as instance size, IAM roles, and number of nodes, for each of these components.

This is the section that will take up the bulk of your templates.

Outputs (optional)

In the Outputs section, you’ll describe the values that are returned when you want to view the properties of your stack.

You can export these outputs for use in other stacks, or simply view them on the CloudFormation console or CLI as a convenient way to get important information about your stack’s components.

That’s a lot of information about CloudFormation sections to digest. All of this will become clearer as we walk through some real-world templates, which we’ll do now.

Want this in a handy eBook? Click here to download our 62-page Infrastructure as Code Handbook, which includes IaC benefits, best practices, tools, and analysis of three AWS CloudFormation scripts!

Creating a VPC with AWS CloudFormation

The template we’ve provided below creates a virtual private cloud (VPC) with public and private subnets. This will allow you to launch AWS resources in a virtual network that you define and have complete control over.

In this template, we create a VPC with a public subnet for your web servers that are publicly addressable, and a private subnet where your backend components like databases or application servers will reside and be safe from the prying eyes of the internet.

The public subnet will connect to the internet via an Internet Gateway, and a Route Table tells the public subnet how to find the Internet Gateway.

We’ll then replicate the public and private subnets in another availability zone for high availability.

This template is pretty straightforward and only contains the Format Version, Description, and Resources sections.

Here’s the CloudFormation template in its entirety:

AWSTemplateFormatVersion: 2010-09-09
Description: Creates a VPC with public and private subnets
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 192.168.101.0/24
  PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 192.168.101.0/28
      MapPublicIpOnLaunch: true
      VpcId: !Ref VPC
  PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 192.168.101.32/28
      MapPublicIpOnLaunch: false
      VpcId: !Ref VPC
  InternetGateway:
    Type: AWS::EC2::InternetGateway
  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC
  PublicRouteTable:
    DependsOn: AttachGateway
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
  PublicDefaultRoute:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway
      RouteTableId: !Ref PublicRouteTable
  PublicRouteAssociationA:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicRouteTable
      SubnetId: !Ref PublicSubnetA
  PublicSubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 192.168.101.16/28
      MapPublicIpOnLaunch: true
      VpcId: !Ref VPC
  PrivateSubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 192.168.101.48/28
      MapPublicIpOnLaunch: false
      VpcId: !Ref VPC
  PublicRouteAssociationB:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicRouteTable
      SubnetId: !Ref PublicSubnetB

You can download this template here.

OK! Let’s dig into what’s going on here.

Creation of the VPC and public and private subnets

First, we add a VPC that encompasses the entire network we’re about to create:

Resources: 
  VPC: 
    Type: AWS::EC2::VPC 
    Properties: 
      CidrBlock: 192.168.101.0/24

Here’s what’s going on:

• VPC: This is the name you give to the resource.
• Type: This defines the CloudFormation resource type. Check out the resource type documentation to find supported properties.
• Properties: Each resource has required and optional properties. In this case, you can define the IP address range in CIDR notation.

Next we add a public and private subnet:

  PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 192.168.101.0/28
      MapPublicIpOnLaunch: true
      VpcId: !Ref VPC
  PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 192.168.101.32/28
      MapPublicIpOnLaunch: false
      VpcId: !Ref VPC

As before, you’ll notice the same resource structure of name, type, and supported properties. The “AWS::EC2::Subnet” type happens to support a few more properties.

The above example also uses some special syntax to make the template a little more dynamic, such as:

Pseudo parameters

A pseudo parameter dynamically resolves to a value, given the context of the CloudFormation stack. For example, “Ref: AWS::Region” gets converted to the region in which you deploy the template (i.e. us-east-1).

Intrinsic functions

A CloudFormation template is configuration, not code. However, you have access to some basic runtime logic.

In the above example, “!GetAZs Ref: AWS::Region” gets all the availability zones in the current region. And the “!Select” function gets the first AZ in the list.

Another use of an intrinsic function is “!Ref VPC”. This gets the VPC ID of the VPC resource defined earlier.

Here’s a breakdown of what’s happening in the code above.

• PublicSubnetA is our public subnet, and any EC2 instances provisioned will be given a public IP address.
• PrivateSubnetA is our private subnet.
• Each subnet is provisioned in the first availability zone in the current region. For example, this could be “us-east-1a”.
• Each subnet is placed inside the VPC defined earlier.
• Each subnet’s IP address range contains 16 addresses. These ranges are non-overlapping, and fall within the overall VPC’s IP range.

We wind up with this architecture:

Adding an Internet Gateway and Route Table

Next, we create an Internet Gateway. This allows instances within the VPC to access the internet. And it also allows the internet to access servers within the public subnet:

  InternetGateway:
    Type: AWS::EC2::InternetGateway
  AttachGateway:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

Behind the scenes, the Internet Gateway has no way to associate itself with our VPC. So you have to create an “AWS::EC2::VPCGatewayAttachment” resource to perform this task.

Next we create a Route Table, plus a few related resources:

  PublicRouteTable:
    DependsOn: AttachGateway
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
  PublicDefaultRoute:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway
      RouteTableId: !Ref PublicRouteTable
  PublicRouteAssociationA:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicRouteTable
      SubnetId: !Ref PublicSubnetA

The Route Table only has one property, “VpcId: !Ref VPC”, which associates itself with a VPC.

Route (“AWS::EC2::Route”) resources contain a single route, which points an IP address range to a gateway. The Route associates itself with the Route Table.

A Subnet Route Table Association wires up the route table to a subnet. In this case, the Route Table directs all traffic from PublicSubnetA to the internet.

One thing to note is the “DependsOn” attribute. CloudFormation is usually pretty good about spinning up dependencies in the correct order. However, you can use the “DependsOn” attribute to explicitly define a dependency.

This architecture shows the addition of the Internet Gateway and Route Table:

Replicate subnets for high availability

Finally, we replicate the subnets into a new availability zone to facilitate high availability.

  PublicSubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 192.168.101.16/28
      MapPublicIpOnLaunch: true
      VpcId: !Ref VPC
  PrivateSubnetB:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 192.168.101.48/28
      MapPublicIpOnLaunch: false
      VpcId: !Ref VPC
  PublicRouteAssociationB:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicRouteTable
      SubnetId: !Ref PublicSubnetB

This is all quite similar to the what we already created for public and private subnet A. Like before, the IP address ranges are unique and non-overlapping. The main difference with set B is that they get provisioned in the second availability zone rather than the first.

One thing to note is that PublicSubnetB needs its own route association, since it’s one association per subnet.

Finally, you wind up with this beautiful VPC architecture:

You now have the network infrastructure to launch EC2 instances, databases, or other AWS resources with some baseline security! You can use this template as a starting point for future projects.

Check out other posts in our IaC series:

• Infrastructure as Code: 5 Reasons Why You Should Implement IaC Now

• 6 best practices to get the most out of IaC

• 15 Infrastructure as Code tools you can use to automate your deployments

• What is AWS CloudFormation and how can it help your IaC efforts?

• How to incorporate S3, EC2, and IAM in a CloudFormation template

• How to create a Redshift stack with AWS CloudFormation

Or you can download all of these articles together in one handy eBook by clicking the link below. Thanks for reading!

Want this in a handy eBook? Click here to download our 62-page Infrastructure as Code Handbook, which includes IaC benefits, best practices, tools, and analysis of three AWS CloudFormation scripts!