Terraform Cookbook

Recipes for Codifying Infrastructure

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

Kerim Satirli and Taylor Dolezal

Terraform Cookbook

by Kerim Satirli and Taylor Dolezal

Copyright © 2023 O’Reilly Media. All rights reserved.

Printed in the United States of America.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

Editors: Michele Cronin and John Devins

Production Editor: Beth Kelly

Copyeditor: FILL IN COPYEDITOR

Proofreader: FILL IN PROOFREADER

Indexer: FILL IN INDEXER

Interior Designer: David Futato

Cover Designer: Karen Montgomery

Illustrator: Kate Dullea

September 2022: First Edition

Revision History for the Early Release

2022-06-22: First Release

2022-10-17: Second Release

See http://oreilly.com/catalog/errata.csp?isbn=9781098108465 for release details.

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Terraform Cookbook, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.

The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-098-10839-7

[FILL IN]

Chapter 1. Terraform Functions

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at mcronin@oreilly.com.

Terraform comes standard with an extensive library of built-in functions. Available in any Terraform file (including modules) and the Terraform Console, functions can validate, combine format, and transform data.

Functions are integral to creating reusable and less error-prone infrastructure by allowing for a more complex definition of your infrastructure as code.

Note

We use Terraform Output Values to illustrate the result of applying a function to an input variable. For traditional Terraform uses, this is not needed as Terraform can process the function without outputting its value to the user.

Running terraform console allows you to experiment with any built-in functions mentioned in these examples.

Note

Currently, Terraform does not support user-defined functions.

1.1 Workstation Configuration

You will need a few things installed to be ready for the recipes in this chapter.

General Setup

You will need to install the terraform binary to ensure that you can run these commands on your machine. For more information on setting up Terraform on your device, you can refer to the HashiCorp Learn Guide to get set up.

Validation

To confirm that everything is working correctly, you should be able to run terraform -help and see all the commands you’re able to run with terraform.

$ terraform -help
Usage: terraform [global options] <subcommand> [args]

          The available commands for execution are listed below.
The primary workflow commands are given first, followed by
less common or more advanced commands.

Main commands:

  init          Prepare your working directory for other commands
  validate      Check whether the configuration is valid
  plan          Show changes required by the current configuration
  apply         Create or update infrastructure
  destroy       Destroy previously-created infrastructure

 All other commands:
  console       Try Terraform expressions at an interactive command prompt
  fmt           Reformat your configuration in the standard style
  force-unlock  Release a stuck lock on the current workspace
  get           Install or upgrade remote Terraform modules
  graph         Generate a Graphviz graph of the steps in an operation
  import        Associate existing infrastructure with a Terraform resource
  login         Obtain and save credentials for a remote host
  logout        Remove locally-stored credentials for a remote host
  output        Show output values from your root module
  providers     Show the providers required for this configuration
  refresh       Update the state to match remote systems
  show          Show the current state or a saved plan
  state         Advanced state management
  taint         Mark a resource instance as not fully functional
  test          Experimental support for module integration testing
  untaint       Remove the 'tainted' state from a resource instance
  version       Show the current Terraform version
  workspace     Workspace management

Global options (use these before the subcommand, if any):

  -chdir=DIR    Switch to a different working directory before executing the
                given subcommand.
  -help         Show this help output or the help for a specified subcommand.
  -version      An alias for the "version" subcommand.

1.2 Using chomp to clean inputs

When working with Terraform, you may have realized that you have the option of working with a lot of text, whether it be a template file that you’re generating or looking to create some more dynamic configuration within your infrastructure projects.

The chomp function is an essential call-out as it can help you remove characters at the end of a string value and provide more reliable information as you use values within your infrastructure as code.

Problem

You’re working with unsanitized input data from an outside source, like a CSV file or JSON from an API endpoint, and you would like to remove unnecessary whitespace from the string to ensure it works with all your downstream needs.

Solution

Using an output variable, you can clean up an unsanitized variable using the chomp function. Terraform’s output variable functionality will show a sanitized output of the initial value.

# define an input variable that contains trailing line breaks
variable "unsanitized_location_1" {
 type        = string
 description = "(unsanitized) Location of Device"
 default     = "Rack 3,\nPosition 12\n\n"
}
# use an output value to display the result of using "chomp" on the "unsanitized_location_1" input variable
output "sanitized_location_1" {
 description = "(sanitized) Location of Device"
 value       = chomp(var.unsanitized_location_1)
}

Steps

This example is straightforward because once you have the solution code above, we can quickly get a feedback loop going to ensure the configuration is as expected.

By running terraform plan you can see if the chomp function has cleaned up the text appropriately.

$ terraform plan
Changes to Outputs:
  + sanitized_location_1 = <<-EOT
        Rack 3,
        Position 12
    EOT
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Since we’re not modifying any infrastructure or using cloud services, we don’t have to terraform apply. Still, we recommend that you do so during these exercises to understand how terraform works in a regular workflow.

By running terraform apply we get the following:

$ terraform apply
Changes to Outputs:
  + sanitized_location_1 = <<-EOT
        Rack 3,
        Position 12
    EOT

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.
  Enter a value: yes
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
sanitized_location_1 = <<EOT
Rack 3,
Position 12
EOT

Clean Up

We did not spin up any infrastructure in this example, but if you’d like to clean up your Terraform state file, you can run terraform destroy to get rid of your outputs.

$ terraform destroy
Changes to Outputs:
  - sanitized_location_1 = <<-EOT
        Rack 3,
        Position 12
    EOT -> null
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.
  Enter a value: yes
Destroy complete! Resources: 0 destroyed.

Discussion

This example was relatively straightforward and showed how you can get started with Terraform and use its built-in functions as it correlates to strings. Now that you’ve been introduced to this functionality, you aren’t limited to using predefined variables and can use any form of external inputs and clean them with the chomp function.

1.3 Cleaning up text with trimspace

The trimspace function removes trailing space from a string. For this operation, the Unicode definition of “space” is used, which includes the traditional “space”, tabs, newline characters, and other various space-like characters.

Problem

When you are not guaranteed a clean set of inputs, whether from a variable or from a data source, you would like to ensure that any downstream use of a value doesn’t have any unnecessary whitespace.

Solution

In this example, we take some text that has additional spacing and removes it with the trimspace function. Then by using Terraform’s output values, we can show that the text has been cleaned up as expected.

# define an input variable that contains trailing spaces
variable "unsanitized_location_2" {
 type        = string
 description = "(unsanitized) Location of Device"
 default     = "Rack 3, Position 12  "
}
# use an output value to display the result of using "trimspace" on the "unsanitized_location_2" input variable
output "sanitized_location_2" {
 description = "(sanitized) Location of Device"
 value       = trimspace(var.unsanitized_location_2)
}

Steps

By running terraform plan we can check to see if the trailing spaces are removed with our use of the trimspace function. Let’s make sure that’s the case:

$ terraform plan
Changes to Outputs:
  + sanitized_location_2 = "Rack 3, Position 12"
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

That looks good! Don’t worry about the note from Terraform detailing that we haven’t used the -out option, as that’s something that we’ll cover in later chapters. Right now, it’s important to see that our outputs are looking as we’d like.

Since the outputs are looking good, let’s go ahead and run terraform apply.

$ terraform apply
Changes to Outputs:
  + sanitized_location_2 = "Rack 3, Position 12"
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.
  Enter a value: yes

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
sanitized_location_2 = "Rack 3, Position 12"

This looks exactly as we want, and now we can move to the next step, or maybe even add in trimspace to some of our other Terraform use cases.

Clean Up

We did not spin up any infrastructure in this example, but if you’d like to clean up your Terraform state file, you can run terraform destroy to get rid of your outputs.

$ terraform destroy
Changes to Outputs:
  - sanitized_location_2 = "Rack 3, Position 12" -> null
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.
  Enter a value: yes

Destroy complete! Resources: 0 destroyed.

Discussion

Like chomp, the trimspace function is a good way of sanitizing inputs you have no direct control over in terms of how they are presented to you. You can use `output` variables or other constructs within Terraform to then pass on these values to your desired use case.

1.4 Fixing prefixes and suffixes

Problem

When dealing with a large grouping of text with similar prefixes or suffixes, and you’d like to remove them for further processing. Whether it be a lot of server names, phone numbers, IP addresses, or otherwise, these functions can come in handy.

Solution

In this example, we will remove prefixes from values that we define as variables and validate using Terraform output values.

# define an input variable of type string that contains a prefix
variable "unsanitized_location_3" {
 type        = string
 description = "(unsanitized) Location of Device"
 default     = "Server Rack 3, Position 12"
}
# define a local value to execute the `trimprefix` function
locals {
 # trim the prefix "Server" from the "unsanitized_location_3" input variable
 sanitized_location_3 = trimprefix(var.unsanitized_location_3, "Server")
}
# use an output value to display the local value of "sanitized_location_3"
output "sanitized_location_3_1" {
 description = "(sanitized) Location of Device"
 value       = local.sanitized_location_3
}

Note that the output of “sanitized_location_3_1” contains a trailing space. Using the trimspace function, this can be cleaned up like so:

# use an output value to display the result of using "trimspace" on the local value of "sanitized_location_3"
output "sanitized_location_3_2" {
 description = "(sanitized) Location of Device"
 value       = trimspace(local.sanitized_location_3)
}

Discussion

The trimprefix and trimsuffix functions remove a specified prefix (or suffix) from an input variable. This operation will only be carried out if the string is present exactly as described.

1.5 Working with regex

Problem

You have a grouping of data, and you’d like to separate it into two distinctly separate groupings of information.

Solution

# define an input variable of type string that contains a prefix
variable "location_4" {
 type        = string
 description = "Location of Device"
 default     = "Rack 3, Position 12"
}
locals {
 # apply a Regular Expression on the `location_4` input variable
 # the expression creates two groups:
 # Group 1 searches for "Rack", followed by a space and one or more digits
 # Group 2 searches for "Position", followed by a space and one or more digits
 location_4 = regex("(Rack [0-9]+), (Position [0-9]+)", var.location_4)
}
# use an output value to display the local value of "location_4"
# this local value
output "location_4_1" {
 description = "(sanitized) Location of Device"
 value       = local.location_4
}

The output of the above function will look like this:

location_4_1 = [
  "Rack 3",
  "Position 12",
]

Note that the string changed into a list with two items. These items correspond to the search pattern groups that we defined.

Discussion

Regular Expressions are a great way of searching for specific (sub)sets of information in textual data.

The regex function returns a single match for one or more groups defined in the search pattern. If more than one match is desired, the regexall function provides a way forward.

Using Regular Expressions to break an input string into multiple parts is a clean and safe way.

Be careful, though: with the power of Regular Expressions also comes the complexity of defining how you expect your data to match.

Warning

For additional information on Regular Expressions, also outside the context of Terraform, we highly recommend “Mastering Regular Expressions.”

If you feel like using a function within Terraform that already has a more specific function for working with your data, we recommend you use that function over a regex function as more specific methods will provide a better-tuned experience for the input and output values for your use case.

1.6 Search and replace

Problem

When you have values that need to be modified by a search and replace operation.

Solution

The replace enables you to modify (parts of) a string and replace it with a different value, including empty strings.

In its basic form, the replace function is a simple search-and-replace operation:

# define an input variable of type string that contains trailing spaces
variable "sensitive_location" {
 type        = string
 description = "(unsanitized) Location of Device"
 default     = "Rack 3, Position 12"
}
# use an output value to replace the word "Rack" in "sensitive_location" with an empty string
output "sensitive_location" {
 description = "(sanitized) Location of Device"
 value       = replace(var.sensitive_location, "Rack ", "")
}

The output of the above recipe will “hide” the word “Rack” (and whitespace immediately after), resulting in an output value of “3, Position 12”.

Alternatively, you can use Regular Expressions (also see previous recipe) to manipulate a string in a more extensive way:

# use an output value to replace both "Rack" and "Position" (including spaces" with an empty string
output "combined_location" {
 description = "(sanitized) Location of Device"
 value       = replace(var.sensitive_location, "/Rack | Position /", "")

}

The output of the above recipe will result in an output value of “3,12”.

Warning

Using the replace function is not recommended for hiding sensitive data as it will surface data in plaintext.

For handling sensitive data in Terraform, check the `sensitive` function (see Recipe X.YY), additionally, for Output Values, consider setting the `sensitive` attribute to a truthy value:

output "combined_location" {
 description = "(sanitized) Location of Device"
 value       = replace(var.sensitive_location, "/Rack | Position /", "")
 sensitive   = true
}

The output of the above recipe will simply read “(sensitive),” indicating that Terraform has hidden the value from printing to the standard output. Internally, Terraform is still able to see the plaintext value.

Discussion

By using `replace`, you can level up your Terraform code and edit values from variables, data sources, or other inputs as you need.

1.7 Templates for title, upper, lower

Problem

When working with templates or if you want to format your output values in a more consistent way, you can address these contexts with the `title`, `upper`, and `lower` functions.

Solution

# define an input variable of type string that contains varying uppercase and lowercase characters
variable "server_location" {
  type        = string
  description = "(unformatted) Location of Device"
  default     = "rack 3, Position 12"
}
# use an output value to show the source string in titlecase
output "title_location" {
  description = "(title formatted) Location of Device"
  value       = title(var.server_location)
}
# use an output value to show the source string in uppercase
output "upper_location" {
  description = "(upper formatted) Location of Device"
  value       = upper(var.server_location)
}
# use an output value to show the source string in lowercase
output "lower_location" {
  description = "(lower formatted) Location of Device"
  value       = lower(var.server_location)
}

Discussion

Converts the first letter of each word in the given string to uppercase, all characters to uppercase, or all characters to lowercase.

1.8 Just that sort of sort

Problem

When you have a list of items and you’d like to sort them lexicographically, the sort function accepts strings, then returns a new list with the strings that have been sorted.

Solution

# define an input variable of type string that contains an unordered list of strings that represent machine names
variable "machine_names" {
  type = list(string)
  description = "An unsorted list of machines that need sorting"
  default = ["tdolezal-D12BC45BFQ5P", "ksatirli-G41NH02BLT1L", "lninomae-T90TY75HMC4E"]
}
# use an output value to show a formatted list of machine names
output "sorted_machines" {
  description = "A formatted list of machines"
  value       = sort(var.machine_names)
}

Discussion

The sort function helps with tidying up lists and putting them into exactly the order you need. This works best when paired with Terraform’s data sources, raw inputs from files, or other sources where unordered data originates from.

1.9 Creating subnets from a CIDR block

Problem

You want to create public and private subnets from a CIDR block, but you don’t want to create them statically.

Solution

By using a locals block and the cidrsubnet function, you can create network spaces in a programmatic way.

Prerequisites

Ensure that you have a CIDR block (ex. 10.0.0.0/16)

Know the bits that you’d like to shift each CIDR subnet block

Preparation

Create a new file for this example, named main.tf, and type in the following code.

variable "cidr" {
  type        = string
  default     = "10.0.0.0/16"
  description = "A network address prefix in CIDR notation"
}
locals {
  private_subnets = [
    cidrsubnet(var.cidr, 8, 1),
    cidrsubnet(var.cidr, 8, 2),
    cidrsubnet(var.cidr, 8, 3)
  ]
  public_subnets = [
    cidrsubnet(var.cidr, 8, 4),
    cidrsubnet(var.cidr, 8, 5),
    cidrsubnet(var.cidr, 8, 6)
  ]
}
output "public_subnets" {
  value       = local.public_subnets
  description = "Computed public subnet CIDR blocks"
}
output "private_subnets" {
  value       = local.private_subnets
  description = "Computed private subnet CIDR blocks"
}
Steps

Since this example isn’t using any providers or modules, there won’t be anything to download if we run terraform init. We can validate that all the Terraform configuration is correct by running `terraform validate` in your command-line interface and it should return a success message.

```

Success! The configuration is valid.

```

Validation Steps
Let’s see if the code works! We can run terraform plan to see if all the values are working as we expect. The code that you typed in should create a total of six different subnets.

```

$ terraform plan

Changes to Outputs:
  + private_subnets = [
      + "10.0.1.0/24",
      + "10.0.2.0/24",
      + "10.0.3.0/24",
    ]
  + public_subnets  = [
      + "10.0.4.0/24",
      + "10.0.5.0/24",
      + "10.0.6.0/24",
    ]
You can apply this plan to save these new output values to the Terraform state, without changing
any real infrastructure.

```

Now, you can run `terraform apply` and confirm all the changes if they match what you saw in the `terraform plan` step.

$ terraform apply
Changes to Outputs:
  + private_subnets = [
      + "10.0.1.0/24",
      + "10.0.2.0/24",
      + "10.0.3.0/24",
    ]
  + public_subnets  = [
      + "10.0.4.0/24",
      + "10.0.5.0/24",
      + "10.0.6.0/24",
    ]

You can apply this plan to save these new output values to the Terraform state, without changing
any real infrastructure.
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.
  Enter a value: yes
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
private_subnets = [
  "10.0.1.0/24",
  "10.0.2.0/24",
  "10.0.3.0/24",
]
public_subnets = [
  "10.0.4.0/24",
  "10.0.5.0/24",
  "10.0.6.0/24",
]

You’ll notice an “Outputs” section returned in your terminal due to the two output variables that we defined. You can run terraform output in your terminal if you’d like to inspect these values once more.

$ terraform output
private_subnets = [
  "10.0.1.0/24",
  "10.0.2.0/24",
  "10.0.3.0/24",
]
public_subnets = [
  "10.0.4.0/24",
  "10.0.5.0/24",
  "10.0.6.0/24",
]

Challenge

This example helps to understand better how you can use the cidrsubnet function and gives you a bit of an idea of what the cidrsubnets function might do, too! Though, say you are looking for a way to set up multiple subnets from a base CIDR block and name them? Well, there’s a Terraform module for that! Try refactoring the solution you just coded and see if you can do so with HashiCorp’s subnets module. ().

You can even try changing the bits for each of your subnets and make them cover different CIDR spaces if you’d like.

We’ll learn more about Terraform modules and various recipes for them in the next chapter.

Clean Up

We did not spin up any infrastructure in this example, but if you’d like to clean up your Terraform state file, you can run terraform destroy to get rid of your outputs.

$ terraform destroy
Changes to Outputs:
  - private_subnets = [
      - "10.0.1.0/24",
      - "10.0.2.0/24",
      - "10.0.3.0/24",
    ] -> null
  - public_subnets  = [
      - "10.0.4.0/24",
      - "10.0.5.0/24",
      - "10.0.6.0/24",
    ] -> null
You can apply this plan to save these new output values to the Terraform state, without changing
any real infrastructure.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.
  Enter a value: yes
Destroy complete! Resources: 0 destroyed.

Discussion

This example showcased how to use networking functions like cidrsubnet to generate subnets with a single base CIDR block. Setting up networks can be difficult, but using functions like cidrsubnet or cidrsubnets can simplify your configuration and what information you require upfront as you create your infrastructure as code.

Chapter 2. Terraform Ecosystem

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 2nd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at mcronin@oreilly.com.

The Terraform ecosystem is vast and covers a lot of different use cases. Everything from public modules in the Terraform Registry to GitHub Actions to Terraform Cloud, you choose to find the best way to work with infrastructure as code with Terraform.

This chapter will take a whirlwind tour around the Terraform Ecosystem and cover some of the most popular use cases used by community members.

Our goal in this chapter is to cover primarily free or low-cost examples, but please note the areas that we call out that might charge you a fee or cost you some money.

While these examples will give you more insight into what you can expect from the Terraform Ecosystem, it is by no means complete, and we encourage you to seek out and find more and more examples as you see fit.

2.1 General Setup

Amazon Web Services

For these examples, we recommend that you access an Amazon Web Services (AWS) account. Most of the examples we use will be on the AWS free tier, but please note that this tier only stays free or low cost for about twelve months.

Azure

Microsoft’s Azure cloud computing service is also available to use. We will attempt to keep these examples lower-cost, though please note that you may be charged with any cloud that’s used. You can sign up for an Azure account with the link below.

GitHub

We will be using GitHub Actions in this chapter, so please ensure that you have a free GitHub account, the ability to create GitHub repositories, and a fundamental understanding of how to use Git.

You can sign up for a GitHub account here.

Terraform Cloud

Terraform Cloud is a service run by HashiCorp that helps store Terraform state and provides you ways to configure and store sensitive information about your clouds or other sensitive infrastructure without having to share that information with your team. Terraform Cloud also has a free tier plus multiple paid tiers that allow you to scale up with your team as necessary.

In these examples, we’ll be using Terraform Cloud to store our state as it’s a straightforward solution and doesn’t require a public cloud offering for storing your Terraform state.

You can sign up for Terraform Cloud by using this link.

2.2 Using Public Modules to create an AWS VPC

Problem

You’re standing up some new infrastructure within AWS, but you need network space to start spinning up infrastructure within a virtual private cloud (VPC).

Solution

Using the public AWS VPC module on the Terraform Registry, you can stand up a VPC with useful defaults and ensure that your network topology is secure.

Prerequisites

AWS account

CIDR (ex. 10.0.0.0/16)

Preparation

First, let’s take a look at the Terraform Registry to understand what modules and information we have available to us.

By searching for “terraform-aws-modules vpc” we can find one of the most widely used modules on the registry itself. Though, what does this module provide us?

Suppose we’re able to specify a few default values like a name, CIDR address, availability zones, public and private subnets, and a few other items. In that case, we can get a pretty complete VPC without having to craft it from scratch.

Steps

Let’s create a few different files to help us build out this VPC and use the public module.

First, let’s start with a variables.tf file as the main place to call out all the variables that we’re going to be using for this example.

variable "project_name" {
  type        = string
  description = "Globally used project name for this example"
  default     = "terraform-registry"
}
variable "region" {
  type        = string
  description = "AWS US-based Region that will be used in this demo."
  default     = "us-west-2"
}
variable "cidr" {
  type        = string
  description = "The CIDR block to use for this demo"
  default     = "10.0.0.0/16"
}

As you can see, we don’t need too much to get started. Three straightforward variables do the trick!

Next, we’re going to want to set up our AWS provider within Terraform inside of a new file that we can name aws.tf. By doing this, we let Terraform know that we want to use and configure the AWS provider.

provider "aws" {
  region = var.region
}

Lastly, we will be setting up our vpc.tf file, which will pull in the public AWS VPC module and we can set up a VPC with three private subnets, three public subnets. We can source in our availability zones and some of the other configuration items, automatically by using Terraform data sources.

# https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones
# Filter out opt-in availability zones (local zones, as an example)
data "aws_availability_zones" "available" {
  filter {
    name   = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "3.7.0"
  name = var.project_name
  cidr = var.cidr
  azs             = slice(data.aws_availability_zones.available.names, 0, 3)
  private_subnets = local.private_subnets
  public_subnets  = local.public_subnets
  enable_nat_gateway   = true
  enable_dns_hostnames = true
  tags = {
    Operator = "Terraform"
  }
}
locals {
  private_subnets = [
    cidrsubnet(var.cidr, 8, 1),
    cidrsubnet(var.cidr, 8, 2),
    cidrsubnet(var.cidr, 8, 3)
  ]
  public_subnets = [
    cidrsubnet(var.cidr, 8, 4),
    cidrsubnet(var.cidr, 8, 5),
    cidrsubnet(var.cidr, 8, 6)
  ]
}

Now that we have all our Terraform configuration in place, we can call `terraform init` to download our dependencies and create our Terraform lockfile.

$ terraform init
Initializing modules...
Downloading terraform-aws-modules/vpc/aws 3.7.0 for vpc...
- vpc in .terraform/modules/vpc
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching ">= 3.38.0"...
- Installing hashicorp/aws v3.58.0...
- Installed hashicorp/aws v3.58.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Validation Steps

We’ve successfully set up all our Terraform configuration and dependencies, so now let’s run a `terraform plan` to inspect and ensure that all is as we expect.

$ terraform plan
...
Plan: 28 to add, 0 to change, 0 to destroy.
───────────────────────────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take
exactly these actions if you run "terraform apply" now.

This looks great and is exactly what we want. If you’d like to apply this infrastructure and inspect it within the AWS console or AWS CLI, you can do so by running `terraform apply` and confirming the run by typing “yes” when prompted.

Clean Up

If you end up applying this code, you can remove it by running `terraform destroy`.

Discussion

In this example, we were able to start with a completely blank slate and use Terraform’s public module registry to source and create an entire virtual private cloud in Amazon in under an hour. The Terraform registry is a powerful resource for finding the best building blocks possible for your cloud or infrastructure.

Even if you’re not able to utilize a module from the registry due to your team’s policies or procedures, you can view the source code for the modules to see how to implement them on your own. You can also contribute patches or fixes to these modules if you’d like.

2.3 Using Public Modules to create an EKS cluster

Problem

After setting up a Virtual Private Cloud (VPC) within AWS, you’d like to start hosting some applications on Kubernetes. Right now, you have a network based on the previous recipe, but you’d like to stand up an Elastic Kubernetes Service (EKS) cluster and get it configured to use for the first time.

Solution

Using the public AWS EKS module on the Terraform Registry, you can supercharge your infrastructure with code configuration with exactly what you need to get your Kubernetes cluster running on AWS.

Preparation

We can browse the Terraform Registry to find many modules, including the one we’ll be using today, by searching “terraform-aws-modules eks.”

The Terraform registry shows documentation for the module (Figure 2-1), as well as what inputs are required, what outputs are provided, and other dependencies and resources available to us as we use the module.

Figure 2-1. Terraform Registry AWS EKS Terraform Module Listing

Steps

Since we’re building off the previous recipe we used for the VPC module, we don’t have to re-specify the AWS provider, or create the VPC module again.

Our `variables.tf` file is mostly set up, but there are more variables we have to add, like the Kubernetes version we want to use, and how we want to configure our worker node pool that are going to host our applications as we add them to our EKS cluster.

First, let’s add these values to our `variables.tf` file that will be necessary for all the required inputs we need for using the EKS module.

variable "cluster_version" {
  type        = string
  description = "The Kubernetes version for our clusters"
  default     = "1.21"
}
variable "cluster_instance_type" {
  type        = string
  description = "EC2 instance type for the EKS autoscaling group."
  default     = "m5.large"
}
variable "cluster_asg_desired_capacity" {
  type        = number
  description = "The default number of EC2 instances our EKS cluster runs."
  default     = 3
}
variable "cluster_asg_max_size" {
  type        = number
  description = "The maximum number of EC2 instances our EKS cluster will have."
  default     = 5
}
variable "cluster_enabled_log_types" {
  type        = list(string)
  description = "The Kubernetes log types that will be enabled for the EKS cluster."
  default     = ["api", "audit", "authenticator", "controllerManager", "scheduler"]
}
variable "cluster_write_kubeconfig" {
  type        = bool
  description = "Specify if Terraform should output the Kubernetes configuration file. "
  default     = false
}

Excellent! Those variables will set us up for success, and now we can continue adding a few more Terraform configuration files to complete this instantiation.

Next, we’ll set up a KMS key for encrypting and securing our secrets within Kubernetes. Please add a `kms.tf` file to your project with the following configuration. This key will be rotated by Amazon and will remove our responsibility from having to ensure regular rotation in a manual sense.

# https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key
resource "aws_kms_key" "eks" {
  description             = "EKS Secret Encryption Key"
  deletion_window_in_days = 7
  enable_key_rotation     = true
}

Lastly, we’ll set up the EKS cluster by passing the variables we defined, passing the KMS key details of what we created, and working with Terraform’s data sources to authenticate to our EKS cluster.

data "aws_eks_cluster" "cluster" {
  name = module.eks.cluster_id
}
data "aws_eks_cluster_auth" "cluster" {
  name = module.eks.cluster_id
}
provider "kubernetes" {
  host                   = data.aws_eks_cluster.cluster.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
  token                  = data.aws_eks_cluster_auth.cluster.token
}
module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "17.23.0"
  cluster_name              = var.project_name
  cluster_version           = var.cluster_version
  subnets                   = module.vpc.private_subnets
  vpc_id                    = module.vpc.vpc_id
  cluster_enabled_log_types = var.cluster_enabled_log_types
  write_kubeconfig          = var.cluster_write_kubeconfig
  cluster_encryption_config = [
    {
      provider_key_arn = aws_kms_key.eks.arn
      resources        = ["secrets"]
    }
  ]
  worker_groups = [
    {
      asg_desired_capacity = var.cluster_asg_desired_capacity
      asg_max_size         = var.cluster_asg_max_size
      instance_type        = var.cluster_instance_type
    }
  ]
}

Now that we have all our Terraform configurations in place, we can call terraform init to download our dependencies and create our Terraform lockfile.

$ terraform init
…
Terraform has been successfully initialized!

You can run a `terraform plan` to validate your configuration has been specified correctly, and you can run a `terraform apply` if you’d like to stand up your cluster.

If you end up applying this code, you can remove it by running terraform destroy.

Discussion

In this example, we were able to build upon the VPC example we set up in the previous recipe and again use Terraform’s public module registry to create an EKS cluster that either we or our teams could use and extend.

One important note about this cluster is that we shouldn’t be using Terraform for configuration management in addition to instantiating the cluster. It makes sense to iterate and to keep the EKS authentication and instantiation separate from the configuration, otherwise, we can run into issues during maintenance or upgrades.

2.4 Linting Terraform with GitHub Actions

Problem

Ensuring that your Terraform configuration is linted properly and running as expected can be difficult as you maintain your code for long periods of time. Operating with larger teams and scales can make this even more unwieldy.

Solution

By using GitHub actions and a dash of YAML configuration, we can create a workflow that allows you to ensure your code is linted properly and ensure that Terraform is able to execute your configuration as you expect.

To start this process, you will need to have a GitHub account and have GitHub Actions enabled on the repository you’ll be pushing your code to. You will need an existing Terraform configuration to test this recipe. You can use the code from the previous examples we’ve worked through if you’d like.

Once you have chosen your Terraform configuration repository, you need to create a `.github` folder at the root of your repository. We’ll need to create a `workflows` folder inside of the `.github` folder to contain our workflows.

First, create a superlinter.yml file in the `workflows` folder as follows.

---
name: "Code Quality: Super-Linter"
on:
  pull_request:
jobs:
  superlinter:
    name: Super-Linter
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 1
      - name: Lint Code
        uses: docker://github/super-linter:v4
        env:
          VALIDATE_ALL_CODEBASE: true
          DEFAULT_BRANCH: "main"
          DISABLE_ERRORS: false
          VALIDATE_BASH: true
          VALIDATE_JSON: true
          VALIDATE_MD: true
          VALIDATE_TERRAFORM: true
          VALIDATE_YAML: true

This GitHub Action workflow will ensure that your Terraform, Bash, JSON, Markdown, and YAML files have all been formatted correctly. This code will run during pull requests, but not at any other time an action is taken on your repository. This will help minimize or even eliminate formatting issues with your configuration and documentation.

Next up, we’ll add a terraform.yml file in the workflows directory we created with the following configuration.

---
name: "Code Quality: Terraform"
on:
  push:
    branches:
      - main
  pull_request:
jobs:
  terraform:
    name: Terraform
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2
        with:
          fetch-depth: 1
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1

      - name: Run `terraform fmt`
        id: fmt
        run: terraform fmt -diff -check -no-color -recursive
      - name: Run `terraform init`
        id: init
        run: terraform init
      - name: Run `terraform validate`
        id: validate
        if: github.event_name == 'pull_request'
        run: terraform validate -no-color

This code example will run during pushes to the main branch, and during pull requests to ensure that the Terraform configuration is formatted correctly and that it has been instantiated correctly in terms of dependencies and references.

Adding these two straightforward actions can reduce a lot of the “administrivia” faced when managing a GitHub project for Terraform and we hope this helps you manage yours, even if it’s just you working with it.

Figure 2-2. GitHub Actions workflows providing Terraform feedback loops

Discussion

These two actions add a lot of value by providing oversight on your Terraform configuration without having to manually check these files on a regular basis. This allows for teams to scale code reviews without additional hiring and more quickly get feedback loops as they test new configurations within their infrastructure.

2.5 Keeping Terraform updated with Dependabot

Problem

When it comes to Terraform and really any code, we always hear about how important it is to keep our dependencies updated. But keeping code, configuration, and dependencies updated is no easy feat, is it? Thankfully, we can use another GitHub feature called Dependabot to let us know when new dependencies are available, and it will even open up pull requests to alert us to those changes.

Solution

You will need an existing Terraform configuration repository on GitHub to try these changes, and you will also need to create a `.github` folder at the root of your repository to store these changes.

Let’s create a `dependabot.yml` file within our `.github` folder with the following details.

version: 2
updates:
  # Maintain dependencies for GitHub Actions
  - package-ecosystem: "terraform"
    directory: "/"
    schedule:
      interval: "daily"

Once we commit this and push up the changes, we will now start getting daily updates for our Terraform repository about what modules and providers we need to update. Dependabot will open up pull requests automatically, which we can then decide to merge directly, or make the changes on our own.

Discussion

Dependabot has a lot of options that you can set to make management of your Terraform or many other languages and dependencies much more straightforward. Using GitHub Actions to keep your dependencies up to date is an incredibly helpful feature that helps you stay focused on your infrastructure and less-so on your dependency management.

Chapter 3. Terraform Cloud

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at mcronin@oreilly.com.

Terraform Cloud is an application that helps teams use Terraform together. Terraform Cloud assists you with managing Terraform runs in a consistent and reliable environment. It includes easy access to shared state and private data, access controls for approving changes to infrastructure, a private registry for sharing Terraform modules, and detailed policy controls for governing the contents of Terraform configurations, and more. There’s a lot that Terraform Cloud can assist you with while allowing you to quickly jump right into your infrastructure as code workflows.

Terraform Cloud is a hosted service that small teams can freely sign up to use, connect Terraform to version control, share variables, run Terraform in a stable remote environment, and securely store remote state. Paid tiers allow you to add more than five users, create teams with different levels of permissions, enforce policies before building infrastructure, and collaborate more effectively. Most of the recipes we share in this chapter will be available with the free tier, and we will specify when that’s not the case.

3.1 Codespaces and Dev Containers

Problem

You need to be able to code while mobile and away from your standard office setting.

Working on problems in the comfortable space of your office can make your infrastructure as code sessions quite lovely. Maybe you have a favorite artist you listen to while you work or a chillwave playlist from Spotify. When you need to leave the comfort of your office, you can be left in the lurch with your coding environment when you’re moving or traveling. Not everyone can bring their desktop machine or coding workstation with them, and this recipe is for anyone on the go or who favors moving their workspace entirely into the cloud.

Solution

Getting this recipe working contains quite a few steps, so buckle up and prepare for all these moving parts. Wel’ll be using GitHub, Visual Studio Code (VSC), and a browser to complete these steps, so be sure to sign up and download those as you need.

To begin, create a .devcontainer folder at the root of your directory, then an empty Dockerfile and devcontainer.json file (Figure 3-1).

Figure 3-1. When viewing your files in VSC, it should look like this.

We’ll start by editing the Dockerfile and adding just a few key lines of configuration.

# See https://hub.docker.com/r/hashicorp/terraform for more information
FROM hashicorp/terraform:1.2.6

Next, we specify the JSON configuration for how our codespace container will work in the devcontainer.json file, with the following lines.

// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.238.1/containers/codespaces-linux
{
	"name": "GitHub Codespaces (Default)",
	"build": {
		"dockerfile": "Dockerfile"
	},
	// Configure tool-specific properties.
	"customizations": {
		// Configure properties specific to VS Code.
		"vscode": {
			// Add the IDs of extensions you want installed when the container is created.
			"extensions": [
				"GitHub.vscode-pull-request-github",
        "GitHub.copilot",
        "HashiCorp.terraform"
			]
		}
	}
}

Now, commit and push your changes to the repository you’re working with on GitHub.

Note

Ensure that your changes are pushed to GitHub or these next steps will not work.

Once your code is on GitHub, you will notice that you have unlocked a new capability, CodeSpaces.

To create your codespace, you can use the button within the GitHub UI (Figure 3-2), or you can try some of the other creation options.

Figure 3-2.  

The first time you load your Codespace, it may take a few moments for it to build and load within your browser or VSC (Figure 3-3).

Figure 3-3. Your codespace will look like this when building.

Once the Codespace has loaded, you will see something resembling Figure 3-4 in your browser (or within VSC).

Figure 3-4. Resulting page after the Codespace has loaded.

As a final step, you can run terraform version in the terminal window of your Codespace and confirm that you can use Terraform in the cloud.

Note

If you’re using Terraform Cloud to manage your state, you will have to set a TF_TOKEN_APP_TERRAFORM_IO environment variable that contains your credentials from running terraform login. This step will allow you to modify your Terraform state stored in Terraform Cloud.

Figure 3-5.  

Discussion

In this recipe, we took many steps to set up a Codespace within GitHub. Once completed, this allows for you to work with your IaC workflows in the cloud without having to worry about which machine it is being run on or even the credentials for your cloud environments.

We first created a metadata folder that houses configuration information that installs the Terraform binary and links together a place where we can work with our IaC, plan, validate, apply, and perform edits in real-time. This example shows the minimal configuration required to run this Codespace, though you can edit this for just about any workflow you can imagine or install any other binaries or tools needed.

3.2 Cloud Credentials

Problem

It’s likely that you’ve been asked to email, Slack, or fax over secret cloud credentials to someone, and NO ONE is proud in that moment. Sure, you can set up something like HashiCorp Vault to generate secrets in real time or use your cloud provider’s secret storage feature. Though wouldn’t it be great if there were something else we could use to share existing credentials with our team instead?

Solution

Terraform Cloud allows for the ability of workspace variables. To start, let’s ensure we have a few key items available to us. We’ll use my credentials for Amazon Web Services (AWS) in this recipe, though note that you can use whichever cloud provider or credentials are most applicable to your workflow.

First, navigate to the Terraform Cloud homepage. Once you’ve signed in with your credentials, choose the workspace you’d like to work with for this recipe. I’m going to use my eks-in-action workspace, because it’s my favorite (Figure 3-6).

Figure 3-6. The eks-in-action workspace.

Once I’ve clicked into the my eks-in-action workspace, I’m going to then navigate to the “Variables” tab to set my credentials (Figure 3-7).

Figure 3-7.  

For AWS, you need to set three environment variables, AWS_REGION, AWS_SECRET_ACCESS_KEY, and AWS_ACCESS_KEY_ID. Terraform Cloud lets you select a few different options when these values are set, such as whether you want them to be Terraform variables, environment variables, or if they need to be in HCL format. You can also set these values as sensitive, which is essentially “write-only” and quite secure. Last, but certainly not least, you have the ability to set a variable description, which we highly recommend you fill out as a reminder for your teammates, or your future self.

Figure 3-8.  

Now that you’ve set the three AWS credentials as environment variables, team members can utilize these during their Terraform Cloud runs and you no longer need to worry about providing the credentials in a less-than-secure fashion.

Figure 3-9.  

Discussion

Terraform Cloud is highly configurable and allows teams to set all their sensitive credentials in a secure way without allowing for any exposure. Teams are able to adopt this workflow at their own pace and in whatever format is most helpful for your specific workflow.

You can set variables specifically for each workspace or you can create variable sets to reuse the same variables across multiple workspaces. For example, you could define a variable set of provider credentials and automatically apply it to all of the workspaces using that provider. You can use the command line to specify variable values for each plan or apply. Otherwise, Terraform Cloud applies workspace variables to all runs within that workspace.

3.3 GitOps Workflows

Problem

As teams get started working with Terraform, it’s quite common to see folks running Terraform commands and spinning up infrastructure on their workstations. Though as teams start to grow, or processes start to mature, we start looking for a more robust and reliable workflow. Terraform Cloud allows us to use our version control system, so long as it uses Git, to gate which changes actually make it to production.

Solution

Todo

Discussion

Terraform Cloud is more powerful when you integrate it with your version control system (VCS) provider. Although you can use many of Terraform Cloud’s features without one, a VCS connection provides additional features and improved workflows.

When workspaces are linked to a VCS repository, Terraform Cloud can automatically initiate Terraform runs when changes are committed to the specified branch. Terraform Cloud makes code review easier by automatically predicting how pull requests will affect infrastructure.

We recommend configuring VCS access when first setting up an organization, and you might need to add additional VCS providers later depending on how your organization grows.

Note

Configuring a new VCS provider requires permission to manage VCS settings for the organization.

About the Authors

Kerim Satirli is a senior developer advocate at HashiCorp, where he works on creating a positive experience for developers and infrastructure engineers. He is passionate about complex systems and enjoys the challenge of codifying them. Kerim enjoys aerial photography, rock climbing, and cake baking.

Taylor Dolezal works at the Cloud Native Computing Foundation (CNCF) as the head of ecosystem. He works closely with open source communities, and is passionate about Kubernetes, Terraform, and solving distributed systems problems. You can usually find Taylor buried deep in a book, preparing a technical talk, or running with his partner, Hannabeth, and two pups.