Terraform Tips and Tricks


Table of contents

Facts about Remote State

  • Remote state is not the default; you must specify which backend to use and configure it to be used.
  • Remote state can be used by multiple team members. Terraform will write state data to a remote data store that users with access can use so there aren’t multiple state files.
  • Remote state uses a backend, which is configured in your configuration’s root module.
  • Remote state allows you to share output values with other configurations. Those configurations can then consume the exposed outputs in additional configurations.

Remote State Storage support

  • Terraform Cloud
  • HashiCorp Consul
  • Amazon S3
  • Azure Blob Storage
  • Google Cloud Storage
  • Alibaba Cloud OSS
  • ...and more

Separate Environments

  1. It is good practice to separate your Terraform configurations per environment.
  2. Separate environments help with code organization, as well as allowing for better and easier CI and automation integration.
  3. Implementing a one-folder-per-environment pattern lets you copy and paste Terraform code from one folder to another. This, used with variables, allows you to quickly change only what is needed per environment.

Use Modules

  1. Modules are containers for multiple resources that are used together.
  2. Every Terraform configuration contains at least one module.
  3. Modules can call other modules. This lets you include a module’s resources in a configuration in a concise way.
  4. Modules can be called multiple times, either in the same Terraform configuration or in separate ones. This allows for resource configurations to be packaged and reused.

Arguments to use with Modules

  • source: This argument is mandatory for all modules.
  • version: This argument is recommended for modules from a registry.
  • meta-arguments: Arguments like for_each and count.
  • input variables: Most other arguments correspond to input variables.
module "zland" {
  source = "git::ssh//[email protected]/zland/module.git"
  version = "1.0.5"
  servers = 3
}

Module Output Values

resource "aws_instance" "appserver" {
    #...
    instance = module.servers.instance_ids
}

Since the resources defined in a module are encapsulated, a calling module cannot access their attributes directly. Instead, the child module can declare output values.

Create a custom Module (Example)

Create those files in the folder modules/ec2:

main.tf

resource "aws_instance" "app_server" {
  ami           = "DUMMY_VALUE_AMI"
  instance_type = "t3.micro"
  subnet_id     = "DUMMY_VALUE_SUBNET_ID"
  tags = {
    Name = "WayneCorp"
  }
}

outputs.tf

output "instance_id" {
  description = "ID of the EC2 instance"
  value       = aws_instance.app_server.id
}

output "instance_public_ip" {
  description = "Public IP address of the EC2 instance"
  value       = aws_instance.app_server.public_ip
}

to use your module, add this snippet at the end of you existing ec2.tf file:

existing ec2.tf:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }
  required_version = ">= 0.14.9"
}
provider "aws" {
  profile = "default"
  region  = "us-east-1"
}

new content to add:

module "ec2-module" {
  source = "./modules/ec2/"
}

now you can run a terraform fmtto format your code.

Now run a terraform init to initialize your terraform backend

run a terraform validate to make sure syntax is correct

finally run a terraform plan and if your happy with the output a terraform apply

To check your state you can now run terraform show or aws ec2 describe-instances

Don’t Repeat Yourself

  • DRY is a principle that promotes modularization, abstraction, and code reuse and discourages repetition.
  • This principle states that “every piece of knowledge must have a single, unambiguous, authoritative representation within a system”.
  • This principle can be applied to not only programming, but to database schemas, test plans, the build system, and even documentation.
  • If applied successfully, a modification of a single piece of the system will not require a change in other logic or unrelated elements of the system.

Not Keeping It DRY looks like this:

Keeping it DRY on the oder hand look like this:

the configurations are symlinked here, this allows us to share the same configurations between those environments

3 Things to Use to Keep It DRY

  • Terraform supports conditionals through the syntax of a ternary operator.
  • The most common use case for conditionals is to create a conditional resource based on an input variable and the meta-parameter count.

Conditional Example

locals {
  make_bucket = "${var.create_bucket == "true" ? True : false}"
}
resource "google_storage_bucket” “twinkiebucket" {
  count = "${local.make_bucket ? 1 : 0}"
  name = "${var.bucket_name}"
  project = "${var.project_name}"
}

The create_bucket=false Conditional

output:

—> test-bucket terraform plan -var=‘create_bucket=false’
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be Persisted to local or remote state storage.

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your configuration and real physical resources that exist. As a result, no actions need to be performed.

The create_bucket=true Conditional

output:

—> test-bucket terraform plan -var=‘create_bucket=true’
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be Persisted to local or remote state storage.

An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols:

+ create
Terraform will perform the following actions:

+ google_storage_bucket.twinkiebucket ...

Use null_resource

  • The null_resource is useful when you need to do something that is not directly associated with the lifecycle of an actual resource.
  • Within a null_resource, you can configure provisioners to run scripts to do pretty much whatever you want.
  • Just like with provisioners, it is a good idea to use null_resource sparingly since it adds to the complexity of your Terraform usage.
  • Make sure, when you do use it, that you vet the scripts being called thoroughly.

Example null_resource

resource "aws_instance" "prod_cluster" {
    count = 4
    #...
}
resource "null_resource" "prod_cluster" {
    triggers = {
        cluster_instance_ids = join("," aws_instance.prod_cluster.*.id)
    }
    connection {
        host = element(aws_instance.prod_cluster.*.public_ip, 0)
    }
    provisioner "remote-exec" {
        inline = [
            "prod_cluster.sh ${join(" ", aws_instance.prod_cluster.*.private_ip)}",
        ]
    }
}

Actions that are done inside a null_resource are not managed by Terraform. If you decide to call a command to create resources in your null_resource, Terraform will not know about the resource creation, and therefore can’t manage its lifecycle and state.

you could for example add those lines to your main.tf:

resource "null_resource" "ec2_status" {
  provisioner "local-exec" {
    command = "./scripts/health.sh"
  }
}

and add the health.sh script to your repository:

#!/bin/bash
echo "   -------------------------------- "
echo "  --> Fetching Instance status."
sleep 25
instance_id=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=TheFastestManAlive" "Name=instance-state-name,Values=running" --query 'Reservations[*].Instances[*].InstanceId' --output text)
size=${#instance_id}
echo "  --> Instance ID: $instance_id"
sleep 2
instance_state=$(aws ec2 describe-instance-status --instance-ids $instance_id --query 'InstanceStatuses[*].InstanceState.Name' --output text)
size=${#instance_state}
echo "  --> Instance Status: $instance_state"
sleep 2
instance_zone=$(aws ec2 describe-instance-status --instance-ids $instance_id --query 'InstanceStatuses[*].AvailabilityZone' --output text)
size=${#instance_zone}
echo "  --> Availability Zone: $instance_zone"
sleep 2
fetch_instance_health=$(aws ec2 describe-instance-status --instance-ids $instance_id --query 'InstanceStatuses[*].InstanceStatus.Status' --output text)
echo "  --> Instance health check : $fetch_instance_health"
echo "  -------------------------------------------"

do run some trivial heal check after the deployment of our EC2 instance.

Use Functions

  • Terraform has built-in interpolation functions that allow you to use interpolation syntax embedded within strings to interpolate other values.
  • Interpolationfunctionsarecalledwiththesyntax${name(arg, arg2, ...)}.
  • The interpolation syntax allows you to call a large list of built-in functions.

The format Function

#format.tf
locals {
hostname = "${format("%s-%s-%s-%s-%04d-%s", var.region, var.env, var.app,
var.type, var.cluster_id, var.id)}" }

this Terraform code defines a local variable named hostname using the locals block. This variable is computed using the `format` function and a string template. Let's break down the components of this function:

1. `${format(...)}:` This part of the code is using Terraform's interpolation syntax `${...}` to execute the `format` function. The `format` function is used to create formatted strings by substituting values into placeholders within a template string.

2. `"${format("%s-%s-%s-%s-%04d-%s", var.region, var.env, var.app, var.type, var.cluster_id, var.id)}"`: This is the template string used in the `format` function. It consists of several placeholders, each represented by `%s` or `%04d`, which are replaced by the values provided after the template string.

  • `%s`: This is a placeholder for a string value.
  • `%04d`: This is a placeholder for a decimal integer value, formatted with leading zeros to ensure a total width of 4 characters.

The values to be substituted into these placeholders come from various Terraform variables:

  • `var.region`: This variable is expected to contain a string representing a region.
  • `var.env`: This variable is expected to contain a string representing an environment.
  • `var.app`: This variable is expected to contain a string representing an application name.
  • `var.type`: This variable is expected to contain a string representing a type.
  • `var.cluster_id`: This variable is expected to contain a numeric cluster identifier.
  • `var.id`: This variable is expected to contain a string or value that is used in the formatted hostname.

The `format` function combines these values using the specified template to generate a formatted hostname. The resulting hostname will be a string that includes the region, environment, application, type, cluster identifier (with leading zeros if necessary), and the additional identifier provided by `var.id`.

For example, if you have the following values for your variables:

  • `var.region` = "us-west"
  • `var.env` = "prod"
  • `var.app` = "web"
  • `var.type` = "frontend"
  • `var.cluster_id` = 42
  • `var.id` = "abc123"

The `hostname` variable will be computed as follows:

us-west-prod-web-frontend-0042-abc123

This computed hostname can then be used in your Terraform configuration as needed, such as for provisioning cloud resources with this specific hostname format.

The matchkeys Function

matchkeys constructs a new list by taking a subset of elements from one list whose indexes match the corresponding indexes of values in another list.

matchkeys identifies the indexes in keyslist that are equal to elements of searchset, and then constructs a new list by taking those same indexes from valueslist. Both valueslist and keyslist must be the same length.

The ordering of the values in valueslist is preserved in the result.

#matchkeys.tf
instances = [ "${matchkeys(
  google_compute_instance.compute_instance.*.self_link,
  google_compute_instance.compute_instance.*.zone,
  data.google_compute_zones.available.names[0])
}" ]

The element Function

The `element` function in Terraform is primarily used for accessing elements within a list or an array. It's a versatile function that can be used for various purposes, including:

  1. Retrieving Values: You can use `element` to retrieve specific values from a list or array. For example, you might use it to access the nth element of a list.
  2. Looping and Iteration: When combined with other Terraform constructs like `count` or `for_each`, `element` can be used to iterate over a list or array, applying the same resource configuration or operation to each element.
  3. Dynamic Resource Creation: In Terraform, you can use `element` to dynamically create multiple instances of a resource by specifying different configurations for each instance based on the elements of a list or array.
  4. Conditional Behavior: It can be used to conditionally set values or attributes in resources or variables based on the index of an element in a list.

Here's an example of how you might use the `element` function in a Terraform configuration:

variable "server_names" {
  type    = list(string)
  default = ["web-server-1", "web-server-2", "web-server-3"]
}

resource "aws_instance" "example" {
  count = length(var.server_names)
  ami   = "ami-12345678"
  instance_type = "t2.micro"
  tags = {
    Name = element(var.server_names, count.index)
  }
}

In this example, the `element` function is used to assign a unique name tag to each AWS EC2 instance being created based on the elements of the "server_names" list. It demonstrates how `element` can be used for dynamic resource creation and conditional behavior.

Overall, the `element` function is a fundamental tool in Terraform for working with lists and arrays, enabling you to make your configurations more dynamic and flexible.

Test Your Code

  • Testing code leads to greater confidence that the code will perform as expected.
  • Terraform has built-in tools to help test your code before deployment.
  • Due to Terraform’s usefulness and popularity, there are many tools which expand upon the built-in tools.

there are a few built-in commands to test your TF code:

  1. terraform fmt
  2. terraform init
  3. terraform validate
  4. terraform plan

Other Testing Tools

  • Terratest: A great, comprehensive tool by Gruntwork. This tool does not do unit testing.
  • Kitchen-Terraform: Spins up, tests, and spins down various Terraform resources.
  • Terraform-compliance: A simple tool for testing and enforcing Terraform compliance rules.

You should also read: