More Consistent Terraform Runs with Docker

For more great Terraform content, check out my course!

Introduction

Like most software, one of the most aggravating things about Terraform is its versioning. It still (as of publication) hasn’t hit 1.0 and is meandering through versions like a Hobbit in the woods. It’s getting there, but 1.0 has been promised for a very long time. Word on the street is it’s close, but I’ve been hearing that since 0.8 and we’re on 0.14.9 now.

Another annoying issue is, also like most software, its behavior on different machines. Terraform itself is pretty solid, but when you’re dealing with multiple providers, provisioners, keys, variables, and every other piece of entropy, it can become a management headache!

Note: Before we get started, please note that this technique is best used in Linux, OSX, or Microsoft’s Windows Subsystem for Linux 2. WSL1 or doing this straight from Powershell probably isn’t the best route. You might be able to get it to work, but it’s best if you’re running with Ubuntu on WSL2. The instructions to get that wired up are here: https://docs.docker.com/docker-for-windows/install/

And, that being said, if you haven’t already, you’ll also need to install Docker as well:
https://docs.docker.com/get-docker/

Enter Containers!

So how does Docker fit into this scenario and potentially solve our woes? Similar to when using it in automation, Docker can be used as an ad-hoc process, meaning the container is run, completes its purpose, and then is removed. By utilizing the hashicorp/terraform container, we can run the latest version of Terraform with a simple command! Although there’s an extra layer of abstraction that can complicate things depending on what you’re deploying, but most if not all, of these issues can be overcome with a few clever Docker Run flags. Now, before everyone skewers me for mentioning Docker and not <your favorite OCI compliant runtime>, I just want to make it perfectly clear that I am aware there are other runtimes, but Docker is still the most popular, so I’ll be using Docker for this article. Feel free to use any runtime you wish as long as the features are the same.

Ok, enough blabbering, let’s build something! As many of you know by now, I like to build stuff vs. talk about it. Let’s build something simple this round, but it’ll be something that will be able to utilize several snags and solutions you may encounter while running Terraform in Docker. Let’s deploy a Docker image and container using Terraform. Go ahead and create a main.tf file and add some Terraform code:

Note: If you want to learn how to write deployments like this, and much more, check out my course!

terraform {
  required_providers {
    docker = {
      source = "kreuzwerker/docker"
    }
  }
}


provider "docker" {}

resource "null_resource" "dockervol" {
  provisioner "local-exec" {
    command = "echo ${docker_container.nodered_container.name} >> containers.txt"
  }
  provisioner "local-exec" {
    command = "rm -f containers.txt"
    when = destroy
  }
}

resource "docker_image" "nodered_image" {
  name = "nodered/node-red"
}

resource "random_string" "random" {
  length  = 4
  special = false
  upper   = false
}


resource "docker_container" "nodered_container" {
  name  = join("-", ["nodered", random_string.random.result])
  image = docker_image.nodered_image.latest
  ports {
    internal = 1880
    external = 1880
  }
}

Ok, so what this code is doing is creating a NodeRED container from the NodeRED image and then creating a containers.txt file that will contain the name of the container you create, illustrating that the Terraform binary still has access to your local filesystem. The container will also be exposed on port 1880, so feel free to access it using http://localhost:1880 if you wish to play around with it, but make sure you add a volume if you want to do anything fancy as the data will not persist. Once the deployment is destroyed, it will remove everything, including the containers.txt file.

So now that you have your file created and code inserted, let’s get down to business!

Using the Terraform Docker Container

Typically, you would install Terraform using apt or by downloading the binary, but this time, we’re going to do it the fun way. Unfortunately, you still need to install Docker, so make sure you’ve done that. Once everything is installed, let’s get to work! You can check out the Terraform Container docs here: https://hub.docker.com/r/hashicorp/terraform

As you can see, the docs are pretty bare, especially for Hashicorp standards. Typically, their docs are phenomenal, but I guess they focus more on the binary usage itself vs. containerized use cases. So let’s make this thing useful!

First, let’s go ahead and just pull the latest image. Run:

docker pull hashicorp/terraform:light 

And you should see the image being pulled:

Now, if you run:

docker history --no-trunc hashicorp/terraform:light 

you can see the “ENTRYPOINT” directive is set to ["/bin/terraform"]. This shows that when you run this container, it’s going to run the terraform command. This is exactly what we’re looking for. So let’s try it by running the container. We’ll set the container to remove itself on creation with --rm and to be interactive on the terminal with -it:

docker run --rm -it hashicorp/terraform:light version

You should see the current version of Terraform output to your console. In the case of this article, it’s Terraform v0.14.9, but perhaps by the time you’re reading this, it will be Terraform 1.x and we can all celebrate, but I digress.

So this is great, we now know that Terraform is working just as if the binary were installed on our machine, well, almost. Go ahead and run:

docker run --rm -it hashicorp/terraform:light init

Well, that’s not what we were hoping for! Since Terraform is running within a container, it has no access to the files in our current directory. Let’s remedy that by mounting a volume to the current working directory by utilizing the “Print Working Directory”, or PWD command. We’ll mount the directory to the directory /data within the container and set /data as the working directory. This will provide the container read/write access to our current directory:

docker run --rm -it -v $PWD:/data -w /data hashicorp/terraform:light init

Alright, so we’re closer! Initialization was successful and all of our providers have been installed! And, if you look at your directory, you can see the Terraform files we expect after a fresh init:

Alright, so now init works, let’s go ahead and attempt a plan and see what breaks next:

So now we have another issue to solve. We need to connect our Docker container to the local Docker socket of the machine. Now, I want to say, I did not come up with the exact syntax on my own. I used the blog linked below, and I think you’ll find a lot of other interesting tidbits that may come in handy as you make this solution work for you:

https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

To utilize our machine’s local Docker socket within the container, we just need to add the socket as a volume to the Docker container like so:

docker run --rm -it -v $PWD:/data -w /data -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker hashicorp/terraform:light plan 

Now run that command and let’s see what happens:

Awesome! It worked! So let’s apply this puppy!

docker run --rm -it -v $PWD:/data -w /data -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker hashicorp/terraform:light apply --auto-approve

We did it! Nice! Everything appears to have applied just fine! If you run a docker ps, you’ll see that the container is up and running:

And if you open containers.txt, you should see the name of the running container within. Before we destroy this stack, let’s make this a little bit easier by using an alias. Go ahead and run:

alias tform="docker run --rm -it -v $PWD:/data -w /data -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker hashicorp/terraform:light"

You shouldn’t have any feedback. Once you’ve done that, run:

tform state list

You should see all of your resources listed! We’ve now simplified the command extensively and we can now run that entire Docker string by using one command:

Perfect! Now, go ahead and destroy:

tform destroy --auto-approve

Now that we’ve seen how this works, let’s make this setup a little more permanent. Depending on your OS, you may want to add this command to your .bashrc file to ensure it persists reboots, logouts, etc. So, if you’re on an OS that supports this file, let’s do this:

Within your ~/.bashrc file, add this line to the very bottom:

alias tform="docker run --rm -it -v $PWD:/data -w /data -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker hashicorp/terraform:light"

And that’s all you need to do! Now, anytime you log back in as your user, you’ll be greeted with your fancy new command!

Alright! So now you’ve got an excellent way to utilize Terraform, manage versioning, and deploy in automation with ease!

Other Fun Things

Well, that’s super neat! Definitely play around with that, there are a lot of things you can do involving automation, custom Dockerfiles. For instance, if you require the Python binary, you can potentially create a new Dockerfile from the Python image and add the files from the Terraform image into it:

# Dockerfile
FROM python 
COPY --from=hashicorp/terraform:light /bin/terraform /bin/ 
ENTRYPOINT ["/bin/terraform"]

You can do the same with Jenkins and other CI/CD platforms as well. The possibilities are endless! You can, of course, utilize any other argument for Docker run as well, such as Environment Variables. If you need to pass an envar, you can run something like:

docker run --rm -it -v $PWD:/data -w /data -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker -e TF_TZ=Europe/London hashicorp/terraform:light

Then you can access that envar within your Terraform scripts using standard syntax to access those variables. But I’ll let you experiment with that.

Alright, so that’s all for this article. If you liked it, please check out my course at https://courses.derekops.com/terraform to learn a lot more about Terraform, and don’t forget to Terraform Apply Yourself!

Resources and More Reading

https://medium.com/@audun.nes/how-to-use-the-official-terraform-docker-image-2609982114b9

https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

https://nodered.org/docs/getting-started/docker

https://www.reddit.com/r/docker/comments/bugpt0/running_terraform_in_docker/

https://docs.docker.com/get-docker/

https://www.terraform.io/downloads.html

https://hub.docker.com/r/hashicorp/terraform/

https://courses.derekops.com/terraform