Software Engineering

Building a Kubernetes Cluster on AWS EKS using Terraform

Part I – configuring Terraform for AWS & building a project structure

Starting this year, Amazon’s new web service, Elastic Container Service for Kubernetes (in short: AWS EKS) will become available in more and more regions. Earlier this year, it was already enabled for european regions. It starts up and manages a Kubernetes master which integrates with Amazons other services, making it easier to start up a service management platform in the cloud.

In esentris project “Lithosphere”, we are capturing our business processes in a microservice architecture relying entirely on cloud resources. We previously used OpenShift, and while that platform has a lot going for it, we still wanted to try out EKS to see how it works and what we can do with it.

To do this, we started setting up infrastructure management using Terraform, which is “a tool for building, changing, and versioning infrastructure safely and efficiently”. Working with Terraform means putting the infrastructure that you want to build into code with all the resources you need and their relations. Once that is done, you can use those files to make Terraform apply those resources to your cloud provider of choice.

In this article series, I want to walk you through the entire process of building a terraform representation of a EKS cluster. This includes a project structure you can use for multiple stages and Continous Integration and the multiple layers of infrastructure and security interactions that need to be set up for a fully working Kubernetes cluster in EKS.

Use a provider to define your connection to the cloud

First off: you can structure the parts of your terraform files very freely. Terraform collects all .tf-files in a folder and evaluates them together, meaning you can have very short or long files based on your project circumstances or personal preference. We will start with setting up the provider we use to contact AWS:

Providers are the connectors used by Terraform to address the platform they need to build on. They are mostly written in Go and provided via official Terraform repositories like this GitHub-page. No worries though: you mostly don’t need to manually install any. Terraform finds and installs providers you defined in your .tf-files once you start the installation process.

The Terraform provider for AWS uses a access key and a secret key. You can set up an account for your Terraform client in AWS IAM – just remember that that account will need permissions to edit and read a lot of different resources on your AWS account to build your cluster. There are multiple ways you can store the credentials so the provider finds them. They can be added directly into the provider definition (like above), or stored in environment variables or a credentials file, equal to the one you might already have set up if you use the aws-cli. In general I recommend not storing the credentials directly in your .tf-files – you shouldn’t push those into your version control. Instead, you can use variables that you input when you start the actual installation of at a later point:

You also need to define those variables so Terraform can work with them:

You can set up another file to declare the content of your variables too. Terraform loads all files with the ending .tfvars when it runs to populate your variables, meaning you just have to put it in the same folder you run terraform in:

Managing the resource state for distributed projects

Terraform uses the provider to read the state of resources managed by it whenever it is started up. It stores this information in a collection of files called the state. When you use Terraform in a project where multiple people work on the same infrastructure, you need to make sure to share this state between all users to avoid overriding or duplicating resources. There are multiple ways to do this. We are using another AWS service to share the state between clients: S3, which is basically just a data hosting service.

Because Terraform builds a state before every action, you can’t really build the S3 resource you need to store the state from Terraform. Setting up a bucket in S3 is easy though. Once your bucket is set up, we add it to the Terraform configuration like so:

With this, we use a bucket with the name “terraform-dev” to store our state files. Additionally, we use AWS DynamoDB to manage another piece of state information: the state-lock. Whenever Terraform starts reading or updating your resources, it tries to check that no other Terraform client is editing the same resources at that time. To do that, it locks the resources from being accessed. This just means that Terraform uses the DynamoDB-Table to basically look up a boolean value. Setting up that DynamoDB-Table isn’t difficult either. Just make sure the primary key of that table is called “LockID” so that Terraform can manage the table properly.

Ready to build

With this, our basic Terraform setup for AWS is done: we’ve set up a provider to make sure Terraform can properly connect to our AWS-Account, and we set up some infrastructure to make sure the state is shared between all users of our resource definitions. You can try running “terraform init” in the folder – if everything is set up right, terraform will download the AWS provider and configure the backend.

You can check out the code in my GitHub-Repository for the article series – don’t forget to enter your values for the access keys and region in the .tfvars file and the bucket configuration before running it!

In the next article, we will start creating the actual infrastructure for our EKS cluster. We need to start setting up our network boundaries first, which includes security groups, subnets and similar resources.