In the previous blog post, I wrote about the principles and best practices of IaC design. To complete the topic of standardization, it is important to consider a standardized codebase. This helps you collaborate more effectively with your teammates in maintaining Terraform modules within your organization and supports development teams in doing so as well. In this post, I’m going to share my experiences with naming conventions for writing a standard block of OpenTofu and Terraform code. Some of these conventions are useful for creating a standard Terraform file, while others are necessary due to restrictions in providers’ APIs, such as those from AWS and Azure.

General Conventions

• Always use _ (underscore) instead of — (dash)

  resource "aws_db_instance" "dev_db" {
    ...
    name = "backend_db_instance"
    ...
  }

• Only use lowercase letters and numbers

  resource "aws_key_pair" "ops" {
    key_name   = "roozbeh_key_1"
    public_key = "ssh-rsa ..."
  }

Resource and Data Source Conventions

• Do not repeat resource type in resource name (not partially, nor completely)

  resource "aws_route_table" "public" {
    vpc_id = aws_vpc.example.id
    ...
  }

• Resource name should be named this if there is no more descriptive and general name available

  resource "aws_nat_gateway" "this" {
    allocation_id = aws_eip.nat.id
    subnet_id     = aws_subnet.example.id
    ...
  }

• Always use singular nouns for names

  resource "aws_eip" "loadbalancer" {
    instance = aws_instance.web.id
    vpc      = true
  }

• Use - inside arguments values and in places where value will be exposed to a human (eg, inside DNS name of RDS instance or Route 53 record)

  resource "aws_route53_record" "www" {
    zone_id = aws_route53_zone.primary.zone_id
    name    = "www.example-domain.com"
    type    = "A"
    ttl     = "300"
    records = [aws_eip.lb.public_ip]
  }

• Include count an argument inside resource blocks as the first argument at the top and separated by a newline after it

  resource "aws_instance" "web" {
    count = "5"
    ...
  }

• Include tags the argument, if supported by resources as the last real argument, followed by depends_on and lifecycle, if necessary. All of these should be separated by a single empty line:

  resource "aws_nat_gateway" "this" {
    count         = "1"
  
    ...
  
    tags = {
      Name = "..."
    }
  
    depends_on = ["aws_internet_gateway.this"]
  
    lifecycle {
      create_before_destroy = true
    }
  }

• When using conditions in count argument use a boolean value if it makes sense, otherwise, use length or other interpolation:

  count = "${length(var.public_subnets) > 0 ? 1 : 0}"

• To make inverted conditions don’t introduce another variable unless necessary, use 1 - boolean value instead:

  count = "${1 - var.create_public_subnets}"

Variable Conventions

• Don’t reinvent the wheel in resource modules — use the same variable names, description, and default as defined in the “Argument Reference” section for the resource you are working on.

• Use type = "list" declaration if there is default = []:

  variable "availability_zone_names" {
    type    = list(string)
    default = [
      "eu-central-1a"
      "eu-central-1b"
      "eu-central-1c"
    ]
  }

• Use type = "map" declaration if there is default = {} :

  variable "images" {
    type = "map"
  
    default = {
      eu-central-1 = "image-1234"
      eu-west-1    = "image-4567"
    }
  }

• Use the plural form in the name of variables of type list and map :

  variable "users" {
    type    = "list"
    ...
  }
  
  variable "images" {
    type = "map"
    ...
  }

• When defining variables order the keys: description , type, default . Always include description for all variables even if you think it is obvious.

  variable "key" {
    description = "description"
    type        = "string"
    default     = "value"
  }

Outputs

A name for the outputs is important to make them consistent and understandable outside of its scope (when the user is using a module it should be obvious what type and attribute of the value are returned).

• The general recommendation for output names is that they should be descriptive of the value they contain and less free-form than you would normally want.

• Good structure for names of output looks like {name}_{type}_{attribute} , where:
  1. {name} is a resource or data source name without a provider prefix. {name} for aws_subnet is subnet, foraws_vpc it is vpc.
  2. {type} is a type of resource source.
  3. {attribute} is an attribute returned by the output

• If the output is returning a value with interpolation functions and multiple resources, {name} and {type} there should be as generic as possible (this is often the most generic and should be preferred).

• If the returned value is a list it should have a plural name.

• Always include description for all outputs even if you think it is obvious.

Conclusion

This approach improves collaboration between you and your teammates, ensuring that everyone is able to work together more efficiently when maintaining and updating Terraform modules across your organization. By establishing a standardized process, it also provides better consistency and clarity, which not only streamlines the workflow within your immediate team but also enables development teams throughout the organization to follow best practices, reduce errors, and achieve better integrations during the development and deployment of infrastructure as code.

There are several principles in designing an Infrastructure as Code (IaC) that DevOps and Cloud Engineering teams should follow from day zero or should be considered for refactoring an existing setup. IaC is not just well-structured code in HCL formatted files and sequential commands (init, plan, apply, etc); it also requires a broad design and, like any other system must have a scalable and extensible architecture.

This blog post is the conclusion of my experiences since 2016 when I started working with Terraform. In the following, eleven principles and best practices of an IaC design are explained briefly although each one would require a separate post or series of posts to cover in detail.

GitOps

That’s a “Must-Have”! Transparency between developers for any changes through pull requests, using Git tags, and setting up a pipeline for automated planning and applying via GitOps is essential. In the following, all Git-related principles will be explained. Therefore, it’s important to incorporate GitOps into all aspects of IaC design.

DRY

Don’t Repeat Yourself! In an IaC codebase, there is often shared or common code across modules, outputs, environments, stacks, regions, etc. As the setup grows, the code tends to be repeated making maintenance increasingly difficult over time. To simplify your IaC structure, consider integrating Terraform or OpenTofu with a thin wrapper like Terragrunt, or include/call reusable code from parent directories in Pulumi. This approach keeps the codebase simpler, shorter, and easier to maintain. So, avoid designing with pure Terraform or OpenTofu.

Isolation

Assign a separate repository for each team. By isolating each team’s resources, this approach reduces pull request traffic and simplifies managing change requests. Besides that, issues with the plan/apply pipeline or broken codebase in one repository won’t affect other teams. Also, maintaining the CODEOWNERS file and assigning reviewers becomes much easier.

Directory Structure

Organizing resource templates simplifies your daily tasks. One of the operations with IaC is managing existing code by adding, updating, or deleting resources. It’s important to find them easily, especially when facing with a large environment. So, it’s more convenient to follow a hierarchy and group template files based on provider, environment, stack, or any other structure that meets your needs.

Modularization

Instead of repeating yourself by writing raw Terraform “resource” blocks, create modules based on developers’ use cases, call them with Terragrunt, and reuse them across multiple stacks and environments. This way, any changes to the modules will automatically be reflected in all the resources across stacks and environments, making it easier to enforce DevOps standards and policies consistently through the modules.

Versioning

It’s important to keep the modules in a separate Git repository, apart from the teams’ repositories. This allows you to manage module versioning using Git tags and protects existing resources in the teams’ repositories from new or breaking changes. Resources within stacks can then be migrated to the newer version when they are ready or once they are compatible with the updated parameters.

Remote State

Keeping state files in remote storage solutions like S3 is a common practice. Storing them remotely allows you to use the versioning feature, enabling you to revert to previous versions in case of state file corruption. This approach is also essential when your entire IaC configuration is managed in Git and you rely on pipelines to plan and apply changes. Also, remote storage is much safer than storing state files locally, providing better security and reliability.

Encrypted Secrets

DO NOT store credentials in plain text within templates. Secrets should always be stored in a secure service like AWS Secrets Manager (or similar services from other providers) or HashiCorp Vault. These secrets can then be fetched using “data source” blocks in modules and passed as input variables in your IaC repositories and stacks. This ensures that secrets remain safe and are never stored in plain text, even in state files. Additionally, this approach supports secret rotation and can be integrated into Kubernetes deployments, enabling secrets to be shared securely between your IaC and containerized workloads.

Auto-Tagging

An auto-tagging mechanism is highly beneficial, allowing each resource managed by IaC to be tagged based on the directory structure and hierarchy. Tags can include the Git repository name, environment, stack, resource type, resource name, and an additional tag like “IaC = true.” This approach simplifies tag tracking to determine where the IaC code resides, where the resource originated from, and whether it was created by IaC. Moreover, this tagging strategy helps the FinOps team in tracking the costs of specific environments, teams, or stacks using the provider Cost Explorer or third-party tools like Looker.

Import-as-Code

The import block was recently introduced by OpenTofu and Terraform. At first glance, it may seem a bit static, but when combined with the locals block in modules and a map of strings as an input variable, it can significantly simplify the process for DevOps teams. This approach enables the implementation of an import-as-code mechanism, allowing resources to be imported by simply defining the corresponding templates. As a result, there’s no need for the tedious use of terraform import commands, making it much easier to import manually created resources—especially when providing DevOps services in a self-service model to developers.

Policy-as-Code

With Policy-as-Code, DevOps teams can more easily define and enforce policies, standards, and best practices across development teams. Policies such as blocking all network protocols and ports to the internet through security groups, preventing IAM FullAccess, or restricting the creation of expensive EC2 instance types, among others, can be controlled and managed using tools like Open Policy Agent and HashiCorp Sentinel. These tools can be integrated at the module level, providing significant benefits in terms of security enforcement, performance optimization, and cost control.

Pipeline

After implementing all principles, especially the Git-related ones, a pipeline becomes essential to achieve full automation. The process starts when you make a change to your IaC templates, create a pull request, and request a code review from a teammate. Like modern software development workflows using CI/CD, the pipeline should then roll out your infrastructure changes to the target environment. This can be accomplished using various solutions, such as an open-source tool like Atlantis, integrating with an existing CI/CD tool like Jenkins, or leveraging cloud-based platforms like Terraform Cloud or Pulumi Cloud. The pipeline continuously monitors the Git repositories via webhooks, triggers a plan, and waits for your approval to apply the changes.

Documentation

Last, but not least, it’s crucial to document every aspect of your IaC for the benefit of other teammates in DevOps. As the design becomes complex and requires periodic maintenance, clear documentation will be essential. Additionally, providing detailed documentation for modules, explaining how to onboard new resources into IaC, and outlining the required and optional input parameters and variables is invaluable. This is especially helpful when your DevOps workflows are designed for a self-service model, enabling developers to easily understand and use the infrastructure.

Conclusion:

This summarizes everything I’ve learned about designing an IaC system over the years. Each point mentioned in this blog post deserves a dedicated article for a more detailed explanation, and I plan to write about them in the future.

As new features are introduced by tools like HashiCorp, OpenTofu, Pulumi, and others, IaC may become even more complex. As a result, IaC is evolving into a specialized field within DevOps, making it crucial to continuously learn and stay up to date with market trends.

By initiating deployment and provisioning of cloud infrastructure resources using an IaC tool, three entities to be built upon the development and apply. I call them sources of truth that they should be in sync to keep the IaC setup properly. Nowadays IaC solutions like OpenTofu, Terraform, and Pulumi follow this structure and approach. These sources are:

• Upstream Resources: The resources that are created by IaC tools on Cloud providers, such as EC2 instances, RDS databases, VPC networks, and so on.

• State File: A file that stores the state of upstream resources in JSON format. it’s a reference used by the IaC tool to what has been created on the cloud and stores the state, IDs, links, etc.

• Template: A file developed by a human and depends on the IaC tool, in various formats like HCL, Python, Golang, etc. This file defines, the desired state of each resource on the cloud. States like EC2 instance types, S3 bucket name, RDS username, and many other parameters for each resource type.

These three sources form the foundation of an Infrastructure as Code setup and understanding each component is crucial for successfully managing your IaC.

Pushing components out of sync is unavoidable and can occur for various reasons, such as quick response to a service outage or incident by modifying infrastructure/upstream resources or human errors. These events cause IaC to become desynchronized state and understanding how these changes affect your setup is crucial for maintaining your IaC system.

Let’s dive into each source of truth and explore what to expect when any of them is out of sync state. We will investigate each source of truth through three events: Creation, Modification, and Deletion.

Upstream Resource:

This situation occurs during an incident or application outage, and quick actions and manual changes on infrastructure resources are necessary to resolve the issue. Also, It is especially common when someone is unfamiliar with IaC and prefers to make direct manual modifications.

• Deleting and recreating a resource can trigger creation by the IaC apply process. The apply process might fail if the resource is referenced in the state file by its name as its ID (like an S3 bucket)

• Modifying a resource triggers an update during the IaC plan/apply process due to re-align sources of truth. This situation is usually referred to as “Drift” when the actual state of the resource diverges from the desired state defined in the state file and the template.

• Removing a resource can trigger recreation during the IaC plan/apply process, as the IaC tool will attempt to restore it to match the desired configuration defined in the template.

State File:

Changes in state files are rare, especially when stored in remote backends like cloud object storage (S3). However, they can occur during manual maintenance tasks, such as modifying or moving states between multiple state files.

• Creating a state can trigger deletion during the IaC plan/apply process if resources with the same ID already exist upstream. This occurs because the state file prioritizes the template as the primary source of truth and attempts to synchronize upstream resources with the desired state defined in the template.

• Modifying the state file triggers change during the plan/apply process. However, if the template and upstream resources remain unchanged, no actual changes will occur in the infrastructure; the modification will only affect the state file.

• Removing a state triggers creation during the plan/apply process, and the IaC tool attempts to synchronize the resource with the template, which serves as the primary source of truth. Additionally, the apply process might fail if the resource exists upstream and is identified by its name as ID ( like S3 bucket)

Template File:

Finally, any changes made to the template file directly reflect your defined and desired configuration. Creation in the template results in resource creation, modifications trigger changes, and removals result in upstream resource termination. This process is the standard procedure for managing your infrastructure by IaC.

Conclusion:

In IaC changes and misalignments are unavoidable, often requiring manual adjustments to restore synchronization. However, it’s best to make changes to upstream resources using the template files. This way, you can minimize manual changes to the state files, helping to maintain consistency and prevent problems. Understanding the effects of changes on each component is crucial for analyzing the situation and making informed decisions on infrastructure stability.

These days, the lockdown situation is a good opportunity to grow your skills by learning new technologies in your spare time or follow the community by reading articles or listening to podcasts. Specially for multi-thread brains, when you are working in a noisy environment at home, listening to podcasts in parallel, can help you to focus on your work, better.

Although you may know some or all of them it’s a good idea to list them here for people who want to be a hero in related technologies. In the following, I’m going to introduce 7 podcasts to follow for every single DevOps, SRE, or Cloud engineer and all are available on Spotify, Apple Podcasts, and Google Podcasts.

Command Line Heroes

This series of podcasts is produced by Red Hat every week and 3 seasons are published up to this time. The stories of this podcast are about the people transforming technology from the command line.

Kubernetes Podcast From Google

A weekly podcast focused on what’s happening in the Kubernetes community covering Kubernetes, cloud-native applications, and other developments in the Kubernetes community. Co-hosts are Adam Glick and Craig Box.

Google Cloud Platform Podcasts

The Google Cloud Platform Podcast, coming to you every week. Discussing everything on Google Cloud Platform from App Engine to Big Query. This is an official podcast by Google.

AWS Podcast

The AWS Podcast is the definitive cloud platform podcast for developers, dev ops, and cloud professionals seeking the latest news and trends in storage, security, infrastructure, serverless, and more. Join Simon Elisha and Jeff Barr for regular updates, deep dives, and interviews. Whether you’re building machine learning and AI models, open-source projects, or hybrid cloud solutions, the AWS Podcast has something for you.

The Cloudcast

The Cloudcast is the industry’s leading, independent Cloud Computing podcast. Since 2011, co-hosts Aaron Delp & Brian Gracely have interviewed technology and business leaders who are shaping the future of computing. Topics are mostly about Cloud Computing, Open Source, AWS, Azure, GCP, Serverless, DevOps, Big Data, ML, AI, Security, Kubernetes, AppDev, SaaS, PaaS, CaaS, IoT, and many more.

The Real Python Podcast

A weekly Python podcast hosted by Christopher Bailey with interviews, coding tips, and conversation with guests from the Python community. The show covers a wide range of topics including Python programming best practices, career tips, and related software development topics. Listen to that every Friday morning to hear what's new in the world of Python programming and become a more effective Pythonista.

Go Time

A diverse panel and special guests discuss cloud infrastructure, distributed systems, microservices, Kubernetes, Docker, and also Go! Panelists include Mat Ryer, Ashley McNamara, Johnny Boursiquot, Carmen Andoh, Jaana B. Dogan (JBD), Mark Bates, and Jon Calhoun. This show records LIVE every Tuesday at 3 pm US Eastern.

As the first official blog post, I would keep it simple and short. This is post-0. But before we get started, it’s necessary to introduce myself briefly to whom it may concern.

Several years ago for more than 10 years, I was a blogger mostly at System Engineering topics. As I moved to Germany 4 years ago and my career slowly switched to development and operation, like the thing we call that DevOps these days, I decided to back to blogging for sharing my experiences and knowledge that I daily gain.

The previous blog archive is available but as the world of software engineering is rapidly changing every day, I think so it’s not worth restoring and continue that anymore. So that’s why I start this blog.

I will try to keep it update every weekend either on Saturday or Sunday and like before, mostly at tech topics. For sure nobody’s perfect but I try to be nobody!

My professional social network links are accessible on sidebar in left. If you are interested in my thoughts, posts. tweets and generally my activities, follow me!