This document provides guidelines and recommendations for effective development with Terraform across multiple team members and work streams. This guide is not an introduction to Terraform. For an introduction to using Terraform with Google Cloud, see Get started with Terraform. The following recommendations cover basic style and structure for your Terraform configurations. The recommendations apply to reusable Terraform modules and to root configurations. Terraform modules must follow the standard module structure. Place examples in an examples/ folder, with a separate subdirectory for each example. Avoid giving every resource its own file. Group resources by their shared purpose. Name all configuration objects using underscores to delimit multiple words. This practice ensures consistency with the naming convention for resource types, data source types, and other predefined values. This convention does not apply to name arguments. To simplify references to a resource that is the only one of its type (for example, a single load balancer for an entire module), name the resource main. To differentiate resources of the same type from each other (for example, primary and secondary), provide meaningful resource names.
Make resource names singular. In the resource name, don't repeat the resource type. Give variables descriptive names that are relevant to their usage or purpose. Google Cloud APIs don't have standard units, so naming variables with units makes the expected input unit clear for configuration maintainers. For units of storage, use binary (powers of 1024) unit prefixes (kilo, mega, giga). For all other units of measurement, use decimal (powers of 1000) unit prefixes. This usage matches the usage within Google Cloud. For variables that have environment-independent values (such as disk size), provide default values. This way, the calling module must provide meaningful values. Adding a variable with a default value is backwards-compatible. Removing a variable is backwards-incompatible. In cases where a literal is reused in multiple places, you can use a local value without exposing it as a variable. Provide meaningful descriptions for all outputs. Auto-generate descriptions on commit with tools like terraform-docs. Output all useful values that root modules might need to refer to or share. Especially for open source or heavily used modules, expose all outputs that have potential for consumption. Don't pass outputs directly through input variables, because doing so prevents them from being properly added to the dependency graph. To ensure that implicit dependencies are created, make sure that outputs reference attributes from resources. Put data sources next to the resources that reference them. For example, if you are fetching an image to be used in launching an instance, place it alongside the instance instead of collecting data resources in their own file. To fetch data relative to the current environment, use variable or resource interpolation.
Avoid custom scripts, if possible.
Use scripts only when necessary. The state of resources created through scripts is not accounted for or managed by Terraform. Avoid custom scripts, if possible. Use them only when Terraform resources don't support the desired behavior. Any custom scripts used must have a clearly documented reason for existing and ideally a deprecation plan. Organize helper scripts that aren't called by Terraform in a helpers/ directory. If helper scripts accept arguments, provide arguments-checking and --help output. Static files that Terraform references but doesn't execute (such as startup scripts loaded onto Compute Engine instances) must be organized into a files/ directory. Place lengthy HereDocs in external files, separate from their HCL. Reference them with the file() function. For files that are read in by using the Terraform templatefile function, use the file extension.tftpl. Templates must be placed in a templates/ directory. For stateful resources, such as databases, ensure that deletion protection is enabled.
Never have more than one ternary operation in a single line.
All Terraform files must conform to the standards of terraform fmt. Limit the complexity of any individual interpolated expressions. If many functions are needed in a single expression, consider splitting it out into multiple expressions by using local values. Never have more than one ternary operation in a single line. Instead, use multiple local values to build up the logic. To instantiate a resource conditionally, use the count meta-argument. Be sparing when using user-specified variables to set the count variable for resources. Terraform can't generate a plan. Instead, Terraform reports the error value of count cannot be computed. Reusable modules: Publish reusable modules to a module registry. Open source modules: Publish open source modules to the Terraform Registry. Private modules: Publish private modules to a private registry. For modules that are meant for reuse, use the following guidelines in addition to the previous guidelines. Including API activation makes demonstrations easier. For all shared modules, include an OWNERS file (or CODEOWNERS on GitHub), documenting who is responsible for the module.
Before any pull request is merged, an owner should approve it. Sometimes modules require breaking changes and you need to communicate the effects to users so that they can pin their configurations to a specific version. Make sure that shared modules follow SemVer v2.0.0 when new versions are tagged or released. When referencing a module, use a version constraint to pin to the major version. Shared modules must not declare providers or backends. Instead, declare providers and backends in root modules. Unless proven otherwise, assume that new provider versions will work. Allow flexibility in the labeling of resources through the module's interface. Variables and outputs let you infer dependencies between modules and resources. Without any outputs, users cannot properly order your module in relation to their Terraform configurations. For every resource defined in a shared module, include at least one output that references the resource. Inline modules let you organize complex Terraform modules into smaller units and de-duplicate common resources. Treat inline modules as private, not to be used by outside modules, unless the shared module's documentation specifically states otherwise. Terraform doesn't track refactored resources. If you start with several resources in the top-level module and then push them into submodules, Terraform tries to recreate all refactored resources. To mitigate this behavior, use moved blocks when refactoring. Outputs defined by internal modules aren't automatically exposed.
To share outputs from internal modules, re-export them. Root configurations (root modules) are the working directories from which you run the Terraform CLI. Make sure that root configurations adhere to the following standards (and to the previous Terraform guidelines where applicable). Explicit recommendations for root modules supersede the general guidelines. It is important to keep a single root configuration from growing too large, with too many resources stored in the same directory and state. All resources in a particular root configuration are refreshed every time Terraform is run. This can cause slow execution if too many resources are included in a single state. A general rule: Don't include more than 100 resources (and ideally only a few dozen) in a single state. To manage applications and projects independently of each other, put resources for each application and project in their own Terraform directories. A service might represent a particular application or a common service such as shared networking. Nest all Terraform code for a particular service under one directory (including subdirectories). When deploying services in Google Cloud, split the Terraform configuration for the service into two top-level directories: a modules directory that contains the actual configuration for the service, and an environments directory that contains the root configurations for each environment. Each environment directory (dev, qa, prod) corresponds to a default Terraform workspace and deploys a version of the service to that environment. Use only the default workspace. Workspaces alone are insufficient for modeling different environments.
Export as outputs information from a root module that other root modules might depend on.
To share code across environments, reference modules. Typically, this might be a service module that includes the base shared Terraform configuration for the service. In service modules, hard-code common inputs and only require environment-specific inputs as variables. Export as outputs information from a root module that other root modules might depend on. In particular, make sure to re-export nested module outputs that are useful as remote state. Other Terraform environments and applications can reference root module-level outputs only. By using remote state, you can reference root module outputs. To allow use by other dependent apps for configuration, export to remote state information that's related to a service's endpoints. In root modules, declare each provider and pin to a minor version. This allows automatic upgrade to new patch releases while still keeping a solid target. For root modules, provide variables by using a.tfvars variables file. For consistency, name variable files terraform.tfvars. Command-line options are ephemeral and easy to forget. Using a default variables file is more predictable.
For root modules, the.terraform.lock.hcl dependency lock file should be checked into source control. This allows for tracking and reviewing changes in provider selections for a given configuration. A common problem that arises when using Terraform is how to share information across different Terraform configurations (possibly maintained by different teams). Generally, information can be shared between configurations without requiring that they be stored in a single configuration directory (or even a single repository). The recommended way to share information between different Terraform configurations is by using remote state to reference other root modules. Cloud Storage or Terraform Enterprise are the preferred state backends. For querying resources that are not managed by Terraform, use data sources from the Google provider. For example, the default Compute Engine service account can be retrieved using a data source. Don't use data sources to query resources that are managed by another Terraform configuration. Doing so can create implicit dependencies on resource names and structures that normal Terraform operations might unintentionally break. Best practices for provisioning Google Cloud resources with Terraform, are integrated into the Cloud Foundation Toolkit modules that Google maintains. This section reiterates some of these best practices.
IAM module from Google.
In general, we recommend that you bake virtual machine images using a tool like Packer. Terraform then only needs to launch machines using the pre-baked images. If pre-baked images are not available, Terraform can hand off new virtual machines to a configuration management tool with a provisioner block. We recommend that you avoid this method and only use it as a last resort. Terraform should provide VM configuration information to configuration management with instance metadata. IAM associations, where the Terraform resources serve as the only source of truth for what permissions can be assigned to the relevant resource. If the permissions change outside of Terraform, Terraform on its next execution overwrites all permissions to represent the policy as defined in your configuration. This might make sense for resources that are wholly managed by a particular Terraform configuration, but it means that roles that are automatically managed by Google Cloud are removed-potentially disrupting the functionality of some services. IAM module from Google. As with other forms of code, store infrastructure code in version control to preserve history and allow easy rollbacks.
The main branch is the primary development branch and represents the latest approved code. The main branch is protected. Development happens on feature and bug-fix branches that branch off of the main branch. For repositories that include root configurations that are directly deployed to Google Cloud, a safe rollout strategy is required. We recommend having a separate branch for each environment. Thus, changes to the Terraform configuration can be promoted by merging changes between the different branches. Make Terraform source code and repositories broadly visible and accessible across engineering organizations, to infrastructure owners (for example, SREs) and infrastructure stakeholders (for example, developers). This ensures that infrastructure stakeholders can have a better understanding of the infrastructure that they depend on. Encourage infrastructure stakeholders to submit merge requests as part of the change request process. Never commit secrets to source control, including in Terraform configuration. Instead, upload them to a system like Secret Manager and reference them by using data sources.