Continuous improvement is better than delayed perfection. Mark Twain

In this short article, you will learn how you can leverage the Terraform provider alias to setup multiple provider configurations with multiple credentials. The main reason to use an alias is usually to support multiple regions or projects for a cloud platform. Here we will focus on using multiple sets of credentials, why use this approach and how it could benefit your Terraform configuration and the security of your infrastructure.

Use cases

Typically, you would not use a single set of credentials to manage your entire infrastructure. You would also most likely not rely on a single Terraform configuration to manage your entire infrastructure, because it would take time to refresh and update the state when managing a lot of resources. The best practice is to split your infrastructure into multiple Terraform configurations (called root modules), which usually use provider configurations with different credentials.

However, a root module can benefit from using multiple credentials for the same Terraform provider configuration.

The main reason to use a Terraform configuration with multiple credentials is to apply the principle of least privilege. This allows your configuration to rely on different credentials with specific permissions, and limit the risk of compromising your infrastructure managed by the same Terraform configuration.

A Terraform provider usually requires some credentials to perform some actions on protected resources. By using aliases, we can use one provider to create some credentials which are then used by a second provider within the same configuration.

Example

Let’s have a look at a simple example. The code for this example is fully available in GitLab. In this scenario, we want to create using Terraform a new Google Cloud project (p1) that will contain some BigQuery resources. Prior to this, we have manually created a Google Cloud Project (ops) which hosts the service account that will be used by Terraform and granted the necessary permissions to create projects in a specific location.


Diagram depicting a Terraform configuration for Google Cloud Platform (GCP). It shows a multi-provider setup with one main 'Google Provider (ops)' connecting to three separate service account providers labeled 'SA p1', 'SA p2', and 'SA p3', each linked to their own GCP resources within an 'ops' project environment

In this ops project, we don’t have the BigQuery API service enabled because the ops project is not related to any BigQuery resources.

Without alias

In the Terraform code without alias, we create the resources for the new project, the BigQuery API and a bigQuery dataset. Running the Terraform code will fail because we configure the provider with the service account from the ops project. Even with the specific access rights, the project hosting the service account used by the provider to create the BigQuery resources doesn’t have access to the BigQuery API.


A Terraform workflow diagram with a failure point. The 'Google Provider (ops)' connects to a Google Cloud Platform setup with an 'SA ops' service account. Two BigQuery resources are shown, with a red line indicating a failure in the connection from 'SA ops' to the first BigQuery resource, while the connection to the BigQuery API is intact.

│ Error: Error creating Dataset: googleapi: Error 403: BigQuery API has not been used in project 934740185609 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/bigquery.googleapis.com/overview?project=934740185609 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

What can we do?

Enabling the Google BigQuery API in the ops project will fix the problem, but what if you have many resources to create, requiring many APIs ? Suddenly, you will enable many APIs in a single project so that your Terraform code can manage all different types of resources in many projects.

Using an alias

Instead, a better approach is to use an alias to clearly define the scope of each service account. In this case, we first have to enable the IAM API in the ops project manually to be able to create IAM resources.

Then, we add a definition for a new service account in the target project (p1) and grant it ownership of the project. We then use the credentials of that service account to create an alias for the provider. Finally, we use the alias to create the resources within the project.


Flowchart diagram showing the structure of service accounts and resource management within Google Cloud Platform, including operations and provider-specific access to BigQuery API


With this approach, the scope of each Terraform provider is clearly defined.

  • Each projects has it own set of credentials

  • Resources can only be created within the authorized project

  • Specific APIs are only enabled within the corresponding projects

Apply complete! Resources: 7 added, 0 changed, 0 destroyed.

Conclusion

We have seen how using an alias to use different credentials within a Terraform root module can limit the risks of compromising your infrastructure. It helps you to reduce coupling between your resources by clearly defining the scope of your credentials.

If you are looking for support on Data Stack or Google Cloud solutions, feel free to reach out to us at sales@astrafy.io.