Create the Terraform Repository

On this page, you will:

Clone your infrastructure repository
Set up the Terraform directory structure
Configure the remote state backend
Set up pre-commit hooks
Configure .gitignore for Terraform
Initialise Terraform and verify the setup

Clone Your Infrastructure Repository

Navigate to your projects directory and clone the data-stack-infrastructure repository you created in the GitHub setup guide.

cd ~/projects/data  # or wherever you store your projects
git clone git@github.com:YOUR-ORG/data-stack-infrastructure.git
cd data-stack-infrastructure

Replace YOUR-ORG with your GitHub organisation name.

Verify you're in the correct repository:

git remote -v

Expected output:

origin  git@github.com:YOUR-ORG/data-stack-infrastructure.git (fetch)
origin  git@github.com:YOUR-ORG/data-stack-infrastructure.git (push)

As we are doing a new piece of work, let's create a new branch:

git checkout -b feature/terraform-setup

Set Up `.envrc` for Environment Variables

We'll use direnv to automatically load environment variables when you enter the project directory. This is much better than manually exporting variables every time. You should have already installed it during Local Development Environment.

Add `.envrc` to `.gitignore`

direnv is a tool that runs script commands whenever you cd into a given directory. Those commands are stored in a .envrc file.

The .envrc file is already excluded by your global .gitignore (you set this up in Local Development Environment). However, if someone else hasn't added it to their own, they could accidentally end up committing it to git. Therefore, it's good practice to also include it in your project .gitignore - you can never be too careful.

Add the following line to the project .gitignore file (if not already present):

# Secrets
.envrc

Create .envrc File

In your repository root, create a .envrc file:

touch ~/projects/data/data-stack-infrastructure/.envrc

As we are using AWS S3 for the backend for all resources, we need to make sure that we are using the correct AWS profile from those we setup in the AWS account setup. Add this to your .envrc file:

# AWS Profile for Terraform Backend
export AWS_PROFILE="data-engineer"

It's also useful to include a template for other developers using the repo to use for reference. This will be held under version control, so should only ever contain placeholders.

touch ~/projects/data/data-stack-infrastructure/.envrc.example

Add the templated profile above - it's useful to include details for other developers about the purpose of the variable:

# AWS Profile for Terraform Backend
# Update this with the profile to use with AWS
# or choose "default" if you don't have profiles configured
export AWS_PROFILE="data-engineer"

Using aws-vault

If you configured aws-vault in AWS Account Setup, the AWS_PROFILE environment variable won't work the same way. Instead, you'll need to run Terraform commands within an aws-vault session:

aws-vault exec data-engineer -- terraform init

Or start a shell session with aws-vault exec data-engineer and run commands from there.

Allow direnv

direnv allow .

Now, whenever you cd into this directory, direnv will automatically load your environment variables. When you leave, it unloads them.

Create the Directory Structure

Terraform projects benefit from a clear, consistent structure. With multiple providers (GitHub, AWS, Snowflake), we'll organise code by provider whilst keeping shared configuration at the root level.

Recommended Structure

Create the following directory structure:

cd ~/projects/data/data-stack-infrastructure
mkdir -p terraform/{github,aws,snowflake,modules}

Add .gitkeep files into the folders to ensure that the directory structure is committed to GitHub:

touch github/.gitkeep
touch aws/.gitkeep
touch snowflake/.gitkeep
touch modules/.gitkeep

Your repository should now look like:

data-stack-infrastructure/
├── .envrc
├── .github/
│   └── CODEOWNERS
├── terraform/
│   ├── github/           # GitHub provider resources
|   │   └── .gitkeep
│   ├── aws/              # AWS provider resources
|   │   └── .gitkeep
│   ├── snowflake/        # Snowflake provider resources
|   │   └── .gitkeep
│   └── modules/          # Reusable Terraform modules (future)
|   │   └── .gitkeep
├── .gitignore
├── LICENSE
└── README.md

Why Split by Provider?

This structure provides several benefits:

Clarity: Easy to find resources by provider
Independence: Each provider has its own backend.tf with a unique state key
Team workflow: Different teams can own different providers
Blast radius: Changes to GitHub resources won't affect AWS or Snowflake

All providers use the same S3 bucket and DynamoDB table, but each provider directory has its own backend.tf file with a provider-specific state key.

Provider-Specific State Keys

Each provider directory will use a different state file key in S3:

github/terraform.tfstate - GitHub resources
aws/terraform.tfstate - AWS resources
snowflake/terraform.tfstate - Snowflake resources

This isolates changes and reduces the risk of conflicts.

Update .gitignore for Terraform-Specific Files

When you created the repository in the GitHub setup guide, you selected the Terraform .gitignore template. This gives you basic Terraform exclusions at the repository root.

However, we need to add a few custom rules to the .gitignore to handle our specific setup:

cd ..  # Back to repository root if you're still in terraform/

Add these lines to the existing .gitignore:

# Exclude all tfvars files which may contain sensitive data
*.tfvars
*.tfvars.json

# BUT include terraform.tfvars since it only has non-sensitive config
!terraform.tfvars

# Ignore lock files for now (we'll commit them later when stable)
.terraform.lock.hcl

Why Ignore .terraform.lock.hcl Initially

We're ignoring the lock file initially to avoid conflicts whilst the team is setting up. Once your provider versions are stable, you'll commit the lock file to ensure everyone uses the same provider versions.

GitHub's Terraform Template

GitHub's Terraform .gitignore template already covers the basics like .terraform/ directories, *.tfstate files, crash logs, and override files. We're just adding our specific rules for tfvars files and the lock file.

Set Up Pre-commit Hooks

Create .pre-commit-config.yaml in the repository root (not in terraform/ directory):

cd ..  # Back to repository root

Create .pre-commit-config.yaml:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-added-large-files
      - id: check-merge-conflict

  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.88.0
    hooks:
      - id: terraform_fmt
      - id: terraform_validate
      - id: terraform_docs
        args:
          - --hook-config=--path-to-file=README.md
          - --hook-config=--add-to-existing-file=true
          - --hook-config=--create-file-if-not-exists=true
      - id: terraform_tflint
        args:
          - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl

  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.88.0
    hooks:
      - id: terraform_checkov
        args:
          - --args=--config-file=__GIT_WORKING_DIR__/.checkov.yaml

default_stages: [push]

This configuration: - Runs basic checks (whitespace, YAML syntax, large files) - Formats Terraform code automatically - Validates Terraform syntax - Generates documentation - Runs security checks with Checkov - Runs linting with TFLint - Only runs on push (not every commit)

Create Pre-commit Configuration Files

TFLint Configuration

Create .tflint.hcl in the repository root:

config {
  module = true
  force   = false
}

plugin "terraform" {
  enabled = true
  preset  = "recommended"
}

plugin "aws" {
  enabled = true
  version = "0.30.0"
  source  = "github.com/terraform-linters/tflint-ruleset-aws"
}

Checkov Configuration

Create .checkov.yaml in the repository root:

framework:
  - terraform

download-external-modules: true

soft-fail: false

quiet: false

compact: true

skip-check:
  # Add any checks you want to skip here
  # Example: CKV_AWS_18 - S3 bucket logging

terraform-docs Configuration

Create .terraform-docs.yml in the repository root to control how module documentation is generated:

formatter: markdown table

output:
  file: README.md
  mode: inject
  template: |-
    <!-- BEGIN_TF_DOCS -->
    {{ .Content }}
    <!-- END_TF_DOCS -->

sort:
  enabled: true
  by: required

settings:
  anchor: true
  color: true
  default: true
  description: true
  escape: true
  hide-empty: false
  indent: 2
  lockfile: true
  read-comments: true
  required: true
  sensitive: true
  type: true

This configuration:

Uses inject mode so terraform-docs inserts between  and  markers, preserving any hand-written content in module READMEs
Sorts variables by required so required inputs appear first
Generates a markdown table format for clean, readable output

EditorConfig

Create .editorconfig in the repository root to ensure consistent formatting across editors:

root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.{tf,tfvars}]
indent_style = space
indent_size = 2

[*.{md,markdown}]
trim_trailing_whitespace = false

[*.{yml,yaml}]
indent_style = space
indent_size = 2

[*.py]
indent_style = space
indent_size = 4

Install Pre-commit Hooks

Install the pre-commit hooks for this repository:

pre-commit install --hook-type pre-push

Expected output:

pre-commit installed at .git/hooks/pre-push

This installs the hooks to run before you push code to the remote repository.

Test Pre-commit

You can test pre-commit hooks manually at any time:

pre-commit run --all-files

Validate the Configuration

Run Terraform's built-in validation:

terraform validate

Expected output:

Success! The configuration is valid.

This checks: - Syntax is correct - Required variables are defined

Format the Code

Format all Terraform files to the standard style:

terraform fmt -recursive

This ensures consistent formatting across all .tf files.

Commit Your Work

Now commit the initial Terraform setup.

cd ../..  # Back to repository root
git checkout -b feature/terraform-setup
git add .
git commit -m "Initial Terraform setup

- Create provider-specific directory structure
- Add pre-commit hooks for code quality
- Create TFLint and Checkov configurations"

Finally push to GitHub, and create the PR.

git push -u origin feature/terraform-setup

Pre-commit Will Run

When you push, pre-commit hooks will automatically:

Format your Terraform code
Validate syntax
Generate documentation
Run security checks

If any checks fail, fix the issues and push again.

Verify Your Setup

Your terraform/ directory structure:

tree terraform/ -L 2 -a

Expected structure:

terraform/
├── github/
│   └── .gitkeep
├── aws/
│   └── .gitkeep
├── snowflake/
│   └── .gitkeep
└── modules/
    └── .gitkeep

Repository root should contain:

.checkov.yaml             # Checkov configuration
.editorconfig             # Editor formatting rules
.github/                  # GitHub configuration
.gitignore                # Git ignore rules
.pre-commit-config.yaml   # Pre-commit hooks
.terraform-docs.yml       # terraform-docs configuration
.tflint.hcl               # TFLint configuration
LICENSE                   # Repository license
README.md                 # Repository README
terraform/                # Terraform configuration

Troubleshooting

Pre-commit Hook Failures

If pre-commit hooks fail:

# See what failed
git push

# Run pre-commit manually to see details
pre-commit run --all-files

# Fix issues, then try again
git add .
git commit --amend --no-edit
git push

What's Next

You now have a complete Terraform repository structure:

✅ Provider-specific directory structure (github/, aws/, snowflake/)
✅ Pre-commit hooks configured
✅ Code quality checks in place (TFLint, Checkov)
✅ Documentation structure ready

Next, you'll add your first resources by configuring the github provider and importing the GitHub organisation settings and teams you created manually into Terraform.

Continue to Add GitHub to Terraform →

Create the Terraform Repository

Clone Your Infrastructure Repository

Set Up .envrc for Environment Variables

Add .envrc to .gitignore

Create .envrc File

Allow direnv

Create the Directory Structure

Recommended Structure

Why Split by Provider?

Update .gitignore for Terraform-Specific Files

Set Up Pre-commit Hooks

Create Pre-commit Configuration Files

TFLint Configuration

Checkov Configuration

terraform-docs Configuration

EditorConfig

Install Pre-commit Hooks

Validate the Configuration

Format the Code

Commit Your Work

Verify Your Setup

Troubleshooting

Pre-commit Hook Failures

What's Next

Set Up `.envrc` for Environment Variables

Add `.envrc` to `.gitignore`