diff --git a/modules/spacelift-automation/README.md b/modules/spacelift-automation/README.md index cb0f939..cb68be0 100644 --- a/modules/spacelift-automation/README.md +++ b/modules/spacelift-automation/README.md @@ -2,7 +2,7 @@ [![Release](https://img.shields.io/github/release/masterpointio/terraform-spacelift-automation.svg)](https://github.com/masterpointio/terraform-spacelift-automation/releases/latest) -This Terraform root module provides infrastructure automation for projects in [Spacelift](https://docs.spacelift.io/). +This Terraform child module provides infrastructure automation for projects in [Spacelift](https://docs.spacelift.io/). ## Overview @@ -17,14 +17,12 @@ It automates the creation of "child" stacks and all the required accompanying Sp 3. [Spacelift AWS Integration Attachment](https://docs.spacelift.io/integrations/cloud-providers/aws#lets-explain) Associates a specific AWS IAM role with a stack to allow it to assume that role. The IAM role typically has permissions to manage specific AWS resources, and Spacelift assumes this role to run the operations required by the stack. 4. [Spacelift Initialization Hook](https://docs.spacelift.io/concepts/run#initializing) - Prepares your environment before executing infrastructure code. This custom script copies corresponding Terraform tfvars files into a working directory before either run or task, so it's automatically loaded into the OpenTofu/Terraform execution environment. - -These files are automatically mounted and available during the Terraform plan and apply stages. + Prepares your environment before executing infrastructure code. This custom script copies corresponding Terraform tfvars files into a working directory before either run or task as a `spacelift.auto.tfvars` file. It's [automatically loaded](https://opentofu.org/docs/v1.7/language/values/variables/#variable-definitions-tfvars-files) into the OpenTofu/Terraform execution environment. ## Usage Spacelift Automation logic is opinionated and heavily relies on the Git repository structure. -The root module `spacelift-automation` is configured to track all the files in the provided root module directory and create the stack based on the provided configuration (if any). +This module is configured to track all the files in the provided root module directory and create the stack based on the provided configuration (if any). Structure requirements are: @@ -48,6 +46,17 @@ Input repo structure: │ │ │ └── stage.tfvars │ │ ├── variables.tf │ │ └── versions.tf +│ ├── k8s-cluster +│ │ ├── stacks +│ │ │ └── dev.yaml +│ │ │ └── prod.yaml +│ │ │ └── common.yaml +│ │ ├── tfvars +│ │ │ └── dev.tfvars +│ │ │ └── prod.tfvars +│ │ ├── variables.tf +│ │ ├── main.tf +│ │ └── versions.tf ... ``` @@ -74,24 +83,31 @@ The configuration above creates the following stacks: These stacks have the following configuration: -- Stacks track changes in GitHub repo `github.com/masterpointio/terraform-spacelift-automation`, branch `main` (default one), and directrory `root-modules`. +- Stacks track changes in GitHub repo `github.com/masterpointio/terraform-spacelift-automation`, branch `main` (the default), and directrory `root-modules`. - Common configuration is defined in `root-modules/spacelift-aws-role/stacks/common.yaml` and applied to both Stacks. However, if there is an override in a Stack config (e.g. `root-modules/spacelift-aws-role/stacks/dev.yaml`), it takes precedence over common configs. -- Corresponding Terraform variables are generated by [Initialization Hook](https://docs.spacelift.io/concepts/run#initializing) and placed in the root of each Stack's working directory during each run or task. For example, the content of the file `root-modules/spacelift-aws-role/tfvars/dev.tfvars` will be copied to working directory of the Stack `spacelift-aws-role-dev` as file `spacelift.auto.tfvars` allowing the OpenTofu/Terraform inputs to be automatically loaded. +- Corresponding Terraform variables are generated by an [Initialization Hook](https://docs.spacelift.io/concepts/run#initializing) and placed in the root of each Stack's working directory during each run or task. For example, the content of the file `root-modules/spacelift-aws-role/tfvars/dev.tfvars` will be copied to working directory of the Stack `spacelift-aws-role-dev` as file `spacelift.auto.tfvars` allowing the OpenTofu/Terraform inputs to be automatically loaded. + +## FAQs + +### Why are variable values provided separately in `tfvars/` and not in the `yaml` file? + +This is to support easy local and outside-spacelift operations. Keeping variable values in a `tfvars` file per workspace allows you to simply pass that file to the relevant CLI command locally via the `-var-file` option so that you don't need to provide values individually. -## Can I create a Stack for Spacelift Automation? +### Can I create a Spacelift Stack for Spacelift Automation? (Recommended) -Spacelift Automation can manage itself as a Stack as well. Follow the next steps to achieve that: +Spacelift Automation can manage itself as a Stack as well and we recommend this so you can fully automate your Stack management on merge to your given branch. Follow these next steps to achieve that: -1. Create a vanilla OpenTofu/Terraform configuration for this module. In other words, a configuration that uses the default capabilities of either OpenTofu or Terraform without any customization, complex logic, or third-party modules or plugins. -1. Create Terraform workspace that will be used for your Automation configuration, e.g.: +1. Create a new vanilla OpenTofu/Terraform root module that consumes this child module and supplies the necessary configuration for your unique setup. In other words, it's a configuration that uses the default capabilities of either OpenTofu or Terraform without any customization, or third-party tools or plugins. +2. Optionally, Create Terraform workspace that will be used for your Automation configuration, e.g.: ```sh tofu workspace new masterpoint ``` -1. Apply the vanilla OpenTofu/Terraform configuration. -1. Move the Automation configs to the `/spacelift-automation/stacks` directory and push the changes to the tracked repo and branch. -1. From this moment Spacelift Automation is tracking the changes for its Stack configs and Terraform variables. + Remember, that Stack config and tfvars file name must be equal to the workspace, which can be `default`. +3. Apply the vanilla OpenTofu/Terraform configuration. +4. Move the Automation configs to the `/spacelift-automation/stacks` directory and push the changes to the tracked repo and branch. +5. From this moment Spacelift Automation is tracking the changes for its Stack configs and Terraform variables. -NOTE to Masterpoint team: We might want to create a small wrapper to automatize this using Taskit. +NOTE to Masterpoint team: We might want to create a small wrapper to automatize this using Taskit. On hold for now. @@ -106,7 +122,7 @@ NOTE to Masterpoint team: We might want to create a small wrapper to automatize | Name | Version | | ------------------------------------------------------------------ | ------- | -| [spacelift](#provider_spacelift) | >= 1.14 | +| [spacelift](#provider_spacelift) | 1.16.1 | ## Modules @@ -117,12 +133,12 @@ NOTE to Masterpoint team: We might want to create a small wrapper to automatize ## Resources -| Name | Type | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------- | -| [spacelift_aws_integration_attachment.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/aws_integration_attachment) | resource | -| [spacelift_drift_detection.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/drift_detection) | resource | -| [spacelift_stack.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack) | resource | -| [spacelift_stack_destructor.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack_destructor) | resource | +| Name | Type | +| --------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | +| [spacelift_aws_integration_attachment.default](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/aws_integration_attachment) | resource | +| [spacelift_drift_detection.default](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/drift_detection) | resource | +| [spacelift_stack.default](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack) | resource | +| [spacelift_stack_destructor.default](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack_destructor) | resource | ## Inputs @@ -180,8 +196,8 @@ NOTE to Masterpoint team: We might want to create a small wrapper to automatize | [tags](#input_tags) | Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`).
Neither the tag keys nor the tag values will be modified by this module. | `map(string)` | `{}` | no | | [tenant](#input_tenant) | ID element \_(Rarely used, not included by default)\_. A customer identifier, indicating who this instance of a resource is for | `string` | `null` | no | | [terraform_smart_sanitization](#input_terraform_smart_sanitization) | Indicates whether runs on this will use terraform's sensitive value system to sanitize
the outputs of Terraform state and plans in spacelift instead of sanitizing all fields. | `bool` | `false` | no | -| [terraform_version](#input_terraform_version) | Terraform version to use. | `string` | `"1.7.2"` | no | -| [terraform_workflow_tool](#input_terraform_workflow_tool) | Defines the tool that will be used to execute the workflow.
This can be one of OPEN_TOFU, TERRAFORM_FOSS or CUSTOM. Defaults to TERRAFORM_FOSS. | `string` | `"OPEN_TOFU"` | no | +| [terraform_workflow_tool](#input_terraform_workflow_tool) | Defines the tool that will be used to execute the workflow.
This can be one of OPEN_TOFU, TERRAFORM_FOSS or CUSTOM. | `string` | `"OPEN_TOFU"` | no | +| [tf_version](#input_tf_version) | Terraform version to use. | `string` | `"1.7.2"` | no | | [worker_pool_id](#input_worker_pool_id) | ID of the worker pool to use.
NOTE: worker_pool_id is required when using a self-hosted instance of Spacelift. | `string` | `null` | no | ## Outputs diff --git a/modules/spacelift-automation/main.tf b/modules/spacelift-automation/main.tf index 542ad26..a197efb 100644 --- a/modules/spacelift-automation/main.tf +++ b/modules/spacelift-automation/main.tf @@ -6,11 +6,11 @@ # # It handles the following: # -# 1. Stack Configurations from Git (see ## Stack Configurations from Git) +# 1. Stack Configurationst (see ## Stack Configurations) # Reads the Spacelift stack configurations strictly based on the root modules structure in Git and file names. # These are the configurations required to be set for a stack, e.g. project_root, terraform_workspace, root_module. # -# 2. Common Stack configurations +# 2. Common Stack configurations (see ## Common Stack configurations) # Some configurations are equal across the whole root module, and can be set it on a root module level: # * Space IDs: in the majority of cases all the workspaces in a root module belong to the same Spacelift space, so # we allow setting a "global" space_id for all stacks on a root module level. @@ -28,6 +28,28 @@ locals { enabled = module.this.enabled # Read and decode Stack YAML files from the root directory + # Example: + # { + # "random-pet" = { + # "common.yaml" = { + # "stack_settings" = { + # "description" = "This stack generates random pet names" + # "manage_state" = true + # } + # "tfvars" = { + # "enabled" = false + # } + # } + # "example.yaml" = { + # "stack_settings" = { + # "manage_state" = true + # } + # "tfvars" = { + # "enabled" = true + # } + # } + # } + # } root_module_yaml_decoded = { for module in var.enabled_root_modules : module => { for yaml_file in fileset("${path.root}/${var.root_modules_path}/${module}/stacks", "*.yaml") : @@ -35,12 +57,40 @@ locals { } } + ## Common Stack configurations # Retrieve common Stack configurations for each root module + # Example: + # { + # "random-pet" = { + # "stack_settings" = { + # "description" = "This stack generates random pet names" + # "manage_state" = true + # } + # "tfvars" = { + # "enabled" = false + # } + # } + # } common_configs = { - for module, file in local.root_module_yaml_decoded : module => lookup(file, var.common_config_file, {}) + for module, files in local.root_module_yaml_decoded : module => lookup(files, var.common_config_file, {}) } - # Merge all Stack configurations from the root modules into a single map + ## Stack Configurations + # Merge all Stack configurations from the root modules into a single map, and filter out the common config. + # Example: + # { + # "random-pet-example" = { + # "project_root" = "examples/complete/components/random-pet" + # "root_module" = "random-pet" + # "stack_settings" = { + # "manage_state" = true + # } + # "terraform_workspace" = "example" + # "tfvars" = { + # "enabled" = true + # } + # } + # } root_module_stack_configs = merge([for module, files in local.root_module_yaml_decoded : { for file, content in files : "${module}-${trimsuffix(file, ".yaml")}" => merge( @@ -55,11 +105,31 @@ locals { ]...) # Get the configs for each stack, merged with the common configurations + # Example: + # { + # "random-pet-example" = { + # "project_root" = "examples/complete/components/random-pet" + # "root_module" = "random-pet" + # "stack_settings" = { + # "manage_state" = true + # } + # "terraform_workspace" = "example" + # "tfvars" = { + # "enabled" = false + # } + # } + # } configs = { for key, value in module.deep : key => value.merged } # Get the Stacks configs, this is just to improve code readability + # Example: + # { + # "random-pet-example" = { + # "manage_state" = true + # } + # } stack_configs = { for key, value in local.configs : key => value.stack_settings } @@ -159,7 +229,7 @@ module "deep" { maps = [local.common_configs[each.value.root_module], each.value] } -resource "spacelift_stack" "this" { +resource "spacelift_stack" "default" { for_each = local.enabled ? local.stacks : toset([]) space_id = coalesce(try(local.stack_configs[each.key].space_id, null), var.space_id) @@ -185,7 +255,7 @@ resource "spacelift_stack" "this" { labels = local.labels[each.key] enable_local_preview = try(local.stack_configs[each.key].enable_local_preview, var.enable_local_preview) terraform_smart_sanitization = try(local.stack_configs[each.key].terraform_smart_sanitization, var.terraform_smart_sanitization) - terraform_version = try(local.stack_configs[each.key].terraform_version, var.terraform_version) + tf_version = try(local.stack_configs[each.key].tf_version, var.tf_version) terraform_workflow_tool = var.terraform_workflow_tool terraform_workspace = local.configs[each.key].terraform_workspace @@ -206,35 +276,42 @@ resource "spacelift_stack" "this" { # as it will delete all resources in the stack when toggled from 'true' to 'false'. # Use the 'deactivated' attribute to disable the stack destructor functionality instead. # https://github.com/spacelift-io/terraform-provider-spacelift/blob/master/spacelift/resource_stack_destructor.go -resource "spacelift_stack_destructor" "this" { +resource "spacelift_stack_destructor" "default" { for_each = local.enabled ? local.stacks : toset([]) - stack_id = spacelift_stack.this[each.key].id + stack_id = spacelift_stack.default[each.key].id deactivated = !try(local.stack_configs[each.key].destructor_enabled, var.destructor_enabled) depends_on = [ - spacelift_drift_detection.this, - spacelift_aws_integration_attachment.this + spacelift_drift_detection.default, + spacelift_aws_integration_attachment.default ] } -resource "spacelift_aws_integration_attachment" "this" { +resource "spacelift_aws_integration_attachment" "default" { for_each = local.enabled ? local.stacks : toset([]) integration_id = try(local.stack_configs[each.key].aws_integration_id, var.aws_integration_id) - stack_id = spacelift_stack.this[each.key].id + stack_id = spacelift_stack.default[each.key].id read = var.aws_integration_attachment_read write = var.aws_integration_attachment_write } -resource "spacelift_drift_detection" "this" { +resource "spacelift_drift_detection" "default" { for_each = local.enabled ? { for key, value in local.stack_configs : key => value if try(local.stack_configs[key].drift_detection_enabled, var.drift_detection_enabled) } : {} - stack_id = spacelift_stack.this[each.key].id + stack_id = spacelift_stack.default[each.key].id ignore_state = try(local.stack_configs[each.key].drift_detection_ignore_state, var.drift_detection_ignore_state) reconcile = try(local.stack_configs[each.key].drift_detection_reconcile, var.drift_detection_reconcile) schedule = try(local.stack_configs[each.key].drift_detection_schedule, var.drift_detection_schedule) timezone = try(local.stack_configs[each.key].drift_detection_timezone, var.drift_detection_timezone) + + lifecycle { + precondition { + condition = can(regex("^([0-9,\\-\\*]+\\s+){4}[0-9,\\-\\*]+$", try(local.stack_configs[each.key].drift_detection_schedule, var.drift_detection_schedule))) + error_message = "Invalid cron schedule format for drift detection" + } + } } diff --git a/modules/spacelift-automation/providers.tf b/modules/spacelift-automation/providers.tf deleted file mode 100644 index 2809331..0000000 --- a/modules/spacelift-automation/providers.tf +++ /dev/null @@ -1,10 +0,0 @@ -# Earlier versions of OpenTofu used empty provider blocks ("proxy provider configurations") -# for child modules to declare their need to be passed a provider configuration by their -# callers. That approach was ambiguous and is now deprecated. -# -# If you control this module, you can migrate to the new declaration syntax by removing all -# of the empty provider "spacelift" blocks and then adding or updating an entry like the -# following to the required_providers block of module.automation: -# spacelift = { -# source = "spacelift-io/spacelift" -# } diff --git a/modules/spacelift-automation/variables.tf b/modules/spacelift-automation/variables.tf index d02e8da..3c8ab8b 100644 --- a/modules/spacelift-automation/variables.tf +++ b/modules/spacelift-automation/variables.tf @@ -226,7 +226,7 @@ variable "terraform_smart_sanitization" { default = false } -variable "terraform_version" { +variable "tf_version" { type = string description = "Terraform version to use." default = "1.7.2"