Cache location is not valid when cache_bucket_suffix_enabled = true #91
Labels
bug
🐛 An issue with the system
Comments
|
Did some troubleshooting and was able to narrow it down to troubleshootingthis works: cache_options = {
"S3" = {
type = "S3"
location = this.module.id
},
...but this does not... cache_bucket_name = "${module.this.id}${var.cache_bucket_suffix_enabled ? "-${join("", random_string.bucket_prefix.*.result)}" : ""}"
cache_options = {
"S3" = {
type = "S3"
location = local.cache_bucket_name
},
...this also works... moo = "moo"
cache_options = {
"S3" = {
type = "S3"
location = local.moo
},
...this does not... cache_bucket_name = "${module.this.id}-${join("", random_string.bucket_prefix.*.result)}"
cache_options = {
"S3" = {
type = "S3"
location = local.cache_bucket_name
},
...this does... cache_bucket_name = module.this.id
cache_options = {
"S3" = {
type = "S3"
location = local.cache_bucket_name
},
...this does... cache_bucket_name = "${module.this.id}-moo"
cache_options = {
"S3" = {
type = "S3"
location = local.cache_bucket_name
},
...this does not... cache_bucket_name = join("", random_string.bucket_prefix.*.result)
cache_options = {
"S3" = {
type = "S3"
location = local.cache_bucket_name
},
...for whatever reason, calling that random_string resource causes this failure. just to be sure i used random_id with no count and got the same result: cache_bucket_name = "${module.this.id}-${random_id.bucket_prefix.hex}"
cache_options = {
"S3" = {
type = "S3"
location = local.cache_bucket_name
},
...updating the normalization logic to exclude the suffix, instead adding that to the cache_bucket resource fails as well: cache_bucket_name_normalised = substr(
join("-", split("_", lower(module.this.id))),
0,
min(length(module.this.id), var.cache_bucket_suffix_enabled ? 50 : 63),
)
...
resource "aws_s3_bucket" "cache_bucket" {
count = module.this.enabled && local.s3_cache_enabled ? 1 : 0
bucket = var.cache_bucket_suffix_enabled ? "${local.cache_bucket_name_normalised}-${random_id.bucket_prefix.hex}" : local.cache_bucket_name_normalised
...Using the I even tried changing resource "random_string" "bucket_prefix" {
length = 12
number = false
upper = false
special = false
lower = true
}
module "codebuild" {
source = "../../"
cache_bucket_suffix_enabled = random_string.bucket_prefix.result
environment_variables = var.environment_variables
cache_expiration_days = var.cache_expiration_days
cache_type = var.cache_type
context = module.this.context
}
}and same as before, changing it to a plain string worked fine... module "codebuild" {
source = "../../"
cache_bucket_suffix_enabled = "moo"
environment_variables = var.environment_variables
cache_expiration_days = var.cache_expiration_days
cache_type = var.cache_type
context = module.this.context
}The original usage is entirely in-line with the Terraform docs example, so I don't know why this would be happening. # The following example shows how to generate a unique name for an AWS EC2
# instance that changes each time a new AMI id is selected.
resource "random_id" "server" {
keepers = {
# Generate a new id each time we switch to a new AMI id
ami_id = "${var.ami_id}"
}
byte_length = 8
}
resource "aws_instance" "server" {
tags = {
Name = "web-server ${random_id.server.hex}"
}
# Read the AMI id "through" the random_id resource to ensure that
# both will change together.
ami = random_id.server.keepers.ami_id
# ... (other aws_instance arguments) ...
}In the debug logs I do see a warning about non-computed attributes but bug reports mentioning that say it's expected and ok. However the bit at the end caught my eye. There is a bug report open for the same error when using random. User found that setting the length to 15 fixed the issue. Updating TL;DR: WTF? |
|
In PR 92, the output of terratest shows the correct suffix attached to the bucket. https://github.com/cloudposse/actions/runs/3220447314?check_suite_focus=true Could you try doing a terraform plan using the This is the plan I see terraform plan # module.codebuild.data.aws_iam_policy_document.permissions_cache_bucket[0] will be read during apply
# (config refers to values not yet known)
<= data "aws_iam_policy_document" "permissions_cache_bucket" {
+ id = (known after apply)
+ json = (known after apply)
+ statement {
+ actions = [
+ "s3:*",
]
+ effect = "Allow"
+ resources = [
+ (known after apply),
+ (known after apply),
]
}
}
# module.codebuild.aws_codebuild_project.default[0] will be created
+ resource "aws_codebuild_project" "default" {
+ arn = (known after apply)
+ badge_enabled = false
+ badge_url = (known after apply)
+ build_timeout = 60
+ description = (known after apply)
+ encryption_key = (known after apply)
+ id = (known after apply)
+ name = "eg-test-codebuild-test"
+ queued_timeout = 480
+ service_role = (known after apply)
+ tags = {
+ "Name" = "eg-test-codebuild-test"
+ "Namespace" = "eg"
+ "Stage" = "test"
}
+ tags_all = {
+ "Name" = "eg-test-codebuild-test"
+ "Namespace" = "eg"
+ "Stage" = "test"
}
+ artifacts {
+ encryption_disabled = false
+ override_artifact_name = false
+ type = "CODEPIPELINE"
}
+ cache {
+ location = (known after apply)
+ modes = (known after apply)
+ type = (known after apply)
}
+ environment {
+ compute_type = "BUILD_GENERAL1_SMALL"
+ image = "aws/codebuild/standard:2.0"
+ image_pull_credentials_type = "CODEBUILD"
+ privileged_mode = false
+ type = "LINUX_CONTAINER"
+ environment_variable {
+ name = "AWS_REGION"
+ type = "PLAINTEXT"
+ value = "us-east-2"
}
+ environment_variable {
+ name = "AWS_ACCOUNT_ID"
+ type = "PLAINTEXT"
+ value = "781590699830"
}
+ environment_variable {
+ name = "IMAGE_REPO_NAME"
+ type = "PLAINTEXT"
+ value = "UNSET"
}
+ environment_variable {
+ name = "IMAGE_TAG"
+ type = "PLAINTEXT"
+ value = "latest"
}
+ environment_variable {
+ name = "STAGE"
+ type = "PLAINTEXT"
+ value = "test"
}
+ environment_variable {
+ name = "APP_URL"
+ type = "PLAINTEXT"
+ value = "https://app.example.com"
}
+ environment_variable {
+ name = "COMPANY_NAME"
+ type = "PLAINTEXT"
+ value = "Cloud Posse"
}
+ environment_variable {
+ name = "TIME_ZONE"
+ type = "PLAINTEXT"
+ value = "America/Los_Angeles"
}
}
+ source {
+ report_build_status = false
+ type = "CODEPIPELINE"
}
}
# module.codebuild.aws_iam_policy.default[0] will be created
+ resource "aws_iam_policy" "default" {
+ arn = (known after apply)
+ id = (known after apply)
+ name = "eg-test-codebuild-test"
+ path = "/service-role/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "ssm:GetParameters",
+ "secretsmanager:GetSecretValue",
+ "logs:PutLogEvents",
+ "logs:CreateLogStream",
+ "logs:CreateLogGroup",
+ "iam:PassRole",
+ "ecs:RunTask",
+ "ecr:UploadLayerPart",
+ "ecr:PutImage",
+ "ecr:InitiateLayerUpload",
+ "ecr:GetAuthorizationToken",
+ "ecr:CompleteLayerUpload",
+ "ecr:BatchCheckLayerAvailability",
+ "codecommit:GitPull",
]
+ Effect = "Allow"
+ Resource = "*"
+ Sid = ""
},
]
+ Version = "2012-10-17"
}
)
+ policy_id = (known after apply)
+ tags_all = (known after apply)
}
# module.codebuild.aws_iam_policy.default_cache_bucket[0] will be created
+ resource "aws_iam_policy" "default_cache_bucket" {
+ arn = (known after apply)
+ id = (known after apply)
+ name = "eg-test-codebuild-test-cache-bucket"
+ path = "/service-role/"
+ policy = (known after apply)
+ policy_id = (known after apply)
+ tags_all = (known after apply)
}
# module.codebuild.aws_iam_role.default[0] will be created
+ resource "aws_iam_role" "default" {
+ arn = (known after apply)
+ assume_role_policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "sts:AssumeRole"
+ Effect = "Allow"
+ Principal = {
+ Service = "codebuild.amazonaws.com"
}
+ Sid = ""
},
]
+ Version = "2012-10-17"
}
)
+ create_date = (known after apply)
+ force_detach_policies = true
+ id = (known after apply)
+ managed_policy_arns = (known after apply)
+ max_session_duration = 3600
+ name = "eg-test-codebuild-test"
+ path = "/"
+ tags = {
+ "Name" = "eg-test-codebuild-test"
+ "Namespace" = "eg"
+ "Stage" = "test"
}
+ tags_all = {
+ "Name" = "eg-test-codebuild-test"
+ "Namespace" = "eg"
+ "Stage" = "test"
}
+ unique_id = (known after apply)
+ inline_policy {
+ name = (known after apply)
+ policy = (known after apply)
}
}
# module.codebuild.aws_iam_role_policy_attachment.default[0] will be created
+ resource "aws_iam_role_policy_attachment" "default" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = (known after apply)
}
# module.codebuild.aws_iam_role_policy_attachment.default_cache_bucket[0] will be created
+ resource "aws_iam_role_policy_attachment" "default_cache_bucket" {
+ id = (known after apply)
+ policy_arn = (known after apply)
+ role = (known after apply)
}
# module.codebuild.aws_s3_bucket.cache_bucket[0] will be created
+ resource "aws_s3_bucket" "cache_bucket" {
+ acceleration_status = (known after apply)
+ acl = "private"
+ arn = (known after apply)
+ bucket = (known after apply)
+ bucket_domain_name = (known after apply)
+ bucket_regional_domain_name = (known after apply)
+ force_destroy = true
+ hosted_zone_id = (known after apply)
+ id = (known after apply)
+ region = (known after apply)
+ request_payer = (known after apply)
+ tags = {
+ "Name" = "eg-test-codebuild-test"
+ "Namespace" = "eg"
+ "Stage" = "test"
}
+ tags_all = {
+ "Name" = "eg-test-codebuild-test"
+ "Namespace" = "eg"
+ "Stage" = "test"
}
+ website_domain = (known after apply)
+ website_endpoint = (known after apply)
+ lifecycle_rule {
+ enabled = true
+ id = "codebuildcache"
+ prefix = "/"
+ tags = {
+ "Name" = "eg-test-codebuild-test"
+ "Namespace" = "eg"
+ "Stage" = "test"
}
+ expiration {
+ days = 7
}
}
+ versioning {
+ enabled = true
+ mfa_delete = false
}
}
# module.codebuild.random_string.bucket_prefix[0] will be created
+ resource "random_string" "bucket_prefix" {
+ id = (known after apply)
+ length = 12
+ lower = true
+ min_lower = 0
+ min_numeric = 0
+ min_special = 0
+ min_upper = 0
+ number = false
+ result = (known after apply)
+ special = false
+ upper = false
}
Plan: 8 to add, 0 to change, 0 to destroy. |
|
All of my testing was done using the complete example. I cannot get a plan to pass with the random_string in use. The issue I referenced in the bug report is from another user who hit the exact same issue. |
terraform plan using examples/complete$ terraform plan -var-file fixtures.us-east-2.tfvars -var cache_bucket_suffix_enabled=true
╷
│ Error: cache location is required when cache type is "S3"
│
│ with module.codebuild.aws_codebuild_project.default[0],
│ on ../../main.tf line 292, in resource "aws_codebuild_project" "default":
│ 292: resource "aws_codebuild_project" "default" {
│
╵ |
|
I put together a minimal test project using random_string and it passes planning no problem. minimal test project using custom random_stringresource "random_string" "bucket_suffix" {
length = 12
number = false
upper = false
special = false
lower = true
}
resource "aws_s3_bucket" "test" {
bucket = "test-bucket-${random_string.bucket_suffix.result}"
}terraform {
required_version = ">= 0.13.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 2.0"
}
template = {
source = "hashicorp/template"
version = ">= 2.0"
}
local = {
source = "hashicorp/local"
version = ">= 1.2"
}
random = {
source = "hashicorp/random"
version = ">= 2.1"
}
null = {
source = "hashicorp/null"
version = ">= 2.0"
}
}
} |
|
If it's working in our tests but not for you, perhaps it has something to do with the version of terraform you're using. I tested locally using terraform 1.0 and the tests use terraform 0.13.x. |
|
it's not just me though: cloudposse/terraform-aws-ecs-codepipeline#71 |
3 tasks
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the Bug
When
cache_bucket_suffix_enabledisTrue, this module raises an error:This is having downstream effects. eg: cloudposse/terraform-aws-ecs-codepipeline#71
Expected Behavior
No error
Steps to Reproduce
The text was updated successfully, but these errors were encountered: