Error reading Schema Registry Clusters when environment is created in the same step

[kaat-nnit]

After upgrading to Terraform provider version 2.*.0 where schema registry cluster resource is removed, schema registry cluster data source read fails when read is deferred to the apply phase.

Terraform Confluent provider: 2.0.0 and higher
Terraform version: 1.9.7

Terraform setup:

module "confluent_environment" {
 ...
}

module "schema_registry" {
  ...

  confluent_environment_id = module.confluent_environment.confluent_environment_id

  ...
}

¨
This works fine only in a setup where the environment already exists and we swapped the schema registry cluster resource for data source following the Confluent Provider 2.0.0: Upgrade Guide.

Issue is encountered when trying to deploy an environment and read the data source in the same plan/apply sequence. The same result is obtained both with an implicit dependency and when using the depends_on block.

Plan output:

# module.schema_registry.data.confluent_schema_registry_cluster.schema_registry will be read during apply
  # (config refers to values not yet known)
 <= data "confluent_schema_registry_cluster" "schema_registry" {
      + api_version   = (known after apply)
      + cloud         = (known after apply)
      + display_name  = (known after apply)
      + id            = (known after apply)
      + kind          = (known after apply)
      + package       = (known after apply)
      + region        = (known after apply)
      + resource_name = (known after apply)
      + rest_endpoint = (known after apply)

      + environment {
          + id = (known after apply)
        }
    }

Expected behaviour:
Data source is read once the environment is created, and the apply step completes successfully.

Actual behaviour:
Apply step fails with error: Error: error reading Schema Registry Clusters: there are no SR clusters in "env-***" environment

Workaround:
Introducing 5 seconds delay between environment creation and data source read with time_sleep resource results in a successful apply phase.

Is it possible to introduce this delay in the data source itself such that using the time_sleep resource is not necessary for this setup to work?

[linouk23]

Thanks for creating this issue @kaat-nnit!

Workaround:
Introducing 5 seconds delay between environment creation and data source read with time_sleep resource results in a successful apply phase. Is it possible to introduce this delay in the data source itself such that using the time_sleep resource is not necessary for this setup to work?

It seems like a great idea, thanks for sharing it 👍

schema registry cluster data source read fails when read is deferred to the apply phase.

Just out of my curiosity, could you share more details on how you managed to defer it to the apply phase?

[kaat-nnit]

Sure thing!
It is an automatic terraform behaviour with data sources, when data source refers to some attributes which are not yet computed. In this instance, schema registry cluster data source uses environment ID to find the schema registry to read, hence the read is deferred to the apply phase, after the environment has been created and it's ID is known.

[linouk23]

@kaat-nnit, by the way, could you confirm that you have the following depends_on block present in your Terraform configuration?

data "confluent_schema_registry_cluster" "essentials" {
  environment {
    id = confluent_environment.staging.id
  }

  depends_on = [
    confluent_kafka_cluster.basic
  ]
}

[linouk23]

@kaat-nnit did you get a chance to try out the following suggestion #452 (comment)? Thank you!

[kaat-nnit]

@linouk23 thank you for the suggestion! We haven't had a chance to try it out yet, I will provide an update once we have.