[Internal] Remove TF annotations on TFSDK models

[rauchy]

Changes

This PR changes how schemas are defined for plugin framework resources and data sources. Instead of encoding the schema attributes in a tf tag, which involves reflection and code that's harder to reason about, structs must now implement the ApplySchemaCustomizations method and declare schema customizations explicitly there. All TFSDK structs will have this method generated for them, and if you want to declare a schema for your own struct, you should do so as follows:

// BEFORE
type TestDeprecatedTfTags struct {
	Foo types.String `tfsdk:"foo" tf:"computed,optional"`
}

// AFTER
type TestDeprecatedTfTags struct {
	Foo types.String `tfsdk:"foo"`
}

func (TestNestedMapTfSdk) ApplySchemaCustomizations(attrs map[string]AttributeBuilder) map[string]AttributeBuilder {
	attrs["foo"] = attrs["foo"].SetComputed().SetOptional()
	return attrs
}

Tests

• make test run locally
• relevant change in docs/ folder
• covered with integration tests in internal/acceptance
• using Go SDK
• using TF Plugin Framework

[alexott]

Imho, it's making it harder to understand from looking into struct what fields are optional/computed/...

[mgyucht]

If we only expose the underlying map, some customizations are a bit more cumbersome right?

[rauchy]

@alexott you're right, but it does simplify the reflection parts and moving forward, tf annotations will limit us from further describing the schema with things like descriptions, plan modifiers and validators. It's basically a language we invented which will grow in maintenance.

Will it help if we add these as inline comments? For example,

type CreateRepoRequest struct {
	Url types.String `tfsdk:"url"` // tf:optional
}

[alexott]

comment may help.

another question - how would we additionally tune the behavior? Right now, many structs in Go SDK just blindly declare everything as omitempty even for required fields. If such tuning needs to be done in OpenAPI spec, then it will be cumbersome

[rauchy]

@alexott OpenAPI should be the preferred way, as it'll help things stay consistent across the board. However, if that process is problematic, you could still provide a customizeSchema function to ResourceStructToSchema or DataSourceStructToSchema and handle any manual customizations there.

[mgyucht]

Missing list validator for single object

[rauchy]

This is fixed now. I had the conditional order wrong and now it's rendering properly. I do wonder how the missing list validator didn't pop up during make diff-schema.

[mgyucht]

Validators may not show up on diff-schema, I don't know if they are part of the JSON representation (because they are code), similar to plan modifiers.

[mgyucht]

@alexott Our goal is to ensure that there is no extra customization needed in TF beyond what is in the API spec. The real issue is actually that the OpenAPI spec maintained by teams doesn't reflect reality of their API (defined using Protobuf). We are asking all teams to migrate to the OpenAPI generator to eliminate this discrepancy; then, there should be perfect alignment between the API and SDKs/Terraform.

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/terraform

Inputs:

• PR number: 4383
• Commit SHA: c313cf223d44ce04ce90b1a5c4f2bafc427b94c1

Checks will be approved automatically on success.

[mgyucht]

LGTM, but I'm just curious if .(tfschema.AttributeBuilder) is needed when assigning the attributebuilder updated with validator to the attribute builder map.

[rauchy]

LGTM, but I'm just curious if .(tfschema.AttributeBuilder) is needed when assigning the attributebuilder updated with validator to the attribute builder map.

Unfortunately that results in a
tfschema.BaseSchemaBuilder does not implement tfschema.AttributeBuilder (missing method BuildDataSourceAttribute)