Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Introduced Terraform Plugin Framework #3751

Merged
merged 47 commits into from
Jul 29, 2024
Merged

[Feature] Introduced Terraform Plugin Framework #3751

merged 47 commits into from
Jul 29, 2024

Conversation

tanmay-db
Copy link
Contributor

@tanmay-db tanmay-db commented Jul 9, 2024 •
edited
Loading

Changes

This PR introduces plugin framework to the Databricks Terraform Provider.

  • Introduce plugin framework
  • Add core support for resource and data source.
  • Add provider schema
  • Add support to configure databricks client
  • Add support for provider fixture
  • Add support to get raw config to be used in provider tests and further in unit testing framework.
  • Add Quality Monitor resource using plugin framework
  • Add UC Volumes data source using plugin framework
  • Fix bugs related in configuring client
  • Fix bugs where provider schema is used instead of resource schema
  • Fix miscellaneous bugs

Tests

  • Run existing provider tests also for plugin framework. All passes.
  • Manually tested the above 2 resources E2E (apply works for data source, for resource -- we need customise schema support for SetReadOnly)

Note: We don't have unit and integration test support as of now, so going ahead with commiting the .tf files for these resources so we can test the changes in the meantime. Once we have proper support, these can be converted to example or removed.

image

tanmay-db and others added 28 commits March 27, 2024 13:20
@tanmay-db
temp work
ac375fa
@tanmay-db
-
b1dacae
@tanmay-db
mux support added, compiling works
f7b6a7e
@tanmay-db
temp work
232cb55
@tanmay-db
-
9f657ef
@tanmay-db
cleanup
fef1b40
@tanmay-db
-
c7be1a6
@tanmay-db
Merge branch 'main' of github.com:databricks/terraform-provider-datab…
e3a5bf0 
…ricks into introduce-plugin-framework-poc
@tanmay-db
more work
7a6dc2d
@tanmay-db
-
4b5966c
@tanmay-db
-
356fbbe
@nkvuong
added isolation mode support for databricks_external_location & `da…
483eb99 
…tabricks_storage_credential` (#3704)

* add isolation mode support for external location & storage credential

* add doc & test
@alexmos-db
Add terraform support for periodic triggers (#3700)
03c71d0 
* Add periodic triggers

* Add acceptance test for periodic triggers

* Fix typo
@hectorcast-db
Release v1.48.2 (#3722)
f42e1fb 
* Added isolation mode support for `databricks_external_location` & `databricks_storage_credential` ([#3704](#3704)).
* Add terraform support for periodic triggers ([#3700](#3700)).
@nkvuong
remove references to basic auth (#3720)
d669d7a
@rayalex
Fix invalid priviledges in grants.md (#3716)
0c252d4
@dependabot
Bump github.com/hashicorp/hcl/v2 from 2.20.1 to 2.21.0 (#3684)
1ba1772 
Bumps [github.com/hashicorp/hcl/v2](https://github.com/hashicorp/hcl) from 2.20.1 to 2.21.0.
- [Release notes](https://github.com/hashicorp/hcl/releases)
- [Changelog](https://github.com/hashicorp/hcl/blob/main/CHANGELOG.md)
- [Commits](hashicorp/hcl@v2.20.1...v2.21.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/hcl/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@nkvuong
Refactored databricks_cluster(s) data sources to Go SDK (#3685)
df210b2 
* relax cluster check

* fix

* fix

* refactor `databricks_cluster` data source to Go SDK

* refactor `databricks_clusters` data source to Go SDK
@nkvuong
Renamed databricks_catalog_workspace_binding to `databricks_workspa…
fc889cc 
…ce_binding` (#3703)

* rename resource

* fix test
@alexott
Exporter: fix generation of run_as blocks in databricks_job (#3724)
c6f949c 
* Exporter: fix generation of `run_as` blocks in `databricks_job`

Because the `run_as` was marked as `computed` it was ignored when generating the code.

* Ignore `run_as` for the current user
@karolusz @alexott @nkvuong
Adds databricks_volume as data source (#3211)
ff837ab 
* data_volume

* data_volume unit and acceptance tests

* docs

* WorkspaceDataWithCustomParams test

* fixed formatting

* Removing unnecessary changes to resource.go

* refactored data_volume

* making change for consitency with GO SDK v0.35.0

* Update catalog/data_volume.go

* Update catalog/data_volume.go

* data source as nested strucutre

* review comments addressed

* acceptance test

---------

Co-authored-by: Alex Ott <alexey.ott@databricks.com>
Co-authored-by: vuong-nguyen <44292934+nkvuong@users.noreply.github.com>
@touchida
Make the schedule.pause_status field read-only (#3692)
0d943ea
@nkvuong
Added support for binding storage credentials and external locations …
75236a6 
…to specific workspaces (#3678)

* add isolation mode

* rename

* doc

* fix doc

* add tests

* add acceptance tests

* add computed

* typo

* add tests

* use correct isolation_mode

* fix test
@alexott
Exporter: use Go SDK structs for databricks_job resource (#3727)
411f85c
@mgyucht
Change TF registry ownership (#3736)
e864065
@tanmay-db
resolve merge conflicts
dfede1a
@tanmay-db
update
4fdaace
@tanmay-db
wip
5980fb2
@tanmay-db tanmay-db requested review from a team as code owners July 9, 2024 16:14
@tanmay-db tanmay-db changed the title [WIP] Introduced Terraform Plugin Framework [Feature] Introduced Terraform Plugin Framework Jul 22, 2024
@tanmay-db tanmay-db requested a review from edwardfeng-db July 29, 2024 00:03
@tanmay-db tanmay-db requested review from alexott and mgyucht July 29, 2024 00:54
edwardfeng-db
edwardfeng-db reviewed Jul 29, 2024
View reviewed changes
pluginframework/resource_quality_monitor.go
c.SetRequired("assets_dir")
c.SetRequired("output_schema_name")
c.SetRequired("table_name")
// TODO: Uncomment this once SetReadOnly is supported in the plugin framework
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how to actually achieve this. As I saw in the provider schema types, all of the isReadOnly functions are hard coded to be returning false

edwardfeng-db
edwardfeng-db approved these changes Jul 29, 2024
View reviewed changes
Copy link
Contributor

@edwardfeng-db edwardfeng-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@tanmay-db tanmay-db merged commit 51411f1 into terraform-plugin-framework Jul 29, 2024
6 checks passed
@tanmay-db tanmay-db deleted the tanmay/introduce-plugin-framework branch July 29, 2024 12:31
@tanmay-db tanmay-db mentioned this pull request Aug 20, 2024
Merged
5 tasks
github-merge-queue bot pushed a commit that referenced this pull request Aug 22, 2024
@tanmay-db
[Feature] Introduced Plugin Framework (#3920)
8899beb 
## Changes
<!-- Summary of your changes that are easy to understand -->
This PR introduces plugin framework to the Databricks Terraform
Provider.
- Introduce plugin framework
- Add provider schema
- Add support to configure databricks client
- Add support for provider fixture
- Add support to get raw config to be used in provider tests and further
in unit testing framework.
- Fix bugs in configuring client + other misc bugs

Note: It is a subpart of:
#3751.
Rest would be added once support for schema customisation and SDK
converted are added.

## Tests
<!-- 
How is this tested? Please see the checklist below and also describe any
other relevant tests
-->
Unit tests, nightly were run over the above PRs. 
- [ ] `make test` run locally
- [ ] relevant change in `docs/` folder
- [ ] covered with integration tests in `internal/acceptance`
- [ ] relevant acceptance tests are passing
- [ ] using Go SDK
@tanmay-db tanmay-db mentioned this pull request Aug 27, 2024
Merged
5 tasks
github-merge-queue bot pushed a commit that referenced this pull request Aug 30, 2024
@tanmay-db @mgyucht
[Internal] Added databricks_quality_monitor resource and `databrick…
f4687d6 
…s_volumes` data source to plugin framework (#3958)

## Changes
<!-- Summary of your changes that are easy to understand -->
Add support for Quality Monitor resource and Volumes data source in
plugin framework. Also add utility method to get workspace and account
client with diagnostics.

Subpart of:
#3751 +
Addressed comments from:
https://github.com/databricks/terraform-provider-databricks/pull/3893/files#diff-8edec0f6289f63bfb61c7c6fb76fc1a61eec69c33fc2ba3a4ee6e77a8103f85b

Note: The changes for flipping
`databricks_quality_monitor_pluginframework` ->
`databricks_quality_monitor` in pluginframework along with making this
legacy for sdkv2 i.e. `databricks_quality_monitor` ->
`databricks_quality_monitor_legacy` will be done in separate PR. Same
for `databricks_volumes`

## Tests
<!-- 
How is this tested? Please see the checklist below and also describe any
other relevant tests
-->
Existing unit tests.

Integration and Unit tests for this resource will be merged after this
PR is merged.

- [ ] `make test` run locally
- [ ] relevant change in `docs/` folder
- [ ] covered with integration tests in `internal/acceptance`
- [ ] relevant acceptance tests are passing
- [ ] using Go SDK

---------

Co-authored-by: Miles Yucht <miles@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edwardfeng-db edwardfeng-db edwardfeng-db approved these changes

@alexott alexott Awaiting requested review from alexott

@mgyucht mgyucht Awaiting requested review from mgyucht

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
10 participants
@tanmay-db @edwardfeng-db @nkvuong @alexmos-db @hectorcast-db @rayalex @alexott @karolusz @touchida @mgyucht