Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Update databricks_permissions resource to support vector-search-endpoints #4209

Merged
merged 10 commits into from
Nov 13, 2024

Conversation

zgcalebp
Copy link
Contributor

@zgcalebp zgcalebp commented Nov 11, 2024

Changes

Databricks permissions API has been updated to support vector-search-endpoints. This corresponding change to the permissions_definitions would enable the use of the databricks_permissions resource to manage ACLs for Vector Search Endpoints leveraging the existing APIs under the hood.

Example CLI call confirming support:

databricks permissions get vector-search-endpoints {endpoint-id} --debug
18:56:25  INFO start pid=77800 version=0.224.1 args="databricks, permissions, get, vector-search-endpoints, {endpoint-id}, --debug"
18:56:25  INFO Ignoring pat auth, because databricks-cli is preferred pid=77800 sdk=true
18:56:25  INFO Ignoring basic auth, because databricks-cli is preferred pid=77800 sdk=true
18:56:25  INFO Ignoring oauth-m2m auth, because databricks-cli is preferred pid=77800 sdk=true
18:56:25  INFO Refreshed OAuth token from Databricks CLI, expires on 2024-11-11 19:00:53.515729 -0500 EST pid=77800 sdk=true
18:56:25 DEBUG Using Databricks CLI authentication with Databricks OAuth tokens pid=77800 sdk=true
18:56:25  INFO Refreshed OAuth token from Databricks CLI, expires on 2024-11-11 19:00:53.515729 -0500 EST pid=77800 sdk=true
18:56:26 DEBUG GET /api/2.0/permissions/vector-search-endpoints/{endpoint-id}
< HTTP/2.0 200 OK
< {
<   "access_control_list": [
....

Tests

  • make test run locally
  • relevant change in docs/ folder
  • covered with integration tests in internal/acceptance
  • relevant acceptance tests are passing
  • using Go SDK

@zgcalebp zgcalebp requested review from a team as code owners November 11, 2024 23:58
@zgcalebp zgcalebp requested review from rauchy and removed request for a team November 11, 2024 23:58
Copy link
Contributor

@alexott alexott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes looks good - thank you.

can you please add a permission resource inside this integration test: https://github.com/databricks/terraform-provider-databricks/blob/main/internal/acceptance/vector_search_test.go#L20?

@alexott
Copy link
Contributor

alexott commented Nov 12, 2024

And please do make fmt

@zgcalebp
Copy link
Contributor Author

Sorry for the typos on that last commit 😅 . Working through getting these integration tests to run locally now. Thanks for the quick review too!

@zgcalebp
Copy link
Contributor Author

Was able to validate the MR updates locally, should be good to merge upon review and Git action completions. Let me know if any further input is needed!

@zgcalebp
Copy link
Contributor Author

Thanks @alexott for the test update. Should've more strongly considered the provisioning time for a permissions test.

@alexott
Copy link
Contributor

alexott commented Nov 12, 2024

It's very specific for vector search endpoint - I realised that test was running too long, and made the update.
I'll merge as soon as tests are passing

thank you for your contribution!

@alexott alexott enabled auto-merge November 12, 2024 19:11
@zgcalebp
Copy link
Contributor Author

Is there some way to view the output of the integration tests? Surprised they aren't passing, happy to update to fix.

auto-merge was automatically disabled November 13, 2024 02:20

Head branch was pushed to by a user without write access

@alexott
Copy link
Contributor

alexott commented Nov 13, 2024

@zgcalebp The tests were failing because I ran the original patch with full-blown permissions test, but VS requires UC, and the test name wasn't adjusted for running only on UC-enabled workspaces

@alexott alexott enabled auto-merge November 13, 2024 08:04
Copy link

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/terraform

Inputs:

  • PR number: 4209
  • Commit SHA: 81605c2aa8673d2f0c7eedae14deeba3276748f1

Checks will be approved automatically on success.

@eng-dev-ecosystem-bot
Copy link
Collaborator

Test Details: go/deco-tests/11813686757

@alexott alexott added this pull request to the merge queue Nov 13, 2024
Merged via the queue into databricks:main with commit e3b2561 Nov 13, 2024
12 checks passed
hectorcast-db added a commit that referenced this pull request Nov 20, 2024
### New Features and Improvements

 * Add `databricks_mws_network_connectivity_config` and `databricks_mws_network_connectivity_configs` data source ([#3665](#3665)).
 * Add support partitions in policy data sources ([#4181](#4181)).
 * Added `databricks_registered_model_versions` data source ([#4100](#4100)).
 * Update databricks_permissions resource to support vector-search-endpoints ([#4209](#4209)).
 * add `databricks_serving_endpoints` data source ([#4226](#4226)).

### Bug Fixes

 * Add validation for `run_as_mode` in `databricks_query` ([#4233](#4233)).
 * Correct handling of updates for empty comments and `force_destroy` in UC catalog, schema, registered models and volumes ([#4244](#4244)).
 * Fix deletion of dashboard if it was trashed out of band ([#4235](#4235)).
 * Fix waiting for `databricks_vector_search_index` readiness ([#4243](#4243)).
 * Remove single-node validation from interactive clusters ([#4222](#4222)).
 * Remove single-node validation from jobs clusters ([#4216](#4216)).
 * Use cluster list API to determine pinned cluster status ([#4203](#4203)).
 * fix issue cased by setting pause_status in update monitor  ([#4242](#4242)).

### Documentation

 * Clarify workspace provider config ([#4208](#4208)).
 * Update "Databricks Workspace Creator" permissions on gcp-workspace.md ([#4201](#4201)).
 * Update `grants.md` references ([#4246](#4246)).
 * Update description of `group_id` in `databricks_mws_ncc_private_endpoint_rule` ([#4238](#4238)).
 * remove subnet sharing limitation in AWS ([#4239](#4239)).

### Internal Changes

 * Bump Go SDK to latest and generate TF structs ([#4249](#4249)).
 * Mark TestUcAccModelServingProvisionedThroughput as flaky. to be rever… ([#4232](#4232)).
 * Rename resources directory to products in pluginframework ([#4139](#4139)).
 * Revert "mark TestUcAccModelServingProvisionedThroughput as flaky. to … ([#4240](#4240)).
 * Set user agent in some resources implemented in plugin framework ([#4187](#4187)).
 * make `ApplyAndExpectData` work with nested set ([#4237](#4237)).

### Dependency Updates

 * Bump dependencies for Plugin Framework and SDK v2 ([#4215](#4215)).
 * Bump github.com/hashicorp/hcl/v2 from 2.22.0 to 2.23.0 ([#4236](#4236)).
 * Bump github.com/hashicorp/terraform-plugin-testing from 1.10.0 to 1.11.0 ([#4247](#4247)).

### Exporter

 * Add `List` operation for `users` service ([#4204](#4204)).
 * Fix interactive selection of services ([#4245](#4245)).
hectorcast-db added a commit that referenced this pull request Nov 20, 2024
 * Add `databricks_mws_network_connectivity_config` and `databricks_mws_network_connectivity_configs` data source ([#3665](#3665)).
 * Add support partitions in policy data sources ([#4181](#4181)).
 * Added `databricks_registered_model_versions` data source ([#4100](#4100)).
 * Update databricks_permissions resource to support vector-search-endpoints ([#4209](#4209)).
 * add `databricks_serving_endpoints` data source ([#4226](#4226)).

 * Add validation for `run_as_mode` in `databricks_query` ([#4233](#4233)).
 * Correct handling of updates for empty comments and `force_destroy` in UC catalog, schema, registered models and volumes ([#4244](#4244)).
 * Fix deletion of dashboard if it was trashed out of band ([#4235](#4235)).
 * Fix waiting for `databricks_vector_search_index` readiness ([#4243](#4243)).
 * Remove single-node validation from interactive clusters ([#4222](#4222)).
 * Remove single-node validation from jobs clusters ([#4216](#4216)).
 * Use cluster list API to determine pinned cluster status ([#4203](#4203)).
 * fix issue cased by setting pause_status in update monitor  ([#4242](#4242)).

 * Clarify workspace provider config ([#4208](#4208)).
 * Update "Databricks Workspace Creator" permissions on gcp-workspace.md ([#4201](#4201)).
 * Update `grants.md` references ([#4246](#4246)).
 * Update description of `group_id` in `databricks_mws_ncc_private_endpoint_rule` ([#4238](#4238)).
 * remove subnet sharing limitation in AWS ([#4239](#4239)).

 * Bump Go SDK to latest and generate TF structs ([#4249](#4249)).
 * Mark TestUcAccModelServingProvisionedThroughput as flaky. to be rever… ([#4232](#4232)).
 * Rename resources directory to products in pluginframework ([#4139](#4139)).
 * Revert "mark TestUcAccModelServingProvisionedThroughput as flaky. to … ([#4240](#4240)).
 * Set user agent in some resources implemented in plugin framework ([#4187](#4187)).
 * make `ApplyAndExpectData` work with nested set ([#4237](#4237)).

 * Bump dependencies for Plugin Framework and SDK v2 ([#4215](#4215)).
 * Bump github.com/hashicorp/hcl/v2 from 2.22.0 to 2.23.0 ([#4236](#4236)).
 * Bump github.com/hashicorp/terraform-plugin-testing from 1.10.0 to 1.11.0 ([#4247](#4247)).

 * Add `List` operation for `users` service ([#4204](#4204)).
 * Fix interactive selection of services ([#4245](#4245)).
github-merge-queue bot pushed a commit that referenced this pull request Nov 20, 2024
### New Features and Improvements

* Add `databricks_mws_network_connectivity_config` and
`databricks_mws_network_connectivity_configs` data source
([#3665](#3665)).
* Add support partitions in policy data sources
([#4181](#4181)).
* Added `databricks_registered_model_versions` data source
([#4100](#4100)).
* Update databricks_permissions resource to support
vector-search-endpoints
([#4209](#4209)).
* add `databricks_serving_endpoints` data source
([#4226](#4226)).


### Bug Fixes

* Add validation for `run_as_mode` in `databricks_query`
([#4233](#4233)).
* Correct handling of updates for empty comments and `force_destroy` in
UC catalog, schema, registered models and volumes
([#4244](#4244)).
* Fix deletion of dashboard if it was trashed out of band
([#4235](#4235)).
* Fix waiting for `databricks_vector_search_index` readiness
([#4243](#4243)).
* Remove single-node validation from interactive clusters
([#4222](#4222)).
* Remove single-node validation from jobs clusters
([#4216](#4216)).
* Use cluster list API to determine pinned cluster status
([#4203](#4203)).
* fix issue cased by setting pause_status in update monitor
([#4242](#4242)).


### Documentation

* Clarify workspace provider config
([#4208](#4208)).
* Update "Databricks Workspace Creator" permissions on gcp-workspace.md
([#4201](#4201)).
* Update `grants.md` references
([#4246](#4246)).
* Update description of `group_id` in
`databricks_mws_ncc_private_endpoint_rule`
([#4238](#4238)).
* remove subnet sharing limitation in AWS
([#4239](#4239)).


### Internal Changes

* Bump Go SDK to latest and generate TF structs
([#4249](#4249)).
* Mark TestUcAccModelServingProvisionedThroughput as flaky. to be rever…
([#4232](#4232)).
* Rename resources directory to products in pluginframework
([#4139](#4139)).
* Revert "mark TestUcAccModelServingProvisionedThroughput as flaky. to …
([#4240](#4240)).
* Set user agent in some resources implemented in plugin framework
([#4187](#4187)).
* make `ApplyAndExpectData` work with nested set
([#4237](#4237)).


### Dependency Updates

* Bump dependencies for Plugin Framework and SDK v2
([#4215](#4215)).
* Bump github.com/hashicorp/hcl/v2 from 2.22.0 to 2.23.0
([#4236](#4236)).
* Bump github.com/hashicorp/terraform-plugin-testing from 1.10.0 to
1.11.0
([#4247](#4247)).


### Exporter

* Add `List` operation for `users` service
([#4204](#4204)).
* Fix interactive selection of services
([#4245](#4245)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants