Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📈 Document a Process for registering and sharing a Database/Table/Column from Ireland APDP to London APC using Lake Formation #4468

Closed
2 tasks done
Tracked by #2955
julialawrence opened this issue Jun 5, 2024 · 2 comments

Comments

@julialawrence
Copy link
Contributor

julialawrence commented Jun 5, 2024

User Story

In order to keep the scope of the QuickSight MVP small, we have agreed to limit the initial implementation to a handful of pre-selected datasources. However, participants might want to add additional ones over the course of the testing, so to aid in that, we need a fully defined, documented process (manual or scripted) to share a resource from Ireland analytical-platform-data-production to London analytical-platform-compute

Value / Purpose

Documenting this would allow us to easily expand the scope of our MVP to new users and new datasets if required.

Useful Contacts

@julialawrence @MichaelJCollins @jamesstott

User Types

Data Engineers

Hypothesis

If we implement this, we can be more flexible with the scope of our MVP testing, expanding it if it appropriate to do so.

Proposal

Document and test the process to do the following:

  1. Register a data location in anaytical-platform-data-production in Lake Formation in hybrid mode.
  2. Share resources from Ireland APDP into analytical-platform-compute Ireland.
  3. Share resources from Ireland APC to London APC
  4. Identity a data engineering user to be the admin of the shared datasource
  5. Assign them grantable permissions over the shared resource to allow them to continue managing access and grant it to others via the new UI.

Additional Information

https://docs.google.com/document/d/1Xwbvc8ipI2m6nlK3et-TUcrksjhV9MmNqEVFtVbtkOQ

Assume the sharing is done via named resources.

Definition of Done

  • Sharing process documented and tested
  • [] Instructions published as a playbookl - No longer required team decision to use lakeformation terraform module for ease of use
  • Another team member has reviewed
@Emterry
Copy link
Contributor

Emterry commented Jul 1, 2024

Document for testing sendong a lakeformation resource from AP Dev(ireland) -> AP Data Dev(ireland) then AP data dev(ireland) to AP data dev(London)
https://docs.google.com/document/d/1bMk5onKLs1RC8oOXjLhlmAu84ZilAdxanvd7AAWqDRY/edit

@Emterry Emterry moved this from 🚀 In Progress to 🛂 In Review in Analytical Platform Jul 1, 2024
@Emterry
Copy link
Contributor

Emterry commented Jul 9, 2024

Closing as document reviewed by team - decision to implement terraform module for lake formation as main usecase going forward due to ease of use

@Emterry Emterry closed this as completed Jul 9, 2024
@github-project-automation github-project-automation bot moved this from 🛂 In Review to 🎉 Done in Analytical Platform Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

2 participants