Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[raz] Make user access for GS compatible with existing RAZ implementation #3511

Merged
merged 9 commits into from
Nov 7, 2023
Merged

[raz] Make user access for GS compatible with existing RAZ implementation #3511

merged 9 commits into from
Nov 7, 2023

Conversation

Harshg999
Copy link
Collaborator

@Harshg999 Harshg999 commented Oct 24, 2023 •
edited
Loading

What changes were proposed in this pull request?

  • Piggybacking on S3, this PR allows the Google Storage access in Hue to pass through the already existing RAZ implementation for fine-grain access control.

How was this patch tested?

  • Manual testing in live cluster

@Harshg999 Harshg999 marked this pull request as draft October 24, 2023 17:13
@Harshg999 Harshg999 changed the title [raz] Add plumbing GS RAZ changes with existing implementation [DO_NOT_MERGE][raz] Add plumbing GS RAZ changes with existing implementation Oct 24, 2023
@Harshg999 Harshg999 changed the title [DO_NOT_MERGE][raz] Add plumbing GS RAZ changes with existing implementation [raz] Make user access for GS compatible with existing RAZ implementation Nov 1, 2023
@Harshg999 Harshg999 marked this pull request as ready for review November 1, 2023 11:03
amitsrivastava
amitsrivastava reviewed Nov 1, 2023
View reviewed changes
desktop/core/src/desktop/lib/fs/gc/client.py Show resolved Hide resolved
desktop/core/src/desktop/lib/fs/gc/gsconnection.py Outdated Show resolved Hide resolved
desktop/core/src/desktop/lib/fs/gc/gsconnection.py
Example of a presigned GS Url declaring a `list all buckets` call:
https://s3-us-west-1.amazonaws.com/?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA23E77ZX2HVY76YGL%2F20210505%2Fus-west-1%2Fs3%2Faws4_request&X-Amz-Date=20210505T171457Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=994d0ec2ca19a00aa2925fe62cab0e727591b1951a8a47504b2b9124facbd6cf
"""
def __init__(self, username, gs_access_key_id=None, gs_secret_access_key=None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need access_key and secret even for signed url's?

Copy link
Collaborator Author

@Harshg999 Harshg999 Nov 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For RAZ scenarios we don't need access key and secret. I had these placeholders here to have better readability of what all values are actually present and what we are changing when kind of monkey patching and injecting RAZ headers in the boto methods.

Any views/best practices you can recommend here? or this looks good?

desktop/core/src/desktop/lib/fs/gc/gsconnection.py Show resolved Hide resolved
desktop/core/src/desktop/lib/fs/gc/gsconnection.py Outdated Show resolved Hide resolved
desktop/core/src/desktop/lib/raz/clients.py Show resolved Hide resolved
desktop/core/src/desktop/lib/raz/raz_client.py Show resolved Hide resolved
JohanAhlen
JohanAhlen approved these changes Nov 7, 2023
View reviewed changes
Copy link
Contributor

@JohanAhlen JohanAhlen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

amitsrivastava
amitsrivastava approved these changes Nov 7, 2023
View reviewed changes
Copy link
Collaborator

@amitsrivastava amitsrivastava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Harshg999 Harshg999 merged commit df26eff into master Nov 7, 2023
2 checks passed
@Harshg999 Harshg999 deleted the raz-gs branch November 7, 2023 15:09
athithyaaselvam pushed a commit that referenced this pull request Feb 5, 2024
@Harshg999
[raz] Make user access for GS compatible with existing RAZ implementa…
e244164 
…tion (#3511)

(cherry picked from commit df26eff)
wing2fly pushed a commit that referenced this pull request Mar 6, 2024
@Harshg999 @wing2fly
CDPD-48829: [raz] Make user access for GS compatible with existing RA…
c46492a 
…Z implementation (#3511)

(cherry picked from commit df26eff)
(cherry picked from commit e244164)
Change-Id: I011f6d81925ec0f9cbc27ec11e7b32f0be382735
(cherry picked from commit d7f486a3739eb5aaa1e3a47a43895511085be4e1)
athithyaaselvam pushed a commit that referenced this pull request Mar 14, 2024
@Harshg999 @wing2fly
CDPD-48829: [raz] Make user access for GS compatible with existing RA…
8ae91b6 
…Z implementation (#3511)

(cherry picked from commit df26eff)
(cherry picked from commit e244164)
Change-Id: I011f6d81925ec0f9cbc27ec11e7b32f0be382735
Alterrien pushed a commit to Alterrien/hue that referenced this pull request Dec 19, 2024
@Harshg999
[raz] Make user access for GS compatible with existing RAZ implementa…
501750c 
…tion (cloudera#3511)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JohanAhlen JohanAhlen JohanAhlen approved these changes

@amitsrivastava amitsrivastava amitsrivastava approved these changes

@bjornalm bjornalm Awaiting requested review from bjornalm

@ranade1 ranade1 Awaiting requested review from ranade1

@athithyaaselvam athithyaaselvam Awaiting requested review from athithyaaselvam

@agl29 agl29 Awaiting requested review from agl29

@wing2fly wing2fly Awaiting requested review from wing2fly

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Harshg999 @wing2fly @amitsrivastava @JohanAhlen