Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added storage namespace validation on non-readonly bare repo creation #8364

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

tkalir
Copy link

@tkalir tkalir commented Nov 13, 2024

Closes #3165

Background

Currently bare repo creation doesn't check if namesapce storage is writable \ if a lakefs repo already exists there

Bug Fix

In Controller.go\CreateRepository, previously the code was -

  1. if bare repo, create and return
  2. if non-readonly repo, do namespace validation
    I only switched them places, so namespace validation happens also for non-readonly bare repos. nothing else is supposed to change. (but that does mean that bare repos create a dummy object at namespace root now, like other repos - see Controller.go\ensureStorageNamespace)

Testing Details

I added a system test to check that creating a bare repo in same namespace as existing repo returns an error. we can remove the test if it's unnecessary, but it will help to check the code.

Contact Details

tkalir@gmail.com \ Tamar Kalir on slack

@tkalir tkalir marked this pull request as ready for review November 16, 2024 14:57
@N-o-Z N-o-Z self-requested a review November 21, 2024 18:01
Copy link
Member

@N-o-Z N-o-Z left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @tkalir great job!

Though this is a small PR I'm requesting changes for the test as this is your first PR and I think this can be a good learning experience going into the lakeFS code.

@@ -215,6 +215,24 @@ func TestController_GetRepoHandler(t *testing.T) {
})
}

t.Run("use same storage namespace twice with bare repo", func(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great test - but note you could have reused the code of "use same storage namespace twice" and parameterize the test.
Look at the example of catalog_test.go:564 as reference

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, thanks for the feedback. I did that and added the parameter (isBareRepo=true\false) into the subtest name, thought it might be helpful.

@tkalir tkalir force-pushed the fix/add-namespace-validation-for-bare-repos-3165 branch from 74e509f to a593aad Compare November 24, 2024 18:58
Copy link
Contributor

@arielshaqed arielshaqed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tkalir for this contribution! You have clearly given much thought to how to perform this change.

Unfortunately I am blocking this as I have second thoughts about the issue. In brief, the intent behind this change is to prevent some ways to use a plumbing command. I believe that this will prevent users from pursuing some perfectly reasonable flows.

Fortunately the core team is meeting 🥳 this week (hi, @N-o-Z)! So we will discuss it and resolve one way or another. In the meanwhile, I apologize for inducing delay.

Copy link
Contributor

@arielshaqed arielshaqed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okaaaaayyyy.... Obviously it's the simplest things that take the most discussions. Sorry about the time this took After discussing with other stakeholders, I am removing my objections to this feature. Let's continue!

I think @N-o-Z is still blocking for relevant reasons. so over to you, N!

Copy link
Member

@N-o-Z N-o-Z left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tkalir Thanks!
Need to merge latest master from repository and resolve conflicts before we can continue to merge this back.
For future reference - please avoid from amending commits (and pushing force) or rebasing during the review process. It makes it much harder to track the changes this way.
The best way is to add additional changes in new commits.

@N-o-Z
Copy link
Member

N-o-Z commented Dec 16, 2024

Note that the unit test is failing

@N-o-Z
Copy link
Member

N-o-Z commented Jan 6, 2025

@tkalir hi, please let us know if you still intend to work on this. In order to merge it you'll first need to resolve the conflicts

@tkalir
Copy link
Author

tkalir commented Jan 6, 2025

@N-o-Z hi, sorry for the delay, I will work on it soon. I understand that I need to resolve the conflicts and update the failing unit test according to the change I made.
I encountered some issues with running the tests locally (with "make test", not running individual tests) so it also took some time to try and resolve that. I will ask on slack if I need further help. Thanks for the follow up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Creating bare repository skips storage namespace validation
3 participants