Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow passing the same PluginGenerator several times #158

Merged
merged 2 commits into from
Jan 27, 2021

Conversation

bjaglin
Copy link
Contributor

@bjaglin bjaglin commented Jan 26, 2021

Trying out support for bjaglin/sbt-protoc@53dbd6a#diff-bbd8a5163e4909c406d2fd2b12cca4345f91859fae3676fd8ee189d121e2c9f1R2-R10 so that https://github.com/scalapb/common-protos/blob/b55b43071974159f9ab05de8457d66e107be86c5/project/BuildHelpers.scala#L61-L65 could package both flat_package and regular stubs in the same JAR (unless we want to publish with a flat classifier?).

Utltimately, my goal is to make scalapb-validate works out of the box with akka-grpc, as I realized the instructions I added in akka/akka-grpc@19f1be0#diff-4780ac674b878a4a2a35e6740dc7efce31de8de2e342719c694de7a36a8e1f83R55-R61 are not enough:

object ValidateProto is not a member of package io.envoyproxy.pgv.validate
[error]     io.envoyproxy.pgv.validate.ValidateProto

Comment on lines -108 to -117
it should "not allow ambigious paths for plugins" in {
intercept[RuntimeException] {
run(
Seq(
Target(gens.plugin("foo", "/path/to/plugin"), TmpPath1),
Target(gens.plugin("foo", "/other/path/to/plugin"), TmpPath1)
)
)
}.getMessage() must be("Different paths found for the plugin: foo")
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't tell if this was a safeguard or something unsupported?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing --protoc-gen-foo=path1 and --protoc-gen-foo=path2 is ambiguous, which one should protoc invoke for --foo_out=/dir ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I guess that's a limitation of the current implementation, as this PR handles it by giving unique names to each target

run(
Seq(
Target(gens.plugin("foo", "/path/to/plugin"), TmpPath),
Target(gens.plugin("foo", "/otherpath/to/plugin"), TmpPath1),
Target(gens.plugin("foo"), TmpPath2),
Target(gens.plugin("bar"), TmpPath)
)
) must be(
Seq(
"--plugin=protoc-gen-foo_0=/path/to/plugin",
"--plugin=protoc-gen-foo_1=/otherpath/to/plugin",
s"--foo_0_out=$TmpPath",
s"--foo_1_out=$TmpPath1",
s"--foo_2_out=$TmpPath2",
s"--bar_out=$TmpPath"
)
)
. In other words, the current implementation in this PR does not leverage the fact that the same plugin may have different outputs, in order to support different options for each output. We could continue using that feature as long as options are the same across outputs, but it would make the implementation harder, and I don't know if there is any gain to that?

Copy link
Contributor Author

@bjaglin bjaglin Jan 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added 8f2f6a9 75124d5 ede30d3 d37025e to showcase this

@bjaglin bjaglin closed this Jan 26, 2021
@bjaglin bjaglin reopened this Jan 27, 2021
@bjaglin bjaglin marked this pull request as ready for review January 27, 2021 06:50
@bjaglin bjaglin force-pushed the repeated-plugin branch 3 times, most recently from ede30d3 to 4ffeb2f Compare January 27, 2021 07:04
Comment on lines +101 to +114
"--plugin=protoc-gen-foo_0=/path/to/plugin",
"--plugin=protoc-gen-foo_1=/path/to/plugin",
"--plugin=protoc-gen-foo_2=/otherpath/to/plugin",
"--plugin=protoc-gen-foo_3=/otherpath/to/plugin",
s"--foo_0_out=$TmpPath",
s"--foo_0_opt=w",
s"--foo_1_out=$TmpPath",
s"--foo_1_opt=x",
s"--foo_2_out=$TmpPath1",
s"--foo_2_opt=y",
s"--foo_3_out=$TmpPath2",
s"--foo_3_opt=y",
s"--foo_4_out=$TmpPath2",
s"--foo_4_opt=z",
Copy link
Contributor Author

@bjaglin bjaglin Jan 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed above, this would also work but would make the suffixing strategy (that the client currently needs to know about when the path is not defined as part of the target) even more complicated.

Suggested change
"--plugin=protoc-gen-foo_0=/path/to/plugin",
"--plugin=protoc-gen-foo_1=/path/to/plugin",
"--plugin=protoc-gen-foo_2=/otherpath/to/plugin",
"--plugin=protoc-gen-foo_3=/otherpath/to/plugin",
s"--foo_0_out=$TmpPath",
s"--foo_0_opt=w",
s"--foo_1_out=$TmpPath",
s"--foo_1_opt=x",
s"--foo_2_out=$TmpPath1",
s"--foo_2_opt=y",
s"--foo_3_out=$TmpPath2",
s"--foo_3_opt=y",
s"--foo_4_out=$TmpPath2",
s"--foo_4_opt=z",
"--plugin=protoc-gen-foo_0=/path/to/plugin",
"--plugin=protoc-gen-foo_1=/path/to/plugin",
"--plugin=protoc-gen-foo_2=/otherpath/to/plugin",
s"--foo_0_out=$TmpPath",
s"--foo_0_opt=w",
s"--foo_1_out=$TmpPath",
s"--foo_1_opt=x",
s"--foo_2_out=$TmpPath1",
s"--foo_2_out=$TmpPath2",
s"--foo_2_opt=y",
s"--foo_3_out=$TmpPath2",
s"--foo_3_opt=z",

@thesamet
Copy link
Contributor

thesamet commented Jan 27, 2021

It seems that passing the same plugin multiple times with different options works prior to this change. I tried this on a sample project:

PB.targets in Compile := Seq(
  scalapb.gen(javaConversions=true) -> (sourceManaged in Compile).value / "scalapb",
  scalapb.gen(javaConversions=true, flatPackage=true) -> (sourceManaged in Compile).value / "scalapb-flat"
)

Looks like there's test coverage for this as well.

Is there a specific case related to running the same plugin twice that is not already supported?

I am not excited about packaging both flat and non-flat files in the same JAR nor publishing them separately (with a flat prefix). I believe it will work, but it will lead to confusion due to the duplicate classes available for the same messages. There are also a few gotchas around duplicate descriptors being around for the same message (some code assumes there's exactly one). I filed scalapb/scalapb-validate#70 so we can identify a solution for users of akka-grpc.

@bjaglin
Copy link
Contributor Author

bjaglin commented Jan 27, 2021

Is there a specific case related to running the same plugin twice that is not already supported?

Yes, this is about running native plugins (the PluginGenerator in the title is maybe not precise enough), not JVM ones, like in https://github.com/scalapb/common-protos/blob/b55b43071974159f9ab05de8457d66e107be86c5/project/BuildHelpers.scala#L60-L64.

@bjaglin
Copy link
Contributor Author

bjaglin commented Jan 27, 2021

By the way, I didn't open a PR but bjaglin/sbt-protoc@53dbd6a#diff-bbd8a5163e4909c406d2fd2b12cca4345f91859fae3676fd8ee189d121e2c9f1R2-R10 demonstrate this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants