Go: add license detection

[arieltorti]

Go mod currently doesn't provide license or copyright information for go modules hence ORT doesn't provide detected licenses for them.

In order to add license detection using go mod I propose to either integrate ORT with https://github.com/google/go-licenses or apply the same technique.

[sschuberth]

Go mod currently doesn't provide license or copyright information for go modules hence ORT doesn't provide detected licenses for them.

Before going deeper into the discussion, let's ensure we're aligned on the terminology: What ORT calls "detected licenses" (and copyrights) are those determined by ORT's scanner tool. I.e. license / copyright findings returned by a scanner like ScanCode.

On the other hand, in ORT's terminology "declared licenses" (and copyrights) stem from package metadata (probably go.mod in this case) or package registries, and these are determined by ORT's analyzer tool.

So, what I assume you're saying is that ORT's analyzer fails to find declared licenses for GoMod?

What's the source for license information that https://github.com/google/go-licenses uses, is it metadata or source code?

[fviernau]

ORT distinguishes between declared and detected licenses. Declared licenses are defined in the package metadata, whereas go.mod files do not provide any means for doing so. Detected licenses are the licenses which reside in any file in the corresponding code respository. As ORT's GoMod integration is capable of figuring out the source code location, either the source artifact '.zip'' file or the VCS location, running ORT's scan stage on top of the analysis result will in fact detect the licenses as well as copyright statements.

The license tool above seems to run a license scanner just against the LICENSE file, while ORT runs it against any file in the code repository (disregarding some exclusion patterns for special files). The primary scanner used by ORT is ScanCode. So, how does ScanCode compare to above tool you mentioned?

All in all, I believe this ticket is invalid.

edit: After writing this I found @sschuberth wrote at the same time, so sorry for the slight redundancy.

[sschuberth]

What's the source for license information that https://github.com/google/go-licenses uses, is it metadata or source code?

So, after having a look it indeed seems as if https://github.com/google/go-licenses looks at license files in the source code (not metadata) and uses https://github.com/google/licenseclassifier to identify the license / SPDX id from the license file text.

As such, I believe all is fine with ORT's GoMod anaylzer. But we could consider adding https://github.com/google/licenseclassifier as yet another simple scanner implementation.

[fviernau]

@sschuberth @arieltorti. So, shall we convert this ticket to 'adding licenseclassifier as scanner', or close this ticket and open-up a new one?

[sschuberth]

So, shall we convert this ticket to 'adding licenseclassifier as scanner', or close this ticket and open-up a new one?

I'd say we're blocked by google/licenseclassifier#47 for now. Let's see how they react and decide depending on that.

[arieltorti]

I'll give it a try tomorrow; If the scanner already gives license information that should be enough for me.

What would be the process, running ORT analyze and then scan on the output of the analyze ?

And further on, can the reporter use the output of the scanner and the analyzer ? Our intention is to have a report of the licenses used across many microservices, #5620 talks a bit more about it.

[sschuberth]

What would be the process, running ORT analyze and then scan on the output of the analyze ?

Yes, the input to the scanner is the output of the analyzer. See the respective Getting Started section for details.

And further on, can the reporter use the output of the scanner and the analyzer ?

That's implicitly so: Analyzer and scanner do not create separate outputs, but ORT result files are always amended by the tool being run. So the analyzer result is passed to the scanner, which amends its data to the analyzer result, creating a new single output file which contains both the analyzer and the scan result.

[sschuberth]

Based on my earlier comment, I'm closing this as "won't fix" because licenses can already be detected with other scanners, and google/licenseclassifier#47 is still unresolved.