properly handle top-level, key files Licenses vs. other files - [was Scan of eudev-3.2.5.tar.gz returns ambiguous license results]

[DennisClark]

A recent scan of eudev-3.2.5.tar.gz using download url https://github.com/gentoo/eudev/archive/v3.2.5.tar.gz returned the following license results:

lgpl-2.1-plus 78
gpl-2.0-plus 34
lgpl-2.0-plus 3
gpl-2.0 2
public-domain 2
cc0-1.0 1
fsf-unlimited 1
lgpl-2.1 1
mit 1

If you were simply to look at the numbers, it seems that the overall project license is lgpl-2.1-plus; however, this package is acutally an excellent example of where providing the Declared License would be extremely helpful. The project has a COPYING license text for GPL 2.0 at the top level, but the file headers are quite mixed (certainly NOT Clearly Defined). Anyway, the applicable license for the whole project is GPL 2.0 and not LGPL 2.1. This is a good case for identifying a declared license at the project level.

[mjherzog]

I downloaded the package and disagree that it is GPL 2.0 for the whole project. My take is that it is LGPL for the libs - see /src/libudev for example, and GPL for the command binaries which is a completely normal case for Linux apps/utilities.
So focusing on the COPYING file seems like a detour.

[pombredanne]

Thanks!
there are several things at play here:

1. minor license detection bugs but they do not influence overall the summary by much
2. properly handling the notion of top-level, key files vs. other files
3. handling clear or common patterns of multiple licenses combination such as the "classical" GPL for command line tools and examples and LGPL for libraries

[mjherzog]

It would make sense at some point to look for patterns and summarize license/copyright patterns a few levels lower in the codebase sub-directories, perhaps with an option for summarizing results at sub-directory depth - say 1 to 3 levels down. Perhaps a plugin so that you can decide on the summarization after seeing the initial results. This could also be useful for ClearlyDefined license metrics.

[pombredanne]

Fix and merged in develop

[pombredanne]

Actually there are new rules... but we still need to handle the top level part