Cross check that all licenses on https://www.openhub.net/licenses are properly detected

[pombredanne]

See https://www.openhub.net/licenses
Several are likely to be variants of BSD and MIT or existing license and will be detected as such.
Some oddities may require a new license addition

[yudhik11]

can I take up this issue?

[pombredanne]

@yudhik11 That would be very nice of you. Thanks!

[yudhik11]

@pombredanne do I need to scrape those license online and check whether they are getting detected OR in the repo there are many licenses I should test them from there.And also the license data is present in src/licensedcode/data/licenses/ and tests/licensedcode/data/licenses/ and to me the thing that confused me was there are more licenses in tests then in src data set.
So with LICENSE data should I use.

[pombredanne]

@yudhik11 so the stepos would be:

1. collect the texts online
2. make license tests from each of these in tests/licensedcode/data/licenses/ ....
3. also test by hand the detection
4. if we do not have 100% detection then either:
   4.1 add new rules
   4.2 add a new license if this is a new license

And also the license data is present in src/licensedcode/data/licenses/ and tests/licensedcode/data/licenses/ and to me the thing that confused me was there are more licenses in tests then in src data set.

• src/licensedcode/data/licenses/ contains the actual real license files used as master data and for detection
• tests/licensedcode/data/licenses/ contains test files that we use to verify that the detection works correctly and are used only when running the py.test test suite

[yudhik11]

@pombredanne what do you meant by [[making license test from each of these in (Addr)]]
and also what is testing by hand the detection

[pombredanne]

• testing by hand: ./scancode --license --diag --license-text --format json-pp
• license tests: see the automated tests in tests/licensedcode/data/licenses/ where each pair of yml + other file with the same base name become a unit test dynamically created by this code https://github.com/nexB/scancode-toolkit/blob/develop/tests/licensedcode/test_detection_datadriven.py

[yudhik11]

@pombredanne it will be nice if you can assign me a small work or any small issue because right now i am trying this from past two weeks and I keep on moving in circle and cannot proceed.

Anything will be much appreciated.

[pombredanne]

@yudhik11 ok :) Let me break thing down in smaller issue that would be bite sized!

[yudhik11]

@pombredanne I am still waiting for your bite-sized issues :)

[pombredanne]

@yudhik11 Can you start by creating a JSON file that lists all the urls for the license page links and names in the 18 pages at https://www.openhub.net/licenses
This should look like:

[
    {"openhub_url": "https://www.openhub.net/licenses/Artistic_License_2_0", "name": "Artistic License 2.0"},
    {"openhub_url": "https://www.openhub.net/licenses/Beerware", "name": "Beerware"},
   .....
]