You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Documentation Style is enforced by using Doc8 and Doc Contribution Guide is updated accordingly.
300+ Style errors fixed.
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
See CONTRIBUTING.rst for details: https://github.com/nexB/scancode-toolkit/blob/master/CONTRIBUTING.rst
4
+
See `CONTRIBUTING.rst <https://github.com/nexB/scancode-toolkit/blob/master/CONTRIBUTING.rst>`_
5
+
for details.
5
6
6
7
Code layout and conventions
7
8
---------------------------
8
9
9
10
Source code is in ``src/`` Tests are in ``tests/``.
10
11
11
-
There is one Python package for each major feature under ``src/`` and a corresponding directory with the same name under ``tests`` (but this is not a package by design).
12
+
There is one Python package for each major feature under ``src/`` and a corresponding directory
13
+
with the same name under ``tests`` (but this is not a package by design).
12
14
13
-
Each test script is named ``test_XXXX`` and while we love to use ``py.test`` as a test runner, most tests have no dependencies on ``py.test``, only on the ``unittest`` module (with the exception of some command line tests that depend on pytest monkeypatching capabilities.
15
+
Each test script is named ``test_XXXX`` and while we love to use ``py.test`` as a test runner,
16
+
most tests have no dependencies on ``py.test``, only on the ``unittest`` module (with the exception
17
+
of some command line tests that depend on pytest monkeypatching capabilities.
14
18
15
19
When source or tests need data files, we store these in a ``data`` subdirectory.
16
20
17
-
We use PEP8 conventions with a relaxed line length that can be up to 90'ish characters long when needed to keep the code clear and readable.
21
+
We use PEP8 conventions with a relaxed line length that can be up to 90'ish characters long when
22
+
needed to keep the code clear and readable.
18
23
19
-
We store pre-built bundled native binaries in ``bin/`` sub-directories of each ``src/`` packages. These binaries are organized by OS and architecture. This ensure that ScanCode works out of the box either using a checkout or a download, without needing a compiler and toolchain to be installed. The corresponding source code for the pre-built binaries are store in a separate repository at https://github.com/nexB/scancode-thirdparty-src
24
+
We store pre-built bundled native binaries in ``bin/`` sub-directories of each ``src/`` packages.
25
+
These binaries are organized by OS and architecture. This ensure that ScanCode works out of the box
26
+
either using a checkout or a download, without needing a compiler and toolchain to be installed.
27
+
The corresponding source code for the pre-built binaries are store in a separate repository at
28
+
https://github.com/nexB/scancode-thirdparty-src.
20
29
21
-
We store bundled thirdparty components and libraries in the ``thirdparty`` directory. Python libraries are stored as wheels, eventually pre-built if the corresponding wheel is not available in the Pypi repository. Some of these components may be advanced builds with bug fixes or advanced patches.
30
+
We store bundled thirdparty components and libraries in the ``thirdparty`` directory. Python
31
+
libraries are stored as wheels, eventually pre-built if the corresponding wheel is not available
32
+
in the Pypi repository. Some of these components may be advanced builds with bug fixes or advanced
33
+
patches.
22
34
23
-
We write tests, a lot of tests, thousands of tests. Several tests are data-driven and use data files as test input and sometimes data files as test expectation (in this case using either JSON or YAML files). The tests should pass on Linux 64 bits, Windows 32 and 64 bits and on MacOSX 10.6.8 and up. We maintain two CI loops with Travis (Linux) at https://travis-ci.org/nexB/scancode-toolkit and Appveyor (Windows) at https://ci.appveyor.com/project/nexB/scancode-toolkit
35
+
We write tests, a lot of tests, thousands of tests. Several tests are data-driven and use data
36
+
files as test input and sometimes data files as test expectation (in this case using either
37
+
JSON or YAML files). The tests should pass on Linux 64 bits, Windows 32 and 64 bits and on
38
+
MacOSX 10.6.8 and up. We maintain two CI loops with Travis (Linux) at
39
+
https://travis-ci.org/nexB/scancode-toolkit and Appveyor (Windows) at
When finding bugs or adding new features, we add tests. See existing test code for examples.
26
43
@@ -29,39 +46,69 @@ When finding bugs or adding new features, we add tests. See existing test code f
29
46
Running tests
30
47
-------------
31
48
32
-
ScanCode comes with over 13,000 unit tests to ensure detection accuracy and stability across Linux, Windows and macOS OSes: we kinda love tests, do we?
49
+
ScanCode comes with over 13,000 unit tests to ensure detection accuracy and stability across Linux,
50
+
Windows and macOS OSes: we kinda love tests, do we?
33
51
34
-
We use pytest to run the tests: call the ``py.test`` script to run the whole test suite. This is installed by ``pytest`` which is bundled with a ScanCode checkout and installed when you run ``./configure``).
52
+
We use pytest to run the tests: call the ``py.test`` script to run the whole test suite. This is
53
+
installed by ``pytest`` which is bundled with a ScanCode checkout and installed when you
54
+
run ``./configure``).
35
55
36
-
If you are running from a fresh git clone and you run ``./configure`` and then ``source bin/activate`` the ``py.test`` command will be available in your path.
56
+
If you are running from a fresh git clone and you run ``./configure`` and then
57
+
``source bin/activate`` the ``py.test`` command will be available in your path.
37
58
38
-
Alternatively if you have already configured but are not in an activated "virtualenv" the ``py.test`` command is available under ``<root of your checkout>/bin/py.test``
59
+
Alternatively if you have already configured but are not in an activated "virtualenv" the
60
+
``py.test`` command is available under ``<root of your checkout>/bin/py.test``
39
61
40
62
(Note: paths here are for POSIX, but mostly the same applies to Windows)
41
63
42
-
If you have a multiprocessor machine you might want to run the tests in parallel (and faster) For instance: ``py.test -n4`` runs the tests on 4 CPUs. We typically run the tests in verbose mode with ``py.test -vvs -n4``
64
+
If you have a multiprocessor machine you might want to run the tests in parallel (and faster)
65
+
For instance: ``py.test -n4`` runs the tests on 4 CPUs. We typically run the tests in
66
+
verbose mode with ``py.test -vvs -n4``.
43
67
44
-
You can also run a subset of the test suite as shown in the CI configs https://github.com/nexB/scancode-toolkit/blob/develop/appveyor.yml#L6 e,g, ``py.test -n 2 -vvs tests/scancode`` runs only the tests scripts present in the ``tests/scancode`` directory. (You can pass a path to a specific test script file there too).
68
+
You can also run a subset of the test suite as shown in the CI configs
``py.test -n 2 -vvs tests/scancode`` runs only the tests scripts present in the ``tests/scancode``
71
+
directory. (You can pass a path to a specific test script file there too).
45
72
46
-
See also https://docs.pytest.org for details or use the ``py.test -h`` command to show the many other options available.
73
+
See also https://docs.pytest.org for details or use the ``py.test -h`` command to show the many
74
+
other options available.
47
75
48
-
One useful option is to run a select subset of the test functions matching a pattern with the ``-k`` option for instance: ``py.test -vvs -k tcpdump`` would only run test functions that contain the string "tcpdump" in their name or their class name or module name .
76
+
One useful option is to run a select subset of the test functions matching a pattern with the
77
+
``-k`` option for instance: ``py.test -vvs -k tcpdump`` would only run test functions that contain
78
+
the string "tcpdump" in their name or their class name or module name .
49
79
50
-
Another useful option after a test run with some failures is to re-run only the failed tests with the ``--lf`` option for instance: ``py.test -vvs --lf`` would only run only test functions that failed in the previous run.
80
+
Another useful option after a test run with some failures is to re-run only the failed tests with
81
+
the ``--lf`` option for instance: ``py.test -vvs --lf`` would only run only test functions that
82
+
failed in the previous run.
51
83
52
84
pip requirements and the configure script
53
85
-----------------------------------------
54
86
55
-
ScanCode use the ``configure`` and ``configure.bat`` (and ``etc/configure.py`` behind the scenes) scripts to install a `virtualenv <https://virtualenv.pypa.io/en/stable/>`_ , install required packaged dependencies as `pip <https://github.com/pypa/pip>`_ requirements and more configure tasks such that ScanCode can be installed in a self-contained way with no network connectivity required.
87
+
ScanCode use the ``configure`` and ``configure.bat`` (and ``etc/configure.py`` behind the scenes)
88
+
scripts to install a `virtualenv <https://virtualenv.pypa.io/en/stable/>`_ , install required
89
+
packaged dependencies as `pip <https://github.com/pypa/pip>`_ requirements and more configure tasks
90
+
such that ScanCode can be installed in a self-contained way with no network connectivity required.
56
91
57
-
Earlier unreleased versions of ScanCode where using ``buildout`` to install and configure eventually complex dependencies. We had some improvements that were merged in the upstream ``buildout`` to support bootstrapping and installing without a network connection and When we migrated to use ``pip`` and ``wheels`` as new, improved and faster way to install and configure dependencies we missed some of the features of ``buildout`` like the ``recipes``, being able to invoke arbitrary Python or shell scripts after installing packages and have scripts or requirements that are operating system-specific.
92
+
Earlier unreleased versions of ScanCode where using ``buildout`` to install and configure
93
+
eventually complex dependencies. We had some improvements that were merged in the upstream
94
+
``buildout`` to support bootstrapping and installing without a network connection and When we
95
+
migrated to use ``pip`` and ``wheels`` as new, improved and faster way to install and configure
96
+
dependencies we missed some of the features of ``buildout`` like the ``recipes``, being able to
97
+
invoke arbitrary Python or shell scripts after installing packages and have scripts or requirements
98
+
that are operating system-specific.
58
99
59
100
ScanCode requirements and third-party Python libraries
In a somewhat unconventional way, all the required libraries are bundled aka. copied in the repo itself in the thirdparty/ directory. If ScanCode were only a library it would not make sense. But its is first an application and having a well defined frozen set of dependent packages is important for an app. The benefit of this approach (combined with the ``configure`` script) means that a mere checkout of the repository contains everything needed to run ScanCode except for a Python interpreter.
103
+
In a somewhat unconventional way, all the required libraries are bundled aka. copied in the repo
104
+
itself in the thirdparty/ directory. If ScanCode were only a library it would not make sense. But
105
+
its is first an application and having a well defined frozen set of dependent packages is important
106
+
for an app. The benefit of this approach (combined with the ``configure`` script) means that a mere
107
+
checkout of the repository contains everything needed to run ScanCode except for a
108
+
Python interpreter.
63
109
64
110
Using ScanCode as a Python library
65
111
----------------------------------
66
112
67
-
ScanCode can be used alright as a Python library and is available as as a Python wheel in Pypi and installed with ``pip install scancode-toolkit``.
113
+
ScanCode can be used alright as a Python library and is available as as a Python wheel in Pypi and
@@ -4,23 +4,130 @@ Contributing to the Documentation
4
4
Continious Integration
5
5
----------------------
6
6
7
-
The documentations are checked on every new commit through Travis-CI, so that common errors are avoided and documentation standards are enforced. Travis-CI presently checks for these 3 aspects of the documentation
7
+
The documentations are checked on every new commit through Travis-CI, so that common errors are
8
+
avoided and documentation standards are enforced. Travis-CI presently checks for these 3 aspects
9
+
of the documentation :
8
10
9
-
1. Successful Builds (By using sphinx-build)
10
-
2. No Broken Links (By Using link-check)
11
-
3. Linting Errors (By Using ``restructuredtext-lint``)
11
+
1. Successful Builds (By using ``sphinx-build``)
12
+
2. No Broken Links (By Using ``link-check``)
13
+
3. Linting Errors (By Using ``Doc8``)
12
14
13
-
Style Checks
14
-
------------
15
+
Style Checks Using ``Doc8``
16
+
---------------------------
15
17
16
-
1. Internal Links
18
+
How To Run Style Tests
19
+
^^^^^^^^^^^^^^^^^^^^^^
17
20
18
-
Using ``:ref:`` is advised over standard reStructuredText links to sections (like ```Section title`_``) because it works across files, when section headings are changed, will raise warnings if incorrect, and works for all builders that support cross-references. However, external links are created by using the standard ```Section title`_`` method.
If a section/file has to be repeated somewhere else, do not write the exact same section/file twice. Use ``.. include: ../README.rst`` instead. Here, ``../`` refers to the documentation root, so file location can be used accordingly. This enables us to link documents from other upstream folders.
25
+
.. note::
23
26
24
-
3. Using ``:ref:`` only when necessary
27
+
Only the scancode-toolkit documentation style standards are enforced presently.
25
28
26
-
Use ``:ref:`` to create internal links only when needed, i.e. it is referanced somewhere. Do not create referances for all the sections and then only referance some of them, because this created unnecessary referances. This also generates ERROR in ``restructuredtext-lint``.
0 commit comments