|
1 | 1 | VulnerableCode |
2 | | -============== |
3 | | - |
4 | | -|Build Status| |License| |Python 3.8| |stability-wip| |Gitter chat| |PRs |
5 | | -Welcome| |
6 | | - |
7 | | -.. image:: README.gif |
8 | | - |
9 | | - |
10 | | -The What |
11 | | --------- |
12 | | - |
13 | | -VulnerableCode is a FOSS database of vulnerabilities and the FOSS |
14 | | -packages they impact. It is made by the FOSS community to improve and |
15 | | -secure the open source software ecosystem. |
16 | | - |
17 | | -The Why |
18 | | -------- |
19 | | - |
20 | | -The existing solutions are commercial proprietary vulnerability |
21 | | -databases, which in itself does not make sense because the data is about |
22 | | -FOSS. |
23 | | - |
24 | | -National Vulnerability Database which is the primary data source for all |
25 | | -things security, is not particularly catered to address FOSS security |
26 | | -issues, because: |
27 | | - |
28 | | -1. It predates the explosion of FOSS software usage |
29 | | -2. It's data format reflects a commercial vendor-centric point of view, |
30 | | - this is due to the usage of |
31 | | - `CPE <https://nvd.nist.gov/products/cpe>`__ to map vulnerabilities |
32 | | - and the packages. |
33 | | -3. CPEs are just not designed to map FOSS to vulnerabilities owing to |
34 | | - their vendor-product centric semantics. This makes it really hard to |
35 | | - answer the fundamental question "Is package foo vulnerable to |
36 | | - vulnerability bar?" |
37 | | - |
38 | | - |
39 | | -The How |
40 | | -------- |
41 | | - |
42 | | -VulnerableCode independently aggregates many software vulnerability data |
43 | | -sources that can easily be recreated in a decentralized fashion. These |
44 | | -data sources include security advisories published by distros, package |
45 | | -managers, etc. Due to this, the data obtained is not generalized to apply |
46 | | -for other ecosystems. This increases the accuracy as the same version of |
47 | | -a package across different distros may or may not be vulnerable to some |
48 | | -vulnerability. |
49 | | - |
50 | | -The packages are identified using |
51 | | -`PURL <https://github.com/package-url/purl-spec>`__ rather than CPEs. |
52 | | -This makes it really easy to answer questions like "Is package foo |
53 | | -vulnerable to vulnerability bar ? ". |
54 | | - |
55 | | -The web interface enables community curation of data by enabling |
56 | | -the addition of new packages, vulnerabilities, and modifying the |
57 | | -relationships between them as shown in GIF. Along with the web interface |
58 | | -the API allows seamless consumption of the data. |
59 | | - |
60 | | -We also plan to mine for vulnerabilities which didn't receive any |
61 | | -exposure due to various reasons like but not limited to the complicated |
62 | | -procedure to receive CVE ID or not able to classify a bug as a security |
63 | | -compromise. |
64 | | - |
65 | | -Check VulnerableCode at `Open Source Summit 2020 |
66 | | -<https://ossna2020.sched.com/event/c46p/why-is-there-no-free-software-vulnerability-database-philippe-ombredanne-aboutcodeorg-and-nexb-inc-michael-herzog-nexb-inc>`__ |
67 | | - |
68 | | -Setting up VulnerableCode |
69 | | -------------------------- |
70 | | - |
71 | | -Clone the source code: |
72 | | - |
73 | | -:: |
74 | | - |
75 | | - git clone https://github.com/nexB/vulnerablecode.git |
76 | | - cd vulnerablecode |
77 | | - |
78 | | -Using Docker Compose |
79 | | -~~~~~~~~~~~~~~~~~~~~ |
80 | | - |
81 | | -An easy way to set up VulnerableCode is with docker containers and |
82 | | -docker compose. For this you need to have the following installed. - |
83 | | -Docker Engine. Find instructions to install it |
84 | | -`here <https://docs.docker.com/get-docker/>`__ - Docker Compose. Find |
85 | | -instructions to install it |
86 | | -`here <https://docs.docker.com/compose/install/#install-compose>`__ |
87 | | - |
88 | | -Use ``sudo docker-compose up`` to start VulnerableCode. Access |
89 | | -VulnerableCode at http://localhost:8000/ or at http://127.0.0.1:8000/ . |
90 | | - |
91 | | -Use ``sudo docker-compose exec web bash`` to access the VulnerableCode |
92 | | -container. From here you can access ``manage.py`` and run management |
93 | | -commands to import data as specified below. |
94 | | - |
95 | | -Without Docker Compose |
96 | | -~~~~~~~~~~~~~~~~~~~~~~ |
97 | | - |
98 | | -**System requirements** |
99 | | - |
100 | | -- Python 3.8+ |
101 | | -- PostgreSQL 9+ |
102 | | -- Compiler toolchain and development files for Python and PostgreSQL |
103 | | - |
104 | | -On Debian-based distros, these can be installed with |
105 | | -``sudo apt install python3-venv python3-dev postgresql libpq-dev build-essential``. |
106 | | - |
107 | | -**Database configuration** - Create a user named ``vulnerablecode``. Use |
108 | | -``vulnerablecode`` as password when prompted: |
109 | | -``sudo -u postgres createuser --no-createrole --no-superuser --login --inherit --createdb --pwprompt vulnerablecode`` |
110 | | - |
111 | | -- Create a databased named ``vulnerablecode``: |
112 | | - ``createdb --encoding=utf-8 --owner=vulnerablecode --user=vulnerablecode --password --host=localhost --port=5432 vulnerablecode`` |
113 | | - |
114 | | -**Application dependencies** |
115 | | - |
116 | | -Create a virtualenv, install dependencies, and run the database |
117 | | -migrations: |
118 | | - |
119 | | -:: |
120 | | - |
121 | | - python3 -m venv venv |
122 | | - source venv/bin/activate |
123 | | - pip install -r requirements.txt |
124 | | - DJANGO_DEV=1 python manage.py migrate |
125 | | - |
126 | | -The environment variable ``DJANGO_DEV`` is used to load settings |
127 | | -suitable for development, defined in ``vulnerablecode/dev.py``. If you |
128 | | -don't want to type it every time use ``export DJANGO_DEV=1`` instead. |
129 | | - |
130 | | -When not running in development mode, an environment variable named |
131 | | -``SECRET_KEY`` needs to be set. The recommended way to generate this key |
132 | | -is to use the code Django includes for this purpose: |
133 | | -``SECRET_KEY=$(python -c "from django.core.management import utils; print(utils.get_random_secret_key())")``. |
134 | | - |
135 | | -Tests |
136 | | ------ |
137 | | - |
138 | | -:: |
139 | | - |
140 | | - pycodestyle --exclude=migrations,settings.py,venv,lib_oval.py,test_ubuntu.py,test_suse.py,test_data_source.py --max-line-length=100 . |
141 | | - DJANGO_DEV=1 pytest |
142 | | - |
143 | | -Data import |
144 | | ------------ |
145 | | - |
146 | | -:: |
147 | | - |
148 | | - DJANGO_DEV=1 python manage.py import --all |
149 | | - |
150 | | -If you want to run the import periodically, you can use a systemd timer: |
151 | | - |
152 | | -:: |
153 | | - |
154 | | - $ cat ~/.config/systemd/user/vulnerablecode.service |
155 | | - |
156 | | - [Unit] |
157 | | - Description=Update vulnerability database |
158 | | - |
159 | | - [Service] |
160 | | - Type=oneshot |
161 | | - Environment="DJANGO_DEV=1" |
162 | | - ExecStart=/path/to/venv/bin/python /path/to/vulnerablecode/manage.py import --all |
163 | | - |
164 | | - $ cat ~/.config/systemd/user/vulnerablecode.timer |
165 | | - |
166 | | - [Unit] |
167 | | - Description=Periodically update vulnerability database |
168 | | - |
169 | | - [Timer] |
170 | | - OnCalendar=daily |
171 | | - |
172 | | - [Install] |
173 | | - WantedBy=multi-user.target |
174 | | - |
175 | | -Start it with |
176 | | - |
177 | | -:: |
178 | | - |
179 | | - systemctl --user daemon-reload && systemctl --user start vulnerablecode.timer |
180 | | - |
181 | | -API |
182 | | ---- |
183 | | - |
184 | | -Start the webserver |
185 | | - |
186 | | -:: |
187 | | - |
188 | | - DJANGO_DEV=1 python manage.py runserver |
189 | | - |
190 | | -In your browser access: |
191 | | - |
192 | | -:: |
193 | | - |
194 | | - http://127.0.0.1:8000/api/docs |
195 | | - |
196 | | -For full documentation about API endpoints. |
197 | | - |
198 | | -.. |Build Status| image:: https://travis-ci.org/nexB/vulnerablecode.svg?branch=develop |
199 | | - :target: https://travis-ci.org/nexB/vulnerablecode |
200 | | -.. |License| image:: https://img.shields.io/badge/License-Apache%202.0-blue.svg |
201 | | - :target: https://opensource.org/licenses/Apache-2.0 |
202 | | -.. |Python 3.8| image:: https://img.shields.io/badge/python-3.8-blue.svg |
203 | | - :target: https://www.python.org/downloads/release/python-360/ |
204 | | -.. |stability-wip| image:: https://img.shields.io/badge/stability-work_in_progress-lightgrey.svg |
205 | | -.. |Gitter chat| image:: https://badges.gitter.im/gitterHQ/gitter.png |
206 | | - :target: https://gitter.im/aboutcode-org/vulnerablecode |
207 | | -.. |PRs Welcome| image:: https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square |
208 | | - :target: http://makeapullrequest.com |
0 commit comments