Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Substitute version placeholders in schema #287

Draft
wants to merge 4 commits into
base: 0.3-dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ _build
.python-version
.vscode
tests/__pycache__
_readthedocs
3 changes: 2 additions & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ build:
# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/conf.py
builder: dirhtml

# If using Sphinx, optionally build your docs in additional formats such as PDF
# formats:
Expand All @@ -26,4 +27,4 @@ sphinx:
# Optionally declare the Python requirements required to build your docs
python:
install:
- requirements: requirements.txt
- requirements: requirements.txt
10 changes: 5 additions & 5 deletions docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
BUILDDIR = _readthedocs

# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
Expand Down Expand Up @@ -56,9 +56,9 @@ html:

.PHONY: dirhtml
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

.PHONY: singlehtml
singlehtml:
Expand Down Expand Up @@ -226,6 +226,6 @@ dummy:

.PHONY: autobuild
autobuild:
sphinx-autobuild -nW -q -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
sphinx-autobuild --re-ignore='docs/_readthedocs/*' -nW -q -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Auto-build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
@echo "Auto-build finished. The HTML pages are in $(BUILDDIR)/html."
72 changes: 70 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
import json
import os
import shutil

# -- General configuration ------------------------------------------------

Expand Down Expand Up @@ -98,7 +101,7 @@
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store','_static/docson']
exclude_patterns = ['_build', '_readthedocs', 'Thumbs.db', '.DS_Store','_static/docson']

# The reST default role (used for this markup: `text`) to use for all
# documents.
Expand Down Expand Up @@ -369,4 +372,69 @@

linkcheck_ignore = [
'https://linux.die.net/man/3/libuuid', # 403 Client Error: Forbidden for url
]
]

# Substitute branch name placeholders in schema.

def create_directory(path):
output_dir = os.path.dirname(path)
os.makedirs(output_dir, exist_ok=True)

def replace_substring_in_json(file_path, search_substring, replace_string, output_path=None):
# Read the JSON file
with open(file_path, 'r') as file:
data = json.load(file)

# Recursively search and replace the substring in the JSON data
_replace_substring_in_json(data, search_substring, replace_string)

# Set output path
if output_path is None:
output_path = file_path

# Create the directory if it does not exist
create_directory(output_path)

# Write the modified JSON data to the output file
with open(output_path, 'w') as file:
json.dump(data, file, indent=4)

def _replace_substring_in_json(data, search_substring, replace_string):
if isinstance(data, dict):
for key, value in data.items():
if isinstance(value, str):
data[key] = value.replace(search_substring, replace_string)
else:
_replace_substring_in_json(value, search_substring, replace_string)
elif isinstance(data, list):
for i, item in enumerate(data):
if isinstance(item, str):
data[i] = item.replace(search_substring, replace_string)
else:
_replace_substring_in_json(item, search_substring, replace_string)


def setup(app):
# Connect handlers to events
app.connect('config-inited', config_inited)
app.connect('env-before-read-docs', env_before_read_docs)
app.connect('build-finished', build_finished)


def config_inited(app, config):
shutil.copytree('../schema', '../.temp', dirs_exist_ok=True)

rtd_version = os.getenv('READTHEDOCS_VERSION')

# Replace {{version}} placeholders
if rtd_version is not None:
replace_substring_in_json('../.temp/network-schema.json', '{{version}}', rtd_version)


def env_before_read_docs(app, env, docnames):
create_directory('_readthedocs/html/')
shutil.copyfile('../.temp/network-schema.json', '_readthedocs/html/network-schema.json')


def build_finished(app, exception):
shutil.rmtree('../.temp')
2 changes: 1 addition & 1 deletion docs/guidance/publication.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ libcoveofds jsontogeojson network-package.json nodes.geojson spans.geojson
To convert data to CSV format:

- [Install Flatten Tool](https://flatten-tool.readthedocs.io/en/latest/getting-started/#getting-started)
- Download the [network schema](../../schema/network-schema.json)
- Download the [network schema](../_readthedocs/html/network-schema.json)
- If your data is a [JSON Lines file](../reference/publication_formats/json.md#streaming-option), segment it into appropriately sized [network packages](../reference/publication_formats/json.md#small-files-and-api-responses-option)
- Run the following command for each network package:

Expand Down
6 changes: 3 additions & 3 deletions docs/primer/scopeandkeyconcepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ In a fully integrated model, one actor takes on all three roles, whilst in open

OFDS defines a network as:

```{jsoninclude-quote} ../../schema/network-schema.json
```{jsoninclude-quote} ../_readthedocs/html/network-schema.json
:jsonpointer: /description
```

Expand All @@ -72,7 +72,7 @@ OFDS defines a network as:

OFDS defines a node as:

```{jsoninclude-quote} ../../schema/network-schema.json
```{jsoninclude-quote} ../_readthedocs/html/network-schema.json
:jsonpointer: /$defs/Node/description
```

Expand All @@ -84,7 +84,7 @@ For more information about nodes, see the [Node reference](../reference/schema.m

OFDS defines a span as:

```{jsoninclude-quote} ../../schema/network-schema.json
```{jsoninclude-quote} ../_readthedocs/html/network-schema.json
:jsonpointer: /$defs/Span/description
```

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/identifiers.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ To ensure that an organisation's `.identifier` is globally unique, it has two co

:::{tab-item} Schema

```{jsonschema} ../../schema/network-schema.json
```{jsonschema} ../_readthedocs/html/network-schema.json
:pointer: /$defs/Organisation
:include: identifier/scheme,identifier/id
```
Expand Down
26 changes: 13 additions & 13 deletions docs/reference/publication_formats/csv.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/networks.csv) or a [blank template](../../../examples/csv/template/networks.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,name,website,publisher,publicationDate,collectionDate,crs,accuracy,accuracyDetails,language
:nocrossref:
```
Expand All @@ -105,7 +105,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/nodes.csv) or a [blank template](../../../examples/csv/template/nodes.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,nodes/0/id,nodes/0/name,nodes/0/phase,nodes/0/status,nodes/0/location,nodes/0/address,nodes/0/type,nodes/0/accessPoint,nodes/0/power,nodes/0/technologies,nodes/0/physicalInfrastructureProvider
:collapse: nodes/0/location
:nocrossref:
Expand All @@ -119,7 +119,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/nodes_internationalConnections.csv) or a [blank template](../../../examples/csv/template/nodes_internationalConnections.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,nodes/0/id,nodes/0/internationalConnections/0/streetAddress,nodes/0/internationalConnections/0/locality,nodes/0/internationalConnections/0/region,nodes/0/internationalConnections/0/postalCode,nodes/0/internationalConnections/0/country
:nocrossref:
```
Expand All @@ -132,7 +132,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/nodes_networkProviders.csv) or a [blank template](../../../examples/csv/template/nodes_networkProviders.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,nodes/0/id,nodes/0/networkProviders/0/id,nodes/0/networkProviders/0/name
:nocrossref:
```
Expand All @@ -146,7 +146,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/spans.csv) or a [blank template](../../../examples/csv/template/spans.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,spans/0/id,spans/0/name,spans/0/phase,spans/0/status,spans/0/readyForServiceDate,spans/0/start,spans/0/end,spans/0/directed,spans/0/route,spans/0/physicalInfrastructureProvider,spans/0/supplier,spans/0/transmissionMedium,spans/0/deployment,spans/0/deploymentDetails,spans/0/darkFibre,spans/0/fibreType,spans/0/fibreTypeDetails,spans/0/fibreCount,spans/0/fibreLength,spans/0/technologies,spans/0/capacity,spans/0/capacityDetails,spans/0/countries
:collapse: spans/0/route
:nocrossref:
Expand All @@ -160,7 +160,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/spans_networkProviders.csv) or a [blank template](../../../examples/csv/template/spans_networkProviders.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,spans/0/id,spans/0/networkProviders/0/id,spans/0/networkProviders/0/name
:nocrossref:
```
Expand All @@ -174,7 +174,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/phases.csv) or a [blank template](../../../examples/csv/template/phases.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,phases/0/id,phases/0/name,phases/0/description
:nocrossref:
```
Expand All @@ -187,7 +187,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/phases_funders.csv) or a [blank template](../../../examples/csv/template/phases_funders.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,phases/0/id,phases/0/funders/0/id,phases/0/funders/0/name
:nocrossref:
```
Expand All @@ -200,7 +200,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/organisations.csv) or a [blank template](../../../examples/csv/template/organisations.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,organisations/0/id,organisations/0/name,organisations/0/identifier,organisations/0/country,organisations/0/roles,organisations/0/roleDetails,organisations/0/website,organisations/0/logo
:nocrossref:
```
Expand All @@ -215,7 +215,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/contracts.csv) or a [blank template](../../../examples/csv/template/contracts.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,contracts/0/id,contracts/0/title,contracts/0/description,contracts/0/type,contracts/0/value,contracts/0/dateSigned
:nocrossref:
```
Expand All @@ -228,7 +228,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/contracts_documents.csv) or a [blank template](../../../examples/csv/template/contracts_documents.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,contracts/0/id,contracts/0/documents/0/title,contracts/0/documents/0/description,contracts/0/documents/0/url,contracts/0/documents/0/format
:nocrossref:
```
Expand All @@ -241,7 +241,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/contracts_relatedPhases.csv) or a [blank template](../../../examples/csv/template/contracts_relatedPhases.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,contracts/0/id,contracts/0/relatedPhases/0/id,contracts/0/relatedPhases/0/name
:nocrossref:
```
Expand All @@ -254,7 +254,7 @@ This table is related to the following tables:

The fields in this table are listed below. You can also download an [example CSV file](../../../examples/csv/links.csv) or a [blank template](../../../examples/csv/template/links.csv) for this table.

```{jsonschema} ../../../schema/network-schema.json
```{jsonschema} ../../_readthedocs/html/network-schema.json
:include: id,links/0/href,links/0/rel
:nocrossref:
```
Loading
Loading