Skip to content

Commit

Permalink
docs: Improve documentation on parallelization + minor fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
drodarie committed Dec 8, 2024
1 parent 90fbd3f commit 43991f2
Show file tree
Hide file tree
Showing 11 changed files with 82 additions and 46 deletions.
2 changes: 1 addition & 1 deletion docs/config/parsers/json.rst
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ In this document the deepest JSON path is ``/walk/down/the/path``.

.. warning::

Pay attention to the initial ``/`` of the reference clause! Without it, you're making
Pay attention to the initial ``/`` of the reference clause! Without it, you are making
a reference relative to the current position. With an initial ``/`` you make a
reference absolute to the root of the document.

Expand Down
4 changes: 2 additions & 2 deletions docs/config/types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -166,9 +166,9 @@ After the configuration has been cast all nodes are visited to check if they are
reference and if so the value from elsewhere in the configuration is retrieved. The
original string from the configuration is also stored in ``node.<ref>_reference``.

After the configuration is loaded it's possible to either give a new reference key
After the configuration is loaded it is possible to either give a new reference key
(usually a string) or a new reference value. In most cases the configuration will
automatically detect what you're passing into the reference:
automatically detect what you are passing into the reference:

.. code-block::
Expand Down
2 changes: 1 addition & 1 deletion docs/connectivity/defining.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ This snippet would connect only the cells of ``type_A`` that are labelled with e
Specifying subcellular regions using morphology labels
======================================================
You can also specify which regions on a morphology you're interested in connecting. By
You can also specify which regions on a morphology you are interested in connecting. By
default axodendritic contacts are enabled, but by specifying different :guilabel:`morphology_labels`
you can alter this behavior. This example lets you form dendrodendritic contacts:
Expand Down
65 changes: 59 additions & 6 deletions docs/core/job-distribution.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
BSB parallelization
===================
Workflow parallelization
========================

The BSB includes a full MPI interface to distribute the reconstruction jobs across the cores
allocated to the reconstruction and simulation of your networks.
The BSB includes a full `MPI` interface to distribute the reconstruction jobs across the cores
allocated to the reconstruction and simulation of your networks. You would have to install the
MPI library and the BSB interface to benefit from this feature (see the corresponding section in
the :doc:`/getting-started/installation`)

Let us see how the jobs are created:

Job parallelization
-------------------

Let us see how the BSB jobs are created:

The BSB decomposes your `Scaffold` topology (`Region` and `Partition`) into a list of
:class:`Chunk <.storage._chunks.Chunk>` (parallelepipeds) of same size (and volume). The idea
Expand All @@ -15,4 +21,51 @@ jobs that happen after the creation of the topology follow this strategy.
The size and shape of the `Chunk` have a strong impact on the efficiency of the BSB parallelization.
Indeed, having a lot of very small `Chunk` would result in a lot of reading and writing in
the `Storage`. On the other hand, a short list of large `Chunk` will take longer to be completed
by each process, as bigger volumes imply a larger amount of constrains to fulfill.
by each process, as bigger volumes imply a larger amount of constrains to fulfill.

The Chunks' dimensions in µm can be set in the `Configuration` in the ``Network`` node:

.. tab-set-code::

.. code-block:: json
"network": {
"x": 200.0,
"y": 200.0,
"z": 200.0,
"chunk_size": [50, 20, 10]
}
.. code-block:: yaml
network:
x: 200
y: 200
z: 200
chunk_size:
- 50
- 20
- 10
.. code-block:: python
config.network.x = 200.0
config.network.y = 200.0
config.network.z = 200.0
config.network.chunk_size = [50, 20, 10]
As you can see the dimensions of the Chunks can differ according to each dimension.

Running the BSB in parallel
---------------------------

You can run any bsb command or python scripts with MPI with the ``mpirun`` command,
specifying the number of core to allocate:

.. code-block:: bash
# run the BSB reconstruction with 5 cores
mpirun -n 5 bsb compile my-config.json -v 3
# run a python script in parallel with 4 cores
mpirun -n 4 python my-script.py
4 changes: 3 additions & 1 deletion docs/dev/guidelines.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ If you need to create an issue on Github, please provide as much context as poss

Fork and create a Pull Request
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you are not part of the BSB maintainers, you should fork the bsb repository on your own account to modify the code.
If you are not part of the BSB maintainers, you should
`fork <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo>`_
the bsb repository on your own account to modify the code.
If you introduce new features to BSB, please provide the associated documentation (docstrings or general documentation),
and unittests. We are trying to improve the coverage for both and would appreciate greatly your contribution.

Expand Down
30 changes: 0 additions & 30 deletions docs/dev/projects.rst
Original file line number Diff line number Diff line change
Expand Up @@ -64,33 +64,3 @@ For each file to link, you need to provide a list of 3 parameters:
Links in ``tools.bsb.links`` are active for all models in your project! It's better to
specify them on a per model basis using the ``tools.bsb.links."my_model_name.hdf5"``
section.


.. rubric:: Next steps:

.. grid:: 1 1 1 2
:gutter: 1

.. grid-item-card:: :octicon:`tools;1em;sd-text-warning` Make custom components
:link: components
:link-type: ref

Learn how to write your own components to e.g. place or connect cells.

.. grid-item-card:: :octicon:`tools;1em;sd-text-warning` Command-Line Interface
:link: cli-guide
:link-type: ref

Familiarize yourself with BSB's CLI.

.. grid-item-card:: :octicon:`gear;1em;sd-text-warning` Learn about components
:link: main-components
:link-type: ref

Explore more about the main components.

.. grid-item-card:: :octicon:`device-camera-video;1em;sd-text-warning` Examples
:link: examples
:link-type: ref

Explore more advanced examples
2 changes: 1 addition & 1 deletion docs/examples/atlas/atlas_placement.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ the CCFv3 mouse brain annotation atlas volume from the Allen Institute website.
Use :guilabel:`mask_source` to provide your own nrrd annotation volume.

The :guilabel:`struct_name` refers to the Allen mouse brain region acronym or name.
You can also replace that with :guilabel:`struct_id`, if you're using the numeric identifiers.
You can also replace that with :guilabel:`struct_id`, if you are using the numeric identifiers.
You can find the ids, acronyms and names in the Allen Brain Atlas brain region hierarchy file.

If we now place our ``my_cell`` in the ``declive``, it will be placed with a fixed
Expand Down
11 changes: 11 additions & 0 deletions docs/getting-started/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,17 @@ To then install the BSB with parallel MPI support:
pip install bsb[parallel]
You can run any bsb command or python scripts with MPI with the ``mpirun`` command,
specifying the number of core to allocate:

.. code-block:: bash
# run the BSB reconstruction with 5 cores
mpirun -n 5 bsb compile my-config.json -v 3
# run a python script in parallel with 4 cores
mpirun -n 4 python my-script.py
Simulator backends
==================

Expand Down
4 changes: 2 additions & 2 deletions docs/morphologies/intro.rst
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ morphology:
:lines: 8-17
:language: python

Once you're done with the morphology you can save it again:
Once you are done with the morphology you can save it again:

.. literalinclude:: ../../examples/morphologies/usage.py
:lines: 19
Expand Down Expand Up @@ -534,7 +534,7 @@ morphology, and returning any other values would be an error.

If you need to break out of the morphologies that were handed to you, morphology
distributors are also allowed to return their own :class:`~.morphologies.MorphologySet`.
Since you're free to pass any list of morphology loaders to create a morphology set, you
Since you are free to pass any list of morphology loaders to create a morphology set, you
can put and assign any morphology you like.

.. tip::
Expand Down
2 changes: 1 addition & 1 deletion docs/morphologies/morphology-set.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Every time a morphology is loaded, it has to be read from disk and pieced togeth
use soft caching, upon loading a morphology it is kept in cache and each time it is
re-used a copy of the cached morphology is created. This means that the storage only has
to be read once per morphology, but additional memory is used for each unique morphology
in the set. If you're iterating, the soft cache is cleared immediately after the iteration
in the set. If you are iterating, the soft cache is cleared immediately after the iteration
stops. Soft caching is available by passing ``cache=True`` to
:meth:`~.morphologies.MorphologySet.iter_morphologies`:

Expand Down
2 changes: 1 addition & 1 deletion docs/simulation/neuron.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ NEURON is a simulator tool used in the computational neuroscience community to m
of multicomportamental neurons.

This simulator must be installed as a dependency when installing the `bsb-neuron <https://github.com/dbbs-lab/bsb-neuron>`_
package. If you're installing NEURON separately, please refer to the official installation
package. If you are installing NEURON separately, please refer to the official installation
`documentation <https://nrn.readthedocs.io/en/8.2.6/install/install.html>`_ for guidance.

The simulation block contains all the necessary components to run the simulation.
Expand Down

0 comments on commit 43991f2

Please sign in to comment.