docs: New documentation (#858)

* Renew placement documentation

* Renew Connectivity docs

* Renew region and partition doc

* Fix deprecated setup-python@v1

* Remove outdated strategies

* docs: Update connectivity documentation

* docs: Improve english

* docs: fix doc refs and initialisms

* docs: update placement documentation

* docs: add note for packing error

* docs: update cell types documentation

* docs: small updates to partition

* docs: Add getting started for Simulations

* docs: Update Simulation intro page

* docs: update simulation

* docs: update nest simulation page

* docs: fix nest simulation docs

* docs: fix configuration usage in gs | remove refs to bsb-plotting

* docs: reshape welcome page and toctrees

* docs: fix refs in regions doc

* fix: bsb commands related to config creation from template.

* fix: bsb simulate command output option. Now is used to specify the output folder.

* docs: update analyze page in gs

* docs: add json recap for simulation gs

* docs: update top-level guide

* docs: improve wording in installation and top level guide.

* docs: improve getting-started_reconstruction.rst.

Make sure the getting-started files match the default json config.
Rewrote top level guide
Improve installation section and dev sections

* docs: Refactor documentations

Remove guides
Add toc for every documentation section
Rewrite getting-started add morphologies and move configurations to subfolder
Separate bsb core classes (Configuration, Scaffold and Storage) from components

* docs: update README

* docs: update GS config templates and line refs

* docs: Fix installations tabs.

fix configurations for simulations

* docs: add targetting doc in simulation page

* docs: fix targetting section

* docs: remove neuromorfo uri scheme ref in gs

* docs: Rewrite simulation. Update workflow figure.

* docs: update placement set

* docs: add connectivity-set page

* docs: Merge examples for accessing storage data.

Move tutorials python script in examples/tutorials
Fix label_cells and repeated_simulations
Minor fixes in interfaces documentation

* docs: Rewrite analyse_spike_results, reorganize configuration section.

* docs: Add example to display morphologies, rewrite project tutorial

* docs: update tutorials

* docs: Minor fixes to tutorial on python projects section

* docs: Add documentation to ConnectionStrategy

Add connect_cells function to Scaffold to mirror place_cells used in Placement.

* docs: Add tutorial on writing components separated from components.rst

* docs: First draft of the tutorial for writing placement strategy

Added utility classes to indicator.py partition.py and _distributions.py

* docs: Fix tutorial placement strategy

Added utility classes to _distributions.py

* docs: Fix some typos and config for nest simulation tutorial.

* docs: separate neuron and nest tutorials.

merge python projects in getting started and move the rest to dev.

* docs: simplify the placement class for the tutorial

* docs: add missing nest config

* docs: add tutorial-template files for NEURON sim | update gs

* docs: Update neuron simulation guide

* docs: Add section for analyze neuron results | update neuron parts of gs

* docs: WIP on guide for neuron simulation

move images in the images folder.
The morphology and python stellate model should be removed from the repo when they will be available on the dbbs models public repo.

* docs: fix naming in neuron gs

* docs: fix ref number in neuron gs

* docs: add section for simulation results

* docs: fix guide neurons json indent and lines

* docs: Add explanation on distribution placement.

fix and comment distrib_placement.py

* docs: Rework guides for simulations

Make analysis more uniform

* docs: simplify writing components tutorial by splitting connection and placement

* docs: Add FAQ for BSB-MUSIC related issue

May be sufficient for dbbs-lab/bsb-nest#13

* docs: fix strategy in yaml gs

* docs: Add figures and rewrite scaffold page.

* docs: Improve documentation on parallelization + minor fixes

* test: add tests for bsb new

split copy_configuration_template into two functions.

* fix: add bsb-json dependencies for bsb project tests

---------

Co-authored-by: filimarc <[email protected]>
Co-authored-by: drodarie <[email protected]>

.gitignore

@@ -23,6 +23,7 @@ htmlcov/
 *.io
 .vscode/
 .idea/
+.vs/
 bsb-*/
 
 

README.md

@@ -25,14 +25,14 @@ Any package in the BSB ecosystem can be installed from PyPI through `pip`. Most
 will want to install the main [bsb](https://pypi.org/project/bsb/) framework:
 
 ```
-pip install "bsb~=4.1"
+pip install "bsb"
 ```
 
 Advanced users looking to control install an unconventional combination of plugins might
 be better of installing just this package, and the desired plugins:
 
 ```
-pip install "bsb-core~=4.1"
+pip install "bsb-core"
 ```
 
 Note that installing `bsb-core` does not come with any plugins installed and the usually

bsb/cli/commands/_commands.py

@@ -3,6 +3,8 @@
 """
 
 import itertools
+import os
+import pathlib
 from uuid import uuid4
 
 import errr
@@ -12,6 +14,7 @@
 from ...core import Scaffold, from_storage
 from ...exceptions import NodeNotFoundError
 from ...option import BsbOption
+from ...reporting import report
 from ...storage import open_storage
 from . import BaseCommand
 
@@ -115,10 +118,10 @@ def _flatten_arr_args(arr):
 
 class MakeConfigCommand(BaseCommand, name="make-config"):
     def handler(self, context):
-        from ...config import copy_template
+        from ...config import copy_configuration_template
 
         args = context.arguments
-        copy_template(args.template, args.output, path=args.path or ())
+        copy_configuration_template(args.template, args.output, path=args.path or ())
 
     def get_options(self):
         return {}
@@ -204,14 +207,20 @@ def handler(self, context):
             for name, sim in extra_simulations.items():
                 if name not in network.simulations and name == sim_name:
                     network.simulations[sim_name] = sim
+        root = pathlib.Path(getattr(context.arguments, "output_folder", "./"))
+        if not root.is_dir() or not os.access(root, os.W_OK):
+            return report(
+                f"Output provided '{root.absolute()}' is not an existing directory with write access.",
+                level=0,
+            )
         try:
             result = network.run_simulation(sim_name)
         except NodeNotFoundError as e:
             append = ", " if len(network.simulations) else ""
             append += ", ".join(f"'{name}'" for name in extra_simulations.keys())
             errr.wrap(type(e), e, append=append)
         else:
-            result.write(getattr(context.arguments, "output", f"{uuid4()}.nio"), "ow")
+            result.write(root / f"{uuid4()}.nio", "ow")
 
     def get_options(self):
         return {
@@ -222,7 +231,7 @@ def get_options(self):
     def add_parser_arguments(self, parser):
         parser.add_argument("network")
         parser.add_argument("simulation")
-        parser.add_argument("-o", "--output")
+        parser.add_argument("-o", "--output_folder")
 
 
 class CacheCommand(BaseCommand, name="cache"):  # pragma: nocover

bsb/cli/commands/_projects.py

@@ -51,7 +51,7 @@ def handler(self, context):
                 input(f"Config filename [network_configuration.{ext}]: ")
                 or f"network_configuration.{ext}"
             )
-        config.copy_template(template, output=root / output)
+        config.copy_configuration_template(template, output=root / output)
         with open(root / "pyproject.toml", "w") as f:
             toml.dump(
                 {

bsb/config/__init__.py

@@ -76,7 +76,14 @@ def get_config_path():
     return [*itertools.chain((os.getcwd(),), env_paths, *plugin_paths.values())]
 
 
-def copy_configuration_template(template, output="network_configuration.json", path=None):
+def get_configuration_template(template, path=None):
+    """
+    Returns the configuration template files matching the provided name.
+
+    :param str template: name of the configuration template
+    :param list path: list of paths to search for configuration templates
+    :rtype: List[str]
+    """
     path = [
         *map(
             os.path.abspath,
@@ -90,7 +97,19 @@ def copy_configuration_template(template, output="network_configuration.json", p
         raise ConfigTemplateNotFoundError(
             "'%template%' not found in config path %path%", template, path
         )
-    copy_file(files[0], output)
+    return files
+
+
+def copy_configuration_template(template, output="network_configuration.json", path=None):
+    """
+    Copy the first configuration template file matching the provided name to the provided
+    output filename.
+
+    :param str template: name of the configuration template
+    :param str output: name of the output file
+    :param list path: list of paths to search for configuration templates
+    """
+    copy_file(get_configuration_template(template, path)[0], output)
 
 
 def format_configuration_content(parser_name: str, config: "Configuration", **kwargs):

bsb/config/_distributions.py

@@ -27,7 +27,9 @@ class Distribution:
     distribution: str = config.attr(
         type=types.in_(_available_distributions), required=True
     )
+    """Name of the scipy.stats distribution function"""
     parameters: dict[str, typing.Any] = config.catch_all(type=types.any_())
+    """parameters to pass to the distribution"""
 
     def __init__(self, **kwargs):
         if self.distribution == "constant":
@@ -42,8 +44,36 @@ def __init__(self, **kwargs):
             )
 
     def draw(self, n):
+        """Draw n random samples from the distribution"""
         return self._distr.rvs(size=n)
 
+    def definition_interval(self, epsilon=0):
+        """
+        Returns the `epsilon` and 1 - `epsilon` values of
+        the distribution Percent point function.
+
+        :param float epsilon: ratio of the interval to ignore
+        """
+        if epsilon < 0 or epsilon > 1:
+            raise ValueError("Epsilon must be between 0 and 1")
+        return self._distr.ppf(epsilon), self._distr.ppf(1 - epsilon)
+
+    def cdf(self, value):
+        """
+        Returns the result of the cumulative distribution function for `value`
+
+        :param float value: value to evaluate
+        """
+        return self._distr.cdf(value)
+
+    def sf(self, value):
+        """
+        Returns the result of the Survival function for `value`
+
+        :param float value: value to evaluate
+        """
+        return self._distr.sf(value)
+
     def __getattr__(self, attr):
         if "_distr" not in self.__dict__:
             raise AttributeError("No underlying _distr found for distribution node.")

bsb/connectivity/detailed/voxel_intersection.py

@@ -17,6 +17,11 @@
 class VoxelIntersection(Intersectional, ConnectionStrategy):
     """
     This strategy finds overlap between voxelized morphologies.
+
+    :param contacts: number or distribution determining the amount of synaptic contacts one cell will form on another
+    :param voxel_pre: the number of voxels into which the morphology will be subdivided.
+    :param voxel_post: the number of voxels into which the morphology will be subdivided.
+    :param favor_cache: choose whether to cache the pre or post morphology.
     """
 
     contacts = config.attr(type=types.distribution(), default=1)

bsb/connectivity/strategy.py

@@ -12,6 +12,7 @@
 from ..mixins import HasDependencies
 from ..profiling import node_meter
 from ..reporting import warn
+from ..storage._chunks import Chunk
 
 if typing.TYPE_CHECKING:
     from ..cell_types import CellType
@@ -82,7 +83,12 @@ def _get_rect_ext(self, chunk_size):
 
 
 class HemitypeCollection:
-    def __init__(self, hemitype, roi):
+    """
+    Class used to iterate over an ``Hemitype`` placement sets within a list of chunks,
+    and over its cell types.
+    """
+
+    def __init__(self, hemitype: Hemitype, roi: typing.List[Chunk]):
         self.hemitype = hemitype
         self.roi = roi
 
@@ -91,6 +97,13 @@ def __iter__(self):
 
     @property
     def placement(self):
+        """
+        List the placement sets for each cell type, filtered according to the class
+        morphology labels and list of chunks.
+
+
+        :rtype: List[bsb.storage.interfaces.PlacementSet]
+        """
         return [
             ct.get_placement_set(
                 chunks=self.roi,
@@ -151,6 +164,17 @@ def __repr__(self):
 
     @abc.abstractmethod
     def connect(self, presyn_collection, postsyn_collection):
+        """
+        Central method of each connection strategy. Given a pair of
+        ``HemitypeCollection`` (one for each connection side), should connect
+        cell population using the scaffold's (available as ``self.scaffold``)
+        :func:`~bsb.core.Scaffold.connect_cells` method.
+
+        :param bsb.connectivity.strategy.HemitypeCollection presyn_collection:
+          presynaptic filtered cell population.
+        :param bsb.connectivity.strategy.HemitypeCollection postsyn_collection:
+          postsynaptic filtered cell population.
+        """
         pass
 
     def get_deps(self):
@@ -162,6 +186,19 @@ def _get_connect_args_from_job(self, pre_roi, post_roi):
         return pre, post
 
     def connect_cells(self, pre_set, post_set, src_locs, dest_locs, tag=None):
+        """
+        Connect cells from a presynaptic placement set to cells of a postsynaptic placement set,
+        and produce a unique name to describe their connectivity set.
+        The description of the hemitype (source or target cell population) `connection location`
+        is stored as a list of 3 ids: the cell index (in the placement set), morphology branch
+        index, and the morphology branch section index.
+        If no morphology is attached to the hemitype, then the morphology indexes can be set to -1.
+
+        :param bsb.storage.interfaces.PlacementSet pre_set: presynaptic placement set
+        :param bsb.storage.interfaces.PlacementSet post_set: postsynaptic placement set
+        :param List[List[int, int, int]] src_locs: list of the presynaptic `connection location`.
+        :param List[List[int, int, int]] dest_locs: list of the postsynaptic `connection location`.
+        """
         names = self.get_output_names(pre_set.cell_type, post_set.cell_type)
         between_msg = f"between {pre_set.cell_type.name} and {post_set.cell_type.name}"
         if len(names) == 0:
@@ -187,10 +224,7 @@ def connect_cells(self, pre_set, post_set, src_locs, dest_locs, tag=None):
             else:
                 name = tag
 
-        cs = self.scaffold.require_connectivity_set(
-            pre_set.cell_type, post_set.cell_type, name
-        )
-        cs.connect(pre_set, post_set, src_locs, dest_locs)
+        self.scaffold.connect_cells(pre_set, post_set, src_locs, dest_locs, name)
 
     def get_region_of_interest(self, chunk):
         """

bsb/core.py

@@ -577,6 +577,24 @@ def get_placement_sets(self) -> typing.List["PlacementSet"]:
         """
         return [cell_type.get_placement_set() for cell_type in self.cell_types.values()]
 
+    def connect_cells(self, pre_set, post_set, src_locs, dest_locs, name):
+        """
+        Connect cells from a presynaptic placement set to cells of a postsynaptic placement set,
+        and into a connectivity set.
+        The description of the hemitype (source or target cell population) connection location
+        is stored as a list of 3 ids: the cell index (in the placement set), morphology branch
+        index, and the morphology branch section index.
+        If no morphology is attached to the hemitype, then the morphology indexes can be set to -1.
+
+        :param bsb.storage.interfaces.PlacementSet pre_set: presynaptic placement set
+        :param bsb.storage.interfaces.PlacementSet post_set: postsynaptic placement set
+        :param List[List[int, int, int]] src_locs: list of the presynaptic `connection location`.
+        :param List[List[int, int, int]] dest_locs: list of the postsynaptic `connection location`.
+        :param str name: Name to give to the `ConnectivitySet`
+        """
+        cs = self.require_connectivity_set(pre_set.cell_type, post_set.cell_type, name)
+        cs.connect(pre_set, post_set, src_locs, dest_locs)
+
     def get_connectivity(
         self, anywhere=None, presynaptic=None, postsynaptic=None, skip=None, only=None
     ) -> typing.List["ConnectivitySet"]:
@@ -778,7 +796,7 @@ def create_job_pool(self, fail_fast=None, quiet=False):
         )
         try:
             # Check whether stdout is a TTY, and that it is larger than 0x0
-            # (e.g. MPI sets it to 0x0 unless an xterm is emulated.
+            # (e.g. MPI sets it to 0x0 unless a xterm is emulated.
             tty = os.isatty(sys.stdout.fileno()) and sum(os.get_terminal_size())
         except Exception:
             tty = False

bsb/mixins.py

@@ -27,7 +27,13 @@ def _all_chunks(iter_):
 
 
 def _queue_connectivity(self, pool: "JobPool"):
-    # Get the queued jobs of all the strategies we depend on.
+    """Get the queued jobs of all the strategies we depend on.
+
+    Parameters
+    ----------
+    param pool : pool where the jobs will be queued
+    type pool: bsb.services.pool.JobPool
+    """
     deps = set(_gutil.ichain(pool.get_submissions_of(strat) for strat in self.get_deps()))
     # Schedule all chunks in 1 job
     pre_chunks = _all_chunks(self.presynaptic.cell_types)

bsb/morphologies/__init__.py

@@ -397,7 +397,7 @@ def bounds(self):
     @property
     def branch_adjacency(self):
         """
-        Return a dictonary containing mapping the id of the branch to its children.
+        Return a dictionary containing mapping the id of the branch to its children.
         """
         idmap = {b: n for n, b in enumerate(self.branches)}
         return {n: list(map(idmap.get, b.children)) for n, b in enumerate(self.branches)}
@@ -529,7 +529,9 @@ def label(self, labels, points=None):
 
     def rotate(self, rotation, center=None):
         """
-        Point rotation
+        Rotate the entire Subtree with respect to the center.
+        The rotation angles are assumed to be in degrees.
+        If the center is not provided, the Subtree will rotate from [0, 0, 0].
 
         :param rotation: Scipy rotation
         :type rotation: Union[scipy.spatial.transform.Rotation, List[float,float,float]]

bsb/morphologies/selector.py

@@ -82,7 +82,7 @@ def pick(self, morphology):
 
 @config.node
 class NeuroMorphoSelector(NameSelector, classmap_entry="from_neuromorpho"):
-    _url = "https://neuromorpho.org/"
+    _url = "http://cng.gmu.edu:8080/neuroMorpho/"  # "https://neuromorpho.org/"
     _meta = "api/neuron/select?q=neuron_name:"
     _files = "dableFiles/"
 

bsb/placement/indicator.py

@@ -46,6 +46,10 @@ def cell_type(self):
     def get_radius(self):
         return self.assert_indication("radius")
 
+    @property
+    def partitions(self):
+        return self._strat.partitions
+
     def use_morphologies(self):
         return bool(self.indication("morphologies"))