Skip to content

Commit

Permalink
Rename to Open Brain Platform/Institute
Browse files Browse the repository at this point in the history
  • Loading branch information
GianlucaFicarelli committed Jan 14, 2025
1 parent 55f5180 commit e6c78a0
Show file tree
Hide file tree
Showing 8 changed files with 22 additions and 20 deletions.
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# **Blue Brain Open Platform** - Documents
# **Open Brain Platform** - Documents

This is a work-in-progress repository containing:

* the [Architecture Documentation](docs/) of the **Blue Brain Open Platform**
* the [Architecture Documentation](docs/) of the **Open Brain Platform**
* the review documents for the [AWS Well-Architected Framework](waf)

For convenience, the file format utilized for the documentation is [Markdown](https://en.wikipedia.org/wiki/Markdown), and for the diagrams is Scalable Vector Graphics (SVG) made with [Draw.io](https://github.com/jgraph/drawio). The plan would be to either publish a GitHub Page at some point, or to generate a proper documentation via other means.
Expand All @@ -12,7 +12,7 @@ One of the main reasons behind choosing the Markdown + Draw.io formats, is the c

**To edit any of the files, including the diagrams, you must simply access the following link to open VSCode:**

[![Edit repository with Visual Studio Code](.vscode/resources/edit_in_vscode.drawio.svg)](https://github.dev/BlueBrain/platform-docs)
[![Edit repository with Visual Studio Code](.vscode/resources/edit_in_vscode.drawio.svg)](https://github.dev/openbraininstitute/platform-docs)

The repository contains a `.vscode` folder with a recommendation to enable the [Draw.io plugin for VSCode](https://www.drawio.com/blog/edit-diagrams-with-github-dev). Simply click on `Install` when prompted in the bottom-right of your screen, and from now on you will be able to edit any of the Draw.io figures inside GitHub.

Expand All @@ -30,3 +30,5 @@ The development of this was supported by funding to the Blue Brain Project, a re
For license see LICENSE.txt.

Copyright (c) 2023-2024 Blue Brain Project/EPFL

Copyright (c) 2025 Open Brain Institute
6 changes: 3 additions & 3 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# **Blue Brain Open Platform** - Architecture Documentation
# **Open Brain Platform** - Architecture Documentation

The main goal of this repository is to provide a technical overview of the **Blue Brain Open Platform** and its overall architecture running on [Amazon Web Services (AWS)](https://aws.amazon.com/). In particular, each section contains some of the most relevant technical decisions and design considerations, as well as provides other relevant details such as which AWS services are required, among others.
The main goal of this repository is to provide a technical overview of the **Open Brain Platform** and its overall architecture running on [Amazon Web Services (AWS)](https://aws.amazon.com/). In particular, each section contains some of the most relevant technical decisions and design considerations, as well as provides other relevant details such as which AWS services are required, among others.

> [!IMPORTANT]
> **Don't forget:** Use [VSCode on GitHub](https://github.dev/BlueBrain/platform-docs) to edit the different sections, as well as to create/modify the diagrams via the recommended [Draw.io plugin](https://www.drawio.com/blog/edit-diagrams-with-github-dev) for VSCode.
> **Don't forget:** Use [VSCode on GitHub](https://github.dev/openbraininstitute/platform-docs) to edit the different sections, as well as to create/modify the diagrams via the recommended [Draw.io plugin](https://www.drawio.com/blog/edit-diagrams-with-github-dev) for VSCode.
2 changes: 1 addition & 1 deletion docs/modules/machinelearning/inference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

- **Description:** Finds similar scientific artefacts
- **Also known as:**
- **Source:** <https://github.com/BlueBrain/kg-inference-api>
- **Source:** <https://github.com/openbraininstitute/kg-inference-api>
- **API:** <>
- **AWS Dashboard:** <https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#dashboards/dashboard/KG-Inference>
- **AWS Service:** <https://us-east-1.console.aws.amazon.com/ecs/v2/clusters/kg_inference_api_cluster/services/kg-inference-api-service/health?region=us-east-1>
Expand Down
4 changes: 2 additions & 2 deletions docs/modules/provisioner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

## Overview

With the purpose of running large-scale simulations in the Blue Brain Open Platform, the **ParallelCluster Provisioner** is a service that provides the capability for deploying ParalellCluster instances on-demand for a given Project inside a Virtual Lab.
With the purpose of running large-scale simulations in the Open Brain Platform, the **ParallelCluster Provisioner** is a service that provides the capability for deploying ParalellCluster instances on-demand for a given Project inside a Virtual Lab.

When a new simulation is created, the [Workflow Service](../workflow/README.md) communicates via REST API with the Provisioner to request the creation of a ParallelCluster that will accommodate the workload from the simulation scheduled in the platform. If required, the Provisioner will then deploy a private HPC cluster that is tailor-made to the necessities of the campaign. Hence, allowing the Workflow Service to manage the different tasks required for the job to succeed.

Expand Down Expand Up @@ -54,7 +54,7 @@ Parallel jobs per Virtual Lab Project always run in a private ParallelCluster an

Scientific data is retrieved through the '`project`' directory in Lustre FSx, which is composed of two S3-DRA mount points targetting two separate S3 buckets for scientific data:

- A '`public`' directory containing all of the public assets of Blue Brain Open Platform stored in a shared S3 bucket. These assets are visible among all of the Virtual Labs.
- A '`public`' directory containing all the public assets of Open Brain Platform stored in a shared S3 bucket. These assets are visible among all of the Virtual Labs.
- A '`private`' directory that contains the assets stored by a particular Project in a Virtual Lab. The assets are implicitly not visible among Projects nor Virtual Labs.

In certain use-cases (e.g., eModel Fitting), Lustre FSx is omitted to optimize the overall operational costs. The simulations rely on EFS instead. The data stored on S3 is still accessible through S3-FUSE, and the data synchronizations are handled through a conventional data upload on the [Metadata and Data Service](../knowledgegraph/README.md).
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/workflow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

## Overview

One of the future use-cases expected in the Blue Brain Open Platform, would be to run large-scale simulations that may require dozens or even hundreds of computational resources and high-performance storage infrastructure (e.g., Lustre FSx). The orchestration for such simulations is mostly led by the **Workflow Service**, whose purpose is to spawn tailor-made resources through the [ParallelCluster Provisioner](../provisioner/README.md) and to control / monitor the execution of the different tasks associated with the workloads, among others.
One of the future use-cases expected in the Open Brain Platform, would be to run large-scale simulations that may require dozens or even hundreds of computational resources and high-performance storage infrastructure (e.g., Lustre FSx). The orchestration for such simulations is mostly led by the **Workflow Service**, whose purpose is to spawn tailor-made resources through the [ParallelCluster Provisioner](../provisioner/README.md) and to control / monitor the execution of the different tasks associated with the workloads, among others.

When a new simulation is requested from the interface of the platform, a request is made to the REST API of the Workflow Service to authorize the request and also to orchestrate the deployment of [`bbp-workflow`](https://github.com/openbraininstitute/bbp-workflow) inside an ECS Container Instance. The Provisioner is in charge of deploying the necessary infrastructure (if required) and provides the Workflow Service with the necessary information to locate and authenticate on the cluster. After this, the service will schedule and monitor the execution of the different tasks running in the cluster, as well as coordinate the data registration in the knowledge graph via the [Metadata and Data Service](../knowledgegraph/README.md).

Expand Down
8 changes: 4 additions & 4 deletions docs/overview/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Brief introduction with an overview of the platform. Lorem ipsum dolor sit amet,
> **TODO**
> Add proper introduction and Draw.io diagram below, instead of PNG
![Blue Brain Open Platform - Main Architecture](resources/1_main.drawio.svg)
![Open Brain Platform - Main Architecture](resources/1_main.drawio.svg)

## Infrastructure Overview

Expand All @@ -16,7 +16,7 @@ Brief introduction with an overview of the infrastructure, such as load balancer
> **TODO**
> Add proper details and Draw.io diagram below, instead of PNG
![Blue Brain Open Platform - Main Architecture](resources/2_infrastructure.drawio.svg)
![Open Brain Platform - Main Architecture](resources/2_infrastructure.drawio.svg)

## Virtual Labs and Projects

Expand All @@ -33,7 +33,7 @@ Team members must be granted access to `Projects`, and they are the only ones wi
It is possible to be part of a `Virtual Lab`, but not part of a `project`.

## Configuration and Deployment
The AWS infrastructure required for the Blue Brain Open Platform is exclusively managed through infrastructure-as-code. We use Terraform to configure and deploy the different components of the platform, including setting up IAM policies, security groups, or tagging.
The AWS infrastructure required for the Open Brain Platform is exclusively managed through infrastructure-as-code. We use Terraform to configure and deploy the different components of the platform, including setting up IAM policies, security groups, or tagging.

Each service is defined on its own Terraform module and operates independently from the rest of the services.

Expand All @@ -43,7 +43,7 @@ Each service is defined on its own Terraform module and operates independently f
## Cost Monitoring Support
Our infrastructure relies heavily on the use of **AWS Tags** in order to understand the overall operational costs for each of the services running in the platform. Our goal is to ensure that we can not only provide realistic costs estimates, but also identify the specific resources that each Virtual Lab and Project utilizes.

![Blue Brain Open Platform - Main Architecture](resources/3_costmonitoring_tags.drawio.svg)
![Open Brain Platform - Main Architecture](resources/3_costmonitoring_tags.drawio.svg)

Further, we have introduced mechanisms for monitoring the tags of each resource after Terraform runs and deploys the infrastructure changes. In particular, a dedicated CI job runs and utilizes **AWS Resource Explorer** to query for untagged resources. We then utilize different information from the resource to determine the component, and in certain cases the resource is tagged automatically (e.g., Terraform does not tag private ENIs).

Expand Down
6 changes: 3 additions & 3 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
site_name: OBP - Architecture Documentation
site_url: https://gianlucaficarelli.github.io/platform-docs/
repo_url: https://github.com/GianlucaFicarelli/platform-docs
site_name: OBP Architecture Documentation
site_url: https://openbraininstitute.github.io/platform-docs/
repo_url: https://github.com/openbraininstitute/platform-docs
copyright: '&copy; 2024 Blue Brain Project/EPFL, 2025 Open Brain Institute'
extra:
generator: false
Expand Down
6 changes: 3 additions & 3 deletions waf/README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# **Blue Brain Open Platform** - AWS Well-Architected Framework Review
# **Open Brain Platform** - AWS Well-Architected Framework Review

The **AWS Well-Architected Framework** describes key concepts, design principles, and architectural best practices for designing and running workloads in the cloud. This repository contains the review documents that are allowing us to understand how well the [Blue Brain Open Platform Architecture](../docs/) aligns with cloud best practices on AWS.
The **AWS Well-Architected Framework** describes key concepts, design principles, and architectural best practices for designing and running workloads in the cloud. This repository contains the review documents that are allowing us to understand how well the [Open Brain Platform Architecture](../docs/) aligns with cloud best practices on AWS.

> [!WARNING]
> This is a work-in-progress repository and the different pillars will be added as soon as the reviews are conducted.
> [!IMPORTANT]
> **Don't forget:** Use [VSCode on GitHub](https://github.dev/BlueBrain/platform-docs) to edit the different sections, as well as to create/modify the diagrams via the recommended [Draw.io plugin](https://www.drawio.com/blog/edit-diagrams-with-github-dev) for VSCode.
> **Don't forget:** Use [VSCode on GitHub](https://github.dev/openbraininstitute/platform-docs) to edit the different sections, as well as to create/modify the diagrams via the recommended [Draw.io plugin](https://www.drawio.com/blog/edit-diagrams-with-github-dev) for VSCode.
## Table of Contents

Expand Down

0 comments on commit e6c78a0

Please sign in to comment.