Skip to content

Commit

Permalink
doc: start to refine RGMapping
Browse files Browse the repository at this point in the history
Signed-off-by: Sijie Shen <[email protected]>
  • Loading branch information
ds-ssj committed May 14, 2024
1 parent a489917 commit e85bbed
Show file tree
Hide file tree
Showing 5 changed files with 59 additions and 56 deletions.
100 changes: 50 additions & 50 deletions docs/documentation/deployment/deploy-distributed.rst
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
.. _deploy-distributed:

Distributed Deployment
====================
========================================

GART is designed to be deployed in a distributed manner.
GART is designed to be deployed in a distributed manner.
This means that you can deploy different components of GART on different nodes of a cluster.
To achieve this, you need to provide a configuration file, and GART provides a Python script to launch GART on a cluster easily.

Expand All @@ -13,56 +13,56 @@ User Configuration
Please ensure that the GART folder is accessible at the same path on all nodes of the cluster.
Then you can create a configuration file in the following format:

```
{
// Database port
"db_host": "127.0.0.1",
// Database port
"db_port": 5432,
// Database type: mysql or postgresql
"db_type": "postgresql",
// Database name
"db_name": "ldbc",
// Database user
"db_user": "dbuser",
// Database password
"db_password": "123456",
"rgmapping_file": "/path/to/rgmapping.yaml",
"v6d_size": "750G",
"etcd_prefix": "gart_meta_",
// entrypoint of the kafka server
"kafka_server": "127.0.0.1:9092",
"total_subgraph_num": 2,
// path to the GART build folder
"gart_bin_path": "/path/to/GART/build/",
// whether to enable bulk load
"enable_bulkload": 1,
// where to launch the capturer
"capturer_host": "127.0.0.1",
// where to launch the converter
"converter_host": "127.0.0.1",
// path to kafka
"kafka_path": "/path/to/kafka",
"writer_hosts": [
{
// where to launch the writer to create subgraph 0
"subgraph_id": 0,
"host": "127.0.0.1"
},
{
// where to launch the writer to create subgraph 1
"subgraph_id": 1,
"host": "127.0.0.1"
}
]
}
```
.. code:: json
:linenos:
{
// Database port
"db_host": "127.0.0.1",
// Database port
"db_port": 5432,
// Database type: mysql or postgresql
"db_type": "postgresql",
// Database name
"db_name": "ldbc",
// Database user
"db_user": "dbuser",
// Database password
"db_password": "123456",
"rgmapping_file": "/path/to/rgmapping.yaml",
"v6d_size": "750G",
"etcd_prefix": "gart_meta_",
// entrypoint of the kafka server
"kafka_server": "127.0.0.1:9092",
"total_subgraph_num": 2,
// path to the GART build folder
"gart_bin_path": "/path/to/GART/build/",
// whether to enable bulk load
"enable_bulkload": 1,
// where to launch the capturer
"capturer_host": "127.0.0.1",
// where to launch the converter
"converter_host": "127.0.0.1",
// path to kafka
"kafka_path": "/path/to/kafka",
"writer_hosts": [
{
// where to launch the writer to create subgraph 0
"subgraph_id": 0,
"host": "127.0.0.1"
},
{
// where to launch the writer to create subgraph 1
"subgraph_id": 1,
"host": "127.0.0.1"
}
]
}
Launch GART with configuration
--------------------
----------------------------------------
GART provides a Python script to launch GART on a cluster easily. You can launch GART with the following command by providing the configuration file in the previous step:

```bash
./path/to/GART/scripts/distributed_deployment.py --config_file /path/to/user_config.json
.. code:: bash
```
./path/to/GART/scripts/distributed_deployment.py --config_file /path/to/user_config.json
2 changes: 1 addition & 1 deletion docs/documentation/deployment/deploy-k8s.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.. _deploy-k8s:

Distributed on Kubernetes
====================
========================================
4 changes: 1 addition & 3 deletions docs/documentation/design/key-concepts/rgmapping.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@ RGMapping
What is RGMapping?
-------------------

GART is an in-memory system for real-time dynamic graph computation. GART can reuse the execution engines of existing OLTP and graph-specific systems. It synchronizes the updated data through transactional logs from OLTP components. GART can be used with relational OLTP systems such as MySQL and PostgreSQL. It follows the architecture of GART.

GART provides a transparent data model conversion. During the use of GART, data changes from relational databases are converted into graph data updates based on user-defined model mapping rules (RGMapping). GART provides a set of interfaces (``def_vetex``, ``def_edge``, ... ) for DBAs to define data model conversion rules, which can be compatible with SQL/PGQ DDL.
GART provides a transparent data model conversion. During the use of GART, data changes from relational databases are converted into graph data updates based on user-defined model mapping rules (RGMapping). GART allows DBAs to define data model conversion rules in multiple ways.
The following figure gives an example of RGMapping.

.. figure:: /images/rgmapping.png
Expand Down
7 changes: 6 additions & 1 deletion docs/documentation/getting-started/quick-start.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
Quick Start
===============

This section gives an experiential process of GART where we provide containers for deployment. Users can also try local deployment, we will give local deployment and usage tips.

Step 0: Get GART
-------------------

Expand All @@ -19,7 +21,10 @@ Step 1: Start the GART Environment
--------------------------------------

We provide a Docker image that contains all the dependencies and configurations needed to run GART. The Docker image is based on Ubuntu 20.04.
If you would like to install it locally, please refer to the `detailed deployment tutorial <../deployment/deploy-local.html>`_ we provide.

.. tip::

If you would like to install it locally, please refer to the `detailed deployment tutorial <../deployment/deploy-local.html>`_ we provide.

To build the Docker container called ``gart-env`` and enter the container:

Expand Down
2 changes: 1 addition & 1 deletion docs/documentation/getting-started/use-cases.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ As shown in the figure below, in a business database (such as MySQL), user infor

Whenever there are new users and new follows (as well as modifications and deletions), these will be executed on this database. To perform friend recommendations for users, it is necessary to apply the PageRank graph computation algorithm on this dataset. However, operating graphs directly in a relational database is inefficient, resulting in a large number of join operations. GART captures the changes in data by fetching the binary log (binlog) from MySQL.

During the use of GART, changes in data from the relational database are transformed into updates of graph data according to the user-defined model mapping rules. In this way, the graph data can reflect changes in real-time, and dynamic graph storage provided by GART allows the use of the graph computing engine (GraphScope) to perform PageRank calculations, similar to how a static graph is accessed.
During the use of GART, changes in data from the relational database are transformed into updates of graph data according to the user-defined model mapping rules. In this way, the graph data can reflect changes in real time, and dynamic graph storage provided by GART allows the use of the graph computing engine (GraphScope) to perform PageRank calculations, similar to how a static graph is accessed.

These model transformation rules include the user's definition of the graph structure and how information in the relational data is mapped to the vertices, edges, and attributes in the graph.

Expand Down

0 comments on commit e85bbed

Please sign in to comment.