Skip to content

Commit

Permalink
update documents (#59)
Browse files Browse the repository at this point in the history 
* update documents

Signed-off-by: Henry Zhang <[email protected]>
  • Loading branch information
@hainingzhang @jiahaoc1993
hainingzhang authored and jiahaoc1993 committed Dec 2, 2019
1 parent c0485ee commit c8456f5
Show file tree
Hide file tree
Showing 4 changed files with 179 additions and 125 deletions.
19 changes: 14 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,23 @@
# Overview
KubeteFATE provides tools to deploy FATE using Docker Compose and Kubernetes.
Federated learning involves multiple parties to collaborately train a machine learning model, therefore it is usually based on a distributed system. KubeteFATE manages federated learning workloads using cloud native technologies such as containers. KubeFATE enables federated learning jobs to run across public, private and hybrid cloud environments.

Currently, KubeFATE supports the deployment of [FATE](https://github.com/FederatedAI/FATE) via Docker Compose and Kubernetes.

## Container images of FATE components
All components of a FATE release are pre-built into Docker images. They can be pulled from Docker Hub directly. It is a preferred approach to install FATE. It saves much time in building FATE from the source code.

[Harbor](https://github.com/goharbor/harbor) can be used as a local registry to store and serve images of FATE. It can replicate container images from Docker Hub for a local environemnt. Harbor significantly improves performance and reduces network consumption, hence it is recommended for environments using containers.

To build images of FATE components from source code, refer to [Building FATE images](https://github.com/FederatedAI/FATE/tree/master/docker-build). To set up Harbor registry for your environment, refer to this [guide](./registry/README.md).

## Deployment with Docker Compose
A user can deploy FATE on a single host by using Docker Compose. Please refer to [Docker Deployment](./docker-deploy/README.md) for more details.
Docker Compose can deploy FATE components on a single host. By using Docker Compose, FATE can be set up for environments of multiple parties which are collaborating in a federated manner. Please refer to [Docker Compose Deployment](./docker-deploy/README.md) for more details.

## Deployment on Kubernetes
For a multi-node deployment scenario, one of the solutions is to use a Kubernetes cluster as an underlying infrastructure to manage the FATE system. Please refer to [Kubernetes Deployment](./k8s-deploy/README.md) for more details.
To deploy FATE in the cloud or in a multi-node environment, a convenient way is to use a Kubernetes cluster as the underlying infrastructure. Helm Charts can be used to deploy FATE on Kubernetes. Please refer to [Kubernetes Deployment](./k8s-deploy/README.md) for more details.

## Usage of ".env"
By default, the script pulls the images from [Docker Hub](https://hub.docker.com/search?q=federatedai&type=image) during the deployment. A user could also modify `.env` to specify a registry to pull images from.
## Note on the usage of ".env"
By default, the installation script pulls the images from [Docker Hub](https://hub.docker.com/u/federatedai) during the deployment. A user could also modify `.env` to specify a local registry (such as Harbor) to pull images from.

## License
[Apache License 2.0](https://github.com/FederatedAI/FATE/blob/master/LICENSE)
89 changes: 64 additions & 25 deletions docker-deploy/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,33 +1,50 @@
# Deployment by Docker Compose
# FATE deployment using Docker Compose

This guide describes the process of deploying FATE using Docker Compose.

## Prerequisites
1. Docker: 18
2. Docker-Compose: 1.24
3. [The FATE Images](https://github.com/FederatedAI/FATE/tree/docker_1.1_contribution/docker-build) have been built and downloaded by nodes.
The nodes (target nodes) to install FATE must meet the following requirements:

1. A Linux host
2. Docker: 18+
3. Docker-Compose: 1.24+
4. Network connection to Internet to pull container images from Docker Hub. If network connection to Internet is not available, consider to set up [Harbor as a local registry](../registry/README.md) or use [offline images](https://github.com/FederatedAI/FATE/tree/master/docker-build).

## Deploying FATE
Use the following command to clone repo if you did not clone before:
A Linux host can be used as a deployment machine to run installation scripts to deploy FATE onto target hosts.

First, on a Linux host, clone KubeFATE source repo:
```bash
$ git clone [email protected]:FederatedAI/KubeFATE.git
```
By default, the script pulls the images from [Docker Hub](https://hub.docker.com/search?q=federatedai&type=image) during the deployment.
By default, the installation script pulls the images from Docker Hub during the deployment. If the target node is not connected to Internet, refer to the below section to set up a local registry such as Harbor and use the offline images.

### Set up a local registry Harbor (Optional)
Please refer to [this guide](../registry/README.md) to install Harbor as a local registry.

After setting up a Harbor registry, update the setting in the `.env` file. Change `RegistryURI` to the hostname or IP address of the Harbor instance. This setting lets the installation script use a local registry instead of Docker Hub.

In the below example, `192.168.10.1` is the IP address of Harbor.

### Use Third Party Registry (Optional)
It is recommended that non-Internet clusters use Harbor as a third-party registry. Please refer to [this guide](https://github.com/FederatedAI/KubeFATE/blob/master/registry/install_harbor.md) to install Harbor. Change the `RegistryURI` to [Harbor](https://goharbor.io/) hostname in the `.env` file. `192.168.10.1` is an example of Harbor IP.
```bash
$ cd KubeFATE/
$ vi .env

...

RegistryURI=192.168.10.1/federatedai

...
```

### Configure Parties
The following steps will illustrate how to deploy two parties on different hosts.
### Configuring multiple parties of FATE
There are usually multiple parties participating a federated training. Each party should install FATE using a set of configuration files and scripts.

The following steps illustrate how to generate necessary configuration files and deploy two parties on different hosts.

### Generate startup files
Before starting the FATE system, the user needs to define their parties in configuration file `./parties.conf`.
Before deploying the FATE system, multiple parties should be defined in the configuration file: `docker-deploy/parties.conf`.

The following sample of `parties.conf` defines two parities, they are party `10000` hosted on a machine *192.168.7.1* and `9999` hosted on a machine *192.168.7.2*.
In the following sample of `docker-deploy/parties.conf` , two parities are specified by id as `10000` and `9999`. They are going to be deployed on hosts with IP addresses of *192.168.7.1* and *192.168.7.2*, respectively.

```bash
user=root
Expand All @@ -36,21 +53,42 @@ partylist=(10000 9999)
partyiplist=(192.168.7.1 192.168.7.2)
exchangeip=192.168.7.1
```
By default, the exchange node co-locates on the same host of the first party. The exchange service runs on port 9371. For this reason, the IP address of the exchange node should be the same as that of the first party. If a standalone exchange node is needed, update the value of `exchangeip` to the IP address of the desired host.

**NOTE**: By default, the machine of the first party will also host the exchange on the 9371 port. A user can change the exchange IP if needed.
After completing the above configuration file, use the following commands to generate configuration of target hosts.
```bash
$ cd docker-deploy
$ bash generate_config.sh
```

Use the following command to deploy each party. Before running the command, ***please make sure host 192.168.7.1 and 192.168.7.2 allow password-less SSH access with SSH key***:
Now, tar files have been generated for each party including the exchange node (party). They are named as ```<party-id>-confs.tar ```.

### Deploying FATE to target hosts

**Note:** Before running the below commands, all target hosts must

* allow password-less SSH access with SSH key;
* meet the requirements specified in [Prerequisites](#Prerequisites).

To deploy FATE to all configured target hosts, use the below command:
```bash
$ bash generate_config.sh # generate the config file
$ bash docker_deploy.sh all # launch the deployment
$ bash docker_deploy.sh all
```

The script will copy `confs-10000.tar` and `confs-9999.tar` to host 192.168.7.1 and 192.168.7.2.
The script copies tar files (e.g. `10000-confs.tar` or `9999-confs.tar`) to corresponding target hosts. It then launches a FATE cluster on each host using `docker-compose` commands.


To deploy FATE to a single target host, use the below command with the party's id (10000 in the below example):
```bash
$ bash docker_deploy.sh 10000
```
To deploy the exchange node to a target host, use the below command:
```bash
$ bash docker_deploy.sh exchange
```

Afterward the script will log in to these hosts and use docker-compose command to start the FATE cluster.

Once the command returns, log in to any host and use `docker ps` to verify the status of cluster, an example output is as follows:
Once the commands finish, log in to any host and use `docker ps` to verify the status of the cluster. A sample output is as follows:

```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Expand All @@ -65,15 +103,15 @@ ed11ce8eb20d federatedai/egg:1.1-release "/bin/sh -c 'cd /da
5386bcb7565f federatedai/federation:1.1-release "/bin/sh -c 'cd /dat…" About a minute ago Up About a minute 9394/tcp confs-10000_federation_1
```

### Verify the Deployment
Since the `confs-10000_python_1` container hosts the `fate-flow` service, so we need to perform the test within that container. Use the following commands to launch:
### Verifying the deployment
On the target node of each party, a container named `confs-<party_id>_python_1` should have been created and running the `fate-flow` service. For example, on Party 10000's node, run the following commands to verify the deployment:
```bash
$ docker exec -it confs-10000_python_1 bash
$ source /data/projects/python/venv/bin/activate
$ cd /data/projects/fate/python/examples/toy_example
$ cd /data/projects/python/examples/toy_example/
$ python run_toy_example.py 10000 9999 1
```
If the test passed, the screen will print some messages like the follows:
If the test passed, the output may look like the following:
```
"2019-08-29 07:21:25,353 - secure_add_guest.py[line:96] - INFO: begin to init parameters of secure add example guest"
"2019-08-29 07:21:25,354 - secure_add_guest.py[line:99] - INFO: begin to make guest data"
Expand All @@ -84,4 +122,5 @@ If the test passed, the screen will print some messages like the follows:
"2019-08-29 07:21:33,920 - secure_add_guest.py[line:114] - INFO: receive host sum from guest"
"2019-08-29 07:21:34,118 - secure_add_guest.py[line:121] - INFO: success to calculate secure_sum, it is 2000.0000000000002"
```
For more details about the testing result, please refer to "/data/projects/fate/python/examples/toy_example/README.md"
For more details about the testing result, please refer to `python/examples/toy_example/README.md` .

101 changes: 101 additions & 0 deletions registry/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Installing Harbor as a local Registry
Harbor is an open source trusted cloud native registry project that stores, signs, and scans content. Harbor extends the open source Docker Distribution by adding the functionalities usually required by users such as security, identity and management. Having a registry closer to the build and run environment can improve the image transfer efficiency.

Detailed document please refer to [here](https://github.com/goharbor/harbor/blob/master/docs/installation_guide.md).

## Prerequisites
The requirements of the host to install Harbor are as follows:
1. A Linux host
2. Docker: 18+
3. Docker-Compose: 1.18+


### Downloading the installer

The binary of the offline installer can be downloaded from the release page of [Harbor](https://github.com/goharbor/harbor/releases/tag/v1.9.3). Use *tar* command to extract the package.

```bash
$ tar xvf harbor-offline-installer-v1.9.3.tgz
```

### Configuring Harbor

Configuration parameters are located in the file `harbor.yml`. Change hostname the local IP adress of your Harbor host, such as 192.168.0.1:

```bash
hostname: 192.168.10.1
```

### Installing Harbor
Run the below command to install Harbor. After installation, Harbor registry service should be up and running:

```bash
$ sudo ./install.sh
```

### Configuring docker client:
The default installation of Harbor uses HTTP - as such, you need to add the option `--insecure-registry` to your client's Docker daemon and restart the Docker service.

```bash
$ sudo vi /etc/docker/daemon.json

{
"insecure-registries" : [ "192.168.10.1" ]
}
```

Restart docker daemon:

```bash
$ sudo service docker restart
```


### Preparing FATE Images in Harbor

After Harbor has been installed, FATE docker images must be pushed to Harbor registry so that other nodes can pull FATE images from Harbor. There are basically two ways to obtain the FATE images:

- Build images from FATE source code and push to Harbor

- Replicate FATE imges from Docker Hub to Harbor

#### Build images from FATE source code

Refer to [Build Document](https://github.com/FederatedAI/FATE/blob/master/docker-build/README.md) to create FATE iamges. Once images are built, push them to Harbor.

Usually, a user does not need to take the lenghty time to build images from source. It is recommended to use te pre-built docker images of FATE directly. Refer to te below section to for more details.

#### Replicating images from Docker Hub

Log into Harbor's management portal using the URL `http://<your_harbor_host_ip`. The default username and password is admin / Harbor12345.

Go to `Projects` tab and click the `+ New Project` button.

<div style="text-align:left", align=left>
<img width=600 src="./images/harbor-project.png" />
</div>

- Create a new project and name the project `federatedai`. Check on the "Access Level" checkbox to make this a public project.

Go to `Registries` tab and click the `+ New Endpoint` button to add Docker Hub for the replication of FATE images.

<div style="text-align:left", align=left>
<img width=600 src="./images/harbor-endpoint.png" />
</div>

- Choose `docker-hub` as `provider` .

Go to `Replications` and click the `+ New Replication Rule` to create a rule to replicate container images from Docker Hub.

<div style="text-align:left", align=left>
<img width=600 src="./images/harbor_replication.png" />
</div>

Choose pull-based `Replication mode` and docker hub endpoint as source registry.

In thhe Source resource filters, under **Name**, enter `federatedai/* `. Click `Save` to create the rule.

Next, select the newly created replication rule and click `REPLICATE`, the FATE images from Docker Hub will be replicated to Harbor immediately. Depending on the network bandwidth, the images will appear under porject `federatedai` after the images have been replicated. Now FATE images are ready for local environment to pull from.

**NOTE:** The replication rule continues to take effect until it is disabled or cancelled. If FATE images are avaiable for a new release, they will be replicated to Harbor automatically.

95 changes: 0 additions & 95 deletions registry/install_harbor.md
View file

This file was deleted.

0 comments on commit c8456f5

Please sign in to comment.