Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JWT token only #73

Merged
merged 11 commits into from
Aug 21, 2023
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -142,5 +142,8 @@ dmypy.json
# Cython debug symbols
cython_debug/

# Bartenders config file
# Bartender files
config.yaml
jobs/
private_key.pem
public_key.pem
36 changes: 27 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ Bartender is a middleware web service to schedule jobs on various
infrastructures.

It can run command line applications for visitors. The application input should
be a configuration file with links to data files in the same directory. After
authenticating with your local account or social login like ORCID or GitHub, you
can upload your configuration file and data files as an archive to the web
service for submission. Once the application has been executed the output files
be a configuration file with links to data files in the same directory.
After acquiring a JWT token, you can upload your configuration file and
data files as an archive to the web service for submission.
Once the job has been executed the output files
can be browsed with a web browser.

Bartender can be configured to run applications on a Slurm batch scheduler,
Expand All @@ -27,6 +27,9 @@ Bartender can be used as the computational backend for a web application, the
web application should guide visitors into the submission and show the results.
See <https://github.com/i-VRESSE/bartended-haddock3> for an example.

Documentation for users and developers is available
at <https://i-vresse-bartender.readthedocs.io> .

## Quickstart

1. Install bartender via github:
Expand Down Expand Up @@ -58,6 +61,16 @@ See <https://github.com/i-VRESSE/bartended-haddock3> for an example.
alembic upgrade "head"
```

1. Generate token to authenticate yourself

```bash
# Generate a rsa key pair
openssl genpkey -algorithm RSA -out private_key.pem \
-pkeyopt rsa_keygen_bits:2048
openssl rsa -pubout -in private_key.pem -out public_key.pem
bartender generate-token --username myname
```

1. Run the application

```bash
Expand All @@ -73,11 +86,16 @@ See <https://github.com/i-VRESSE/bartended-haddock3> for an example.
The interactive API documentation generated by FastAPI is at
<http://localhost:8000/api/docs>

To consume the bartender web service you need to authenticate yourself.
Authentication is done by passing a JWT token in the HTTP header `Authorization:
Bearer <token>` in the HTTP request. This token can be aquired by using the
register+login routes or using a [socal
login](https://i-vresse-bartender.readthedocs.io/en/latest/user_management.html#user-management).
### Authentication

To consume the bartender web service you need to authenticate yourself
with a [JWT token](https://jwt.io/) in the

* HTTP header `Authorization: Bearer <token>` or
* query parameter `?token=<token>` or
* Cookie `bartenderToken=<token>`of the HTTP request.

For more info see [Configuration docs](https://i-vresse-bartender.readthedocs.io/en/latest/configuration.html#authentication)

### Word count example

Expand Down
2 changes: 2 additions & 0 deletions deploy/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ RUN poetry install --without=dev --no-interaction --no-ansi
ENV BARTENDER_HOST=0.0.0.0
ENV BARTENDER_PORT=8000

# Mounting config.yaml and public_key.pem should be done when running the container

CMD ["/usr/local/bin/bartender", "serve"]

HEALTHCHECK --interval=5m --timeout=3s CMD curl -f http://${BARTENDER_HOST}:${BARTENDER_PORT}/api/health || exit 1
1 change: 1 addition & 0 deletions deploy/README.md
15 changes: 10 additions & 5 deletions deploy/docker-compose.dirac.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,17 @@ services:
BARTENDER_DB_USER: bartender
BARTENDER_DB_PASS: bartender
BARTENDER_DB_BASE: bartender
BARTENDER_PUBLIC_KEY: /app/src/public_key.pem
volumes:
- type: bind
source: ../config.yaml
target: /app/src/config.yaml
- type: bind
source: ../public_key.pem
target: /app/src/public_key.pem
- type: volume
source: bartender-jobs
target: /tmp/jobs

db:
image: postgres:15.2-bullseye
Expand All @@ -55,7 +62,7 @@ services:
migrator:
image: bartender-with-dirac:${BARTENDER_VERSION:-latest}
restart: "no"
command: '"alembic upgrade head"'
command: alembic upgrade head
environment:
BARTENDER_DB_HOST: bartender-db
BARTENDER_DB_PORT: 5432
Expand All @@ -65,11 +72,9 @@ services:
depends_on:
db:
condition: service_healthy
volumes:
- type: bind
source: ../config.yaml
target: /app/src/config.yaml

volumes:
bartender-jobs:
name: bartender-jobs
bartender-db-data:
name: bartender-db-data
19 changes: 17 additions & 2 deletions deploy/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@ version: '3.9'
services:
api:
build:
context: .
context: ..
dockerfile: ./deploy/Dockerfile
image: bartender:${BARTENDER_VERSION:-latest}
ports:
- "8000:8000"
restart: always
env_file:
- .env
- ../.env
depends_on:
db:
condition: service_healthy
Expand All @@ -19,6 +21,17 @@ services:
BARTENDER_DB_USER: bartender
BARTENDER_DB_PASS: bartender
BARTENDER_DB_BASE: bartender
BARTENDER_PUBLIC_KEY: /app/src/public_key.pem
volumes:
- type: bind
source: ../config.yaml
target: /app/src/config.yaml
- type: bind
source: ../public_key.pem
target: /app/src/public_key.pem
- type: volume
source: bartender-jobs
target: /tmp/jobs

db:
image: postgres:15.2-bullseye
Expand Down Expand Up @@ -51,5 +64,7 @@ services:
condition: service_healthy

volumes:
bartender-jobs:
name: bartender-jobs
bartender-db-data:
name: bartender-db-data
7 changes: 1 addition & 6 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,6 @@

intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
# Commonly used libraries, uncomment when used in package
# 'numpy': ('http://docs.scipy.org/doc/numpy/', None),
# 'scipy': ('http://docs.scipy.org/doc/scipy/reference/', None),
# 'scikit-learn': ('https://scikit-learn.org/stable/', None),
# 'matplotlib': ('https://matplotlib.org/stable/', None),
# 'pandas': ('http://pandas.pydata.org/docs/', None),
"sqlalchemy": ("https://docs.sqlalchemy.org/en/14/", None),
"asyncssh": ("https://asyncssh.readthedocs.io/en/latest/", None),
}
46 changes: 40 additions & 6 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,46 @@ BARTENDER_ENVIRONMENT="dev"
You can read more about BaseSettings class here:
<https://pydantic-docs.helpmanual.io/usage/settings/>

## Authentication

The bartender web service uses [JWT tokens](https://jwt.io/) for authentication.

The tokens should use the RS256 algorithm,
which requires a public and private RSA key pair.
A key pair can be generated with

```bash
openssl genpkey -algorithm RSA -out private_key.pem -pkeyopt rsa_keygen_bits:2048
openssl rsa -pubout -in private_key.pem -out public_key.pem
```

The private key of the RSA key pair is used to generate a token in
an another web application or with the `bartender generate-token` command.

The public key of the RSA key pair is used to verify that the token comes
from a trusted source.
The public key file location is `public_key.pem`
or value of `BARTENDER_PUBLIC_KEY` environment variable.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this description, who is the owner of the keypair? Is it the bartender or the consumer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If my understanding is correct, in this case, the "sender" should be the owner of the keypair. The sender uses their private key to create a token and the "recipient" uses the senders' public key to verify that it could only have been the sender (i.e. the only one knowing their private key) to have generated the token. Correct?

So does that mean that bartender could potentially store multiple public keys for various senders?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create #75 for multiple public keys. Will implement when we have a concrete use case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If my understanding is correct, in this case, the "sender" should be the owner of the keypair. The sender uses their private key to create a token and the "recipient" uses the senders' public key to verify that it could only have been the sender (i.e. the only one knowing their private key) to have generated the token. Correct?

Yep, correct. That is how https://en.wikipedia.org/wiki/Digital_signature works. The signer owns the private key and gives a public key to the verifier.

So does that mean that bartender could potentially store multiple public keys for various senders?


The token payload should contain the following claims:

* `sub`: The user id. Used to identifiy who submitted a job.
* `exp`: The expiration time of the token.
* `iss`: The issuer of the token. Used to track from where jobs are submitted.
* `roles`: The roles of the user.
See [Applications](#applications) how roles are used.

## Configuration file

Bartender uses a configuration file for setting up applications and
destinations. An [example configuration
file](https://github.com/i-VRESSE/bartender/blob/main/config-example.yaml) is
shipped with the repository. Here, we explain the options in more detail.
Bartender uses a configuration file for setting up applications and destinations.

The configuration file is `config.yaml` or
value of `BARTENDER_CONFIG_FILENAME` environment variable.
An
[example configuration file](https://github.com/i-VRESSE/bartender/blob/main/config-example.yaml)
is shipped with the repository.

Here, we explain the options in more detail.

## Applications

Expand Down Expand Up @@ -65,8 +99,7 @@ applications:
replaced with value of the config key.
* The `allowed_roles` key holds an array of role names, one of which a submitter
should have. When key is not set or list is empty then any authorized user
is allowed. See [User management docs](user_management.md#roles) how to
assign/unassign roles to/from users.
is allowed. See [Authentication](#authentication) how to set roles on users.
* The application command should not overwrite files uploaded during submission
as these might not be downloaded from location where application is run.

Expand Down Expand Up @@ -237,6 +270,7 @@ The destination picker could look something like:
def picker(
job_dir: Path,
application_name: str,
user: User,
context: "Context",
) -> str:
# Calculate size of job_dir in bytes
Expand Down
21 changes: 21 additions & 0 deletions docs/deploy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Deploy with docker compose

Create config file `config.yaml` as described at [configuration.md](configuration.md).
The `job_root_dir` property should be set to `/tmp/jobs`
which is a Docker compose volume.
Comment on lines +3 to +5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first I couldn't find that setting on the linked page. It would be helpful if the job_root_dir section on that page was closer to this header. Perhaps just right above "applications"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in a20d098


Store public RSA key for JWT auth in `public_key.pem` file next to `config.yaml`.

Start with

```bash
docker compose -f deploy/docker-compose.yml up
```

Web service will running on <http://0.0.0.0:8000>.

To login to web service you need to generate token with
the private counterpart of the public key.
See [configuration.md#authentication](configuration.md#authentication).
To use `bartender generate-token` command inside container you need make
the private key available in the container.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps add an instruction on how to do that (make private key available to container)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 1505b9c

2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@

self
develop
user_management
configuration
deploy
```
Loading