Merge pull request #73 from i-VRESSE/asym-jwt-only

JWT token only

.dockerignore

@@ -145,3 +145,5 @@ cython_debug/
 
 # The app config
 /config.yaml
+/private_key.pem
+/public_key.pem

.gitignore

@@ -142,5 +142,8 @@ dmypy.json
 # Cython debug symbols
 cython_debug/
 
-# Bartenders config file
+# Bartender files
 config.yaml
+jobs/
+private_key.pem
+public_key.pem

README.md

@@ -11,10 +11,10 @@ Bartender is a middleware web service to schedule jobs on various
 infrastructures.
 
 It can run command line applications for visitors. The application input should
-be a configuration file with links to data files in the same directory. After
-authenticating with your local account or social login like ORCID or GitHub, you
-can upload your configuration file and data files as an archive to the web
-service for submission. Once the application has been executed the output files
+be a configuration file with links to data files in the same directory.
+After acquiring a JWT token, you can upload your configuration file and
+data files as an archive to the web service for submission.
+Once the job has been executed the output files
 can be browsed with a web browser.
 
 Bartender can be configured to run applications on a Slurm batch scheduler,
@@ -27,6 +27,9 @@ Bartender can be used as the computational backend for a web application, the
 web application should guide visitors into the submission and show the results.
 See <https://github.com/i-VRESSE/bartended-haddock3> for an example.
 
+Documentation for users and developers is available
+at <https://i-vresse-bartender.readthedocs.io> .
+
 ## Quickstart
 
 1. Install bartender via github:
@@ -58,6 +61,16 @@ See <https://github.com/i-VRESSE/bartended-haddock3> for an example.
     alembic upgrade "head"
     ```
 
+1. Generate token to authenticate yourself
+
+    ```bash
+    # Generate a rsa key pair
+    openssl genpkey -algorithm RSA -out private_key.pem \
+        -pkeyopt rsa_keygen_bits:2048
+    openssl rsa -pubout -in private_key.pem -out public_key.pem
+    bartender generate-token --username myname
+    ```
+
 1. Run the application
 
     ```bash
@@ -73,11 +86,16 @@ See <https://github.com/i-VRESSE/bartended-haddock3> for an example.
 The interactive API documentation generated by FastAPI is at
 <http://localhost:8000/api/docs>
 
-To consume the bartender web service you need to authenticate yourself.
-Authentication is done by passing a JWT token in the HTTP header `Authorization:
-Bearer <token>` in the HTTP request. This token can be aquired by using the
-register+login routes or using a [socal
-login](https://i-vresse-bartender.readthedocs.io/en/latest/user_management.html#user-management).
+### Authentication
+
+To consume the bartender web service you need to authenticate yourself
+with a [JWT token](https://jwt.io/) in the
+
+* HTTP header `Authorization: Bearer <token>` or
+* query parameter `?token=<token>` or
+* Cookie `bartenderToken=<token>`of the HTTP request.
+
+For more info see [Configuration docs](https://i-vresse-bartender.readthedocs.io/en/latest/configuration.html#authentication)
 
 ### Word count example
 

deploy/Dockerfile

@@ -23,6 +23,8 @@ RUN poetry install --without=dev --no-interaction --no-ansi
 ENV BARTENDER_HOST=0.0.0.0
 ENV BARTENDER_PORT=8000
 
+# Mounting config.yaml and public_key.pem should be done when running the container
+
 CMD ["/usr/local/bin/bartender", "serve"]
 
 HEALTHCHECK --interval=5m --timeout=3s CMD curl -f http://${BARTENDER_HOST}:${BARTENDER_PORT}/api/health || exit 1

deploy/README.md

@@ -0,0 +1 @@
+../docs/deploy.md

deploy/docker-compose.dirac.yml

@@ -31,10 +31,17 @@ services:
       BARTENDER_DB_USER: bartender
       BARTENDER_DB_PASS: bartender
       BARTENDER_DB_BASE: bartender
+      BARTENDER_PUBLIC_KEY: /app/src/public_key.pem
     volumes:
     - type: bind
       source: ../config.yaml
       target: /app/src/config.yaml
+    - type: bind
+      source: ../public_key.pem
+      target: /app/src/public_key.pem
+    - type: volume
+      source: bartender-jobs
+      target: /tmp/jobs
 
   db:
     image: postgres:15.2-bullseye
@@ -55,7 +62,7 @@ services:
   migrator:
     image: bartender-with-dirac:${BARTENDER_VERSION:-latest}
     restart: "no"
-    command: '"alembic upgrade head"'
+    command: alembic upgrade head
     environment:
       BARTENDER_DB_HOST: bartender-db
       BARTENDER_DB_PORT: 5432
@@ -65,11 +72,9 @@ services:
     depends_on:
       db:
         condition: service_healthy
-    volumes:
-    - type: bind
-      source: ../config.yaml
-      target: /app/src/config.yaml
 
 volumes:
+  bartender-jobs:
+    name: bartender-jobs
   bartender-db-data:
     name: bartender-db-data

deploy/docker-compose.yml

@@ -3,12 +3,14 @@ version: '3.9'
 services:
   api:
     build:
-      context: .
+      context: ..
       dockerfile: ./deploy/Dockerfile
     image: bartender:${BARTENDER_VERSION:-latest}
+    ports:
+    - "8000:8000"
     restart: always
     env_file:
-    - .env
+    - ../.env
     depends_on:
       db:
         condition: service_healthy
@@ -19,6 +21,22 @@ services:
       BARTENDER_DB_USER: bartender
       BARTENDER_DB_PASS: bartender
       BARTENDER_DB_BASE: bartender
+      BARTENDER_PUBLIC_KEY: /app/src/public_key.pem
+    volumes:
+    - type: bind
+      source: ../config.yaml
+      target: /app/src/config.yaml
+    - type: bind
+      source: ../public_key.pem
+      target: /app/src/public_key.pem
+    # If you want to generate a token for testing purposes with the `bartender generate-token` command
+    # also mount private key by uncommenting the following lines
+    # - type: bind
+    #   source: ../private_key.pem
+    #   target: /app/src/private_key.pem
+    - type: volume
+      source: bartender-jobs
+      target: /tmp/jobs
 
   db:
     image: postgres:15.2-bullseye
@@ -51,5 +69,7 @@ services:
         condition: service_healthy
 
 volumes:
+  bartender-jobs:
+    name: bartender-jobs
   bartender-db-data:
     name: bartender-db-data

docs/conf.py

@@ -39,11 +39,6 @@
 
 intersphinx_mapping = {
     "python": ("https://docs.python.org/3", None),
-    # Commonly used libraries, uncomment when used in package
-    # 'numpy': ('http://docs.scipy.org/doc/numpy/', None),
-    # 'scipy': ('http://docs.scipy.org/doc/scipy/reference/', None),
-    # 'scikit-learn': ('https://scikit-learn.org/stable/', None),
-    # 'matplotlib': ('https://matplotlib.org/stable/', None),
-    # 'pandas': ('http://pandas.pydata.org/docs/', None),
+    "sqlalchemy": ("https://docs.sqlalchemy.org/en/14/", None),
     "asyncssh": ("https://asyncssh.readthedocs.io/en/latest/", None),
 }

docs/configuration.md

@@ -29,12 +29,56 @@ BARTENDER_ENVIRONMENT="dev"
 You can read more about BaseSettings class here:
 <https://pydantic-docs.helpmanual.io/usage/settings/>
 
+## Authentication
+
+The bartender web service uses [JWT tokens](https://jwt.io/) for authentication.
+
+The tokens should use the RS256 algorithm,
+which requires a public and private RSA key pair.
+A key pair can be generated with
+
+```bash
+openssl genpkey -algorithm RSA -out private_key.pem -pkeyopt rsa_keygen_bits:2048
+openssl rsa -pubout -in private_key.pem -out public_key.pem
+```
+
+The private key of the RSA key pair is used to sign a token in
+an another web application or with the `bartender generate-token` command.
+
+The public key of the RSA key pair is used to verify that the token comes
+from a trusted source.
+The public key file location is `public_key.pem`
+or value of `BARTENDER_PUBLIC_KEY` environment variable.
+
+The token payload should contain the following claims:
+
+* `sub`: The user id. Used to identifiy who submitted a job.
+* `exp`: The expiration time of the token.
+* `iss`: The issuer of the token. Used to track from where jobs are submitted.
+* `roles`: Optionally. The roles of the user.
+  See [Applications](#applications) how roles are used.
+
 ## Configuration file
 
-Bartender uses a configuration file for setting up applications and
-destinations. An [example configuration
-file](https://github.com/i-VRESSE/bartender/blob/main/config-example.yaml) is
-shipped with the repository. Here, we explain the options in more detail.
+Bartender uses a configuration file for setting up applications and destinations.
+
+The configuration file is `config.yaml` or
+value of `BARTENDER_CONFIG_FILENAME` environment variable.
+An
+[example configuration file](https://github.com/i-VRESSE/bartender/blob/main/config-example.yaml)
+is shipped with the repository.
+
+Here, we explain the options in more detail.
+
+## Job root dir
+
+By default, the files of jobs are stored in `/tmp/jobs`. To change the
+directory, set the `job_root_dir` parameter in the configuration file to a valid
+path.
+
+```yaml
+job_root_dir: /tmp/jobs
+```
 
 ## Applications
 
@@ -53,8 +97,11 @@ applications:
   haddock3:
     command: haddock3 $config
     config: workflow.cfg
+  adminapp:
+    command: some-admin-application $config
+    config: config.yaml
     allowed_roles:
-      - easy
+      - admin  # Only users with admin role can submit jobs for this application
 ```
 
 * The key is the name of the application
@@ -63,10 +110,10 @@ applications:
 * The `command` key is the command executed in the directory of the unpacked
   archive that the consumer uploaded. The `$config` in command string will be
   replaced with value of the config key.
-* The `allowed_roles` key holds an array of role names, one of which a submitter
-  should have. When key is not set or list is empty then any authorized user
-  is allowed. See [User management docs](user_management.md#roles) how to
-  assign/unassign roles to/from users.
+* Optionally, the `allowed_roles` key holds an array of role names,
+  one of which a submitter should have.
+  When key is not set or list is empty then any authorized user
+  is allowed. See [Authentication](#authentication) how to set roles on users.
 * The application command should not overwrite files uploaded during submission
   as these might not be downloaded from location where application is run.
 
@@ -237,6 +284,7 @@ The destination picker could look something like:
 def picker(
     job_dir: Path,
     application_name: str,
+    user: User,
     context: "Context",
 ) -> str:
   # Calculate size of job_dir in bytes
@@ -329,16 +377,6 @@ destination use:
 destination_picker: bartender.picker.pick_round
 ```
 
-## Job root dir
-
-By default, the files of jobs are stored in `/tmp/jobs`. To change the
-directory, set the `job_root_dir` parameter in the configuration file to a valid
-path.
-
-```yaml
-job_root_dir: /tmp/jobs
-```
-
 ## Job flow
 
 Diagram of a job flowing through web service, schedulers and filesystems.

docs/deploy.md

@@ -0,0 +1,22 @@
+# Deploy with docker compose
+
+Create config file `config.yaml` as described at [configuration.md](configuration.md).
+The `job_root_dir` property should be set to `/tmp/jobs`
+which is a Docker compose volume.
+
+Store public RSA key for JWT auth in `public_key.pem` file next to `config.yaml`.
+
+Start with
+
+```bash
+docker compose -f deploy/docker-compose.yml up
+```
+
+Web service will running on <http://0.0.0.0:8000>.
+
+To login to web service you need to generate token and sign it with
+the private counterpart of the public key.g
+If you want to generate a token with the
+`docker compose -f deploy/docker-compose.yml exec api bartender generate-token` command
+you should uncomment the private key volume bind in `deploy/docker-compose.yml`.
+See [configuration.md#authentication](configuration.md#authentication).

docs/index.md

@@ -8,6 +8,6 @@
 
 self
 develop
-user_management
 configuration
+deploy
 ```