Skip to content

Commit

Permalink
First implementation
Browse files Browse the repository at this point in the history
  • Loading branch information
pbrisbin committed Nov 2, 2023
1 parent b3305f4 commit bb7d8f8
Show file tree
Hide file tree
Showing 22 changed files with 2,023 additions and 2,978 deletions.
57 changes: 57 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,13 @@ on:
push:
branches: main

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
S3_BUCKET: aws-s3-lock-ci

jobs:
build:
runs-on: ubuntu-latest
Expand All @@ -17,3 +24,53 @@ jobs:
- run: yarn install
- run: yarn build
- run: yarn test --passWithNoTests

integration:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: ${{ vars.AWS_REGION }}
role-to-assume: ${{ secrets.AWS_ROLE }}

- id: lock-1
name: Acquire a lock
uses: ./
with:
bucket: ${{ env.S3_BUCKET }}
expires: 5s

- id: lock-2
name: Wait on previous lock then acquire our own
uses: ./
with:
bucket: ${{ env.S3_BUCKET }}
timeout: 10s

- name: Verify locks
run: |
[[ -n '${{ steps.lock-1.outputs.acquired-at }}' ]] # lock-1
[[ -n '${{ steps.lock-2.outputs.acquired-at }}' ]] # lock-2
# lock-2 acquired at should be +5s
integration-post:
needs: integration
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: ${{ vars.AWS_REGION }}
role-to-assume: ${{ secrets.AWS_ROLE }}
- name: Assert lock was released
uses: ./
with:
bucket: ${{ env.S3_BUCKET }}
timeout: 0s
13 changes: 0 additions & 13 deletions .github/workflows/example.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion .restyled.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ restylers:
- whitespace:
include:
- "**/*"
- "!dist/index.js"
- "!dist/**/*"
- "*"
71 changes: 61 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,75 @@
# TypeScript Action Template
# AWS S3 Lock

Our custom template repository for GitHub Actions implemented in TypeScript.

[Creating a repository from a template][docs].

[docs]: https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template

**NOTE**: Be sure to look for strings like "TODO" or "Action name" and update
them accordingly.
Wait for, acquire, and release a distributed lock via S3 storage.

## Usage

```yaml
- uses: freckle/TODO-action@v1
- uses: aws-actions/configure-aws-credentials@v4
with:
# ...

# This blocks until the lock is acquired, or errors if timeout is reached
- uses: freckle/aws-s3-lock-action@v1
with:
# Required
bucket: an-existing-s3-bucket

# Optional, defaults shown
# name: {workflow}/{job}
# expires: 15m
# timeout: {matches expires}
# timeout-poll: 5s

- run: echo "Lock held, do work here"
```
![](./screenshot.png)
The lock is released (the S3 object deleted) in our Post step, which provides a
pretty robust guarantee of release. Expired locks are ignored (not deleted), so
it's recommended you put a Lifecyle policy on the Bucket to clean them up after
some time.
## Inputs and Outputs
See [action.yml](./action.yml) for a complete list of inputs and outputs.
## Implementation Details
### Algorithm
This tool implements a version of the locking algorithm described in this
[StackOverflow answer][answer].
[answer]: https://stackoverflow.com/questions/45222819/can-pseudo-lock-objects-be-used-in-the-amazon-s3-api/75347123#75347123
- Upload a lock object to S3 at `<name>.<created>.<uuid>.<expires>`

All time values are milliseconds since epoch.

- List all other lock objects (prefix `<name>.`)

Filter out any expired keys (looking at `expires`) and sort, which implicitly
means by `created` then `uuid` as desired.

- If the first one (i.e. oldest) is our own, we've acquired the lock
- If not, we lost the race; remove our object, wait, and try again

### Ordering Semantics

Each time we attempt to acquire the lock, we create a new key name (e.g.
`<created>` and `<expires>` both change), this effectively loses our "place in
line" but ensures that expiry is measured from time of acquisition and not time
of first attempt. There are trade-offs either way, and possible room for
improvement, so this is just how we're doing it for now.

### Caveat

**This tool is not meant to be bullet-proof**. We built it for our needs and
accept that there are simply no strong guarantees in this locking mechanism's
operation at scale. Your mileage may vary; patches welcome.

## Versioning

Versioned tags will exist, such as `v1.0.0` and `v2.1.1`. Branches will exist
Expand Down
35 changes: 17 additions & 18 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,37 +2,36 @@ name: aws-s3-lock
description: Wait on, acquire, and clean-up S3-based locks
author: "Freckle"
inputs:
name:
description: "Name for the lock"
bucket:
description: |
Name of an existing S3 bucket to use.
required: true
s3-bucket:
description: "Name of existing S3 bucket to use. If empty, one will be created."
name:
description: |
Name for the lock object. Include any prefix you want within the bucket.
The key will be built as "name.created.uuid.expires".
required: true
s3-prefix:
description: "Prefix for lock files within s3-bucket"
required: false
default: ""
lease:
default: "${{ github.workflow }}/${{ github.job }}"
expires:
description: |
How long to aquire the lock for. Default is 5m.
How long before concurrent operations consider this lock expired.
required: true
default: 5m
default: 15m
timeout:
description: |
How long to wait for the lock to become available. Default matches lease.
How long to wait for the lock to become available. Default matches
expires.
required: false
timeout-poll:
description: |
How long to wait between checks for the lock. Default is 5s.
How long to wait between attempts for the lock. Default is 5s.
required: true
default: 5s

outputs:
key:
description: "Key of the S3 object representing the lock"
acquired-at:
description: "Timestamp of acquisition, or null"
released-at:
description: "Timestamp of release, or null"

description: "Timestamp the lock was acquired"
runs:
using: "node20"
main: "dist/acquire/index.js"
Expand Down
33 changes: 33 additions & 0 deletions dist/acquire/index.js

Large diffs are not rendered by default.

Loading

0 comments on commit bb7d8f8

Please sign in to comment.