Skip to content
This repository has been archived by the owner on Feb 20, 2024. It is now read-only.
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: digicatapult/dscp-api
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v5.0.1
Choose a base ref
...
head repository: digicatapult/dscp-api
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
  • 13 commits
  • 15 files changed
  • 5 contributors

Commits on Mar 21, 2023

  1. Fix yq

    mattdean-digicatapult committed Mar 21, 2023

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    a0b195d View commit details
  2. 5.0.2

    mattdean-digicatapult committed Mar 21, 2023
    Copy the full SHA
    a94b8fd View commit details
  3. Chore/add l3 roles (#86)

    * Add l3 roles
    
    * 5.0.3
    mattdean-digicatapult authored Mar 21, 2023
    Copy the full SHA
    2ca187d View commit details

Commits on Mar 24, 2023

  1. Remove 'Default role must be in roles' check (#87)

    * remove owner in roles check
    
    * v bump
    
    * removed wrong test...
    jonmattgray authored Mar 24, 2023
    Copy the full SHA
    ab11e3e View commit details

Commits on Mar 31, 2023

  1. Update docker - all dependencies (#75)

    * Update docker - all dependencies
    
    * 5.0.4
    
    ---------
    
    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Co-authored-by: Matthew Dean <[email protected]>
    renovate[bot] and mattdean-digicatapult authored Mar 31, 2023
    Copy the full SHA
    cc0b054 View commit details
  2. Chore/update simple nft events (#88)

    * Update dependency mock-jwks to v2
    
    * Switch events to ProcessRan
    
    * Update refered to docker image
    
    * 5.0.5
    
    * Fix tests after upgrading mock-jwks
    
    ---------
    
    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    mattdean-digicatapult and renovate[bot] authored Mar 31, 2023
    Copy the full SHA
    d048b79 View commit details

Commits on Apr 18, 2023

  1. simpleNFT -> utxoNFT (#90)

    * simpleNFT -> utxoNFT
    
    * use latest node
    jonmattgray authored Apr 18, 2023
    Copy the full SHA
    6399345 View commit details

Commits on Aug 29, 2023

  1. Copy the full SHA
    78a3990 View commit details
  2. Upgrade deps + new check-version action (#98)

    * use c8
    
    * add docker build test
    jonmattgray authored Aug 29, 2023
    Copy the full SHA
    081b588 View commit details
  3. Merge pull request #99 from rmlearney-digicatapult/rename-veritable-d…

    …scp-readme
    
    Update README.md to use dscp in place of veritable
    rmlearney-digicatapult authored Aug 29, 2023
    Copy the full SHA
    269b33f View commit details

Commits on Aug 30, 2023

  1. Copy the full SHA
    351ee03 View commit details

Commits on Sep 14, 2023

  1. Copy the full SHA
    72110fa View commit details

Commits on Feb 20, 2024

  1. Update README.md

    dblane-digicatapult authored Feb 20, 2024
    Copy the full SHA
    e7cf5cb View commit details
Showing with 2,706 additions and 1,779 deletions.
  1. 0 .nycrc → .c8rc
  2. +4 −11 .github/workflows/release.yml
  3. +35 −10 .github/workflows/tests.yml
  4. +0 −3 .gitignore
  5. +42 −31 README.md
  6. +5 −8 app/api-v3/api-doc.js
  7. +0 −2 app/env.js
  8. +2 −6 app/server.js
  9. +7 −16 app/util/appUtil.js
  10. +4 −5 docker-compose.yaml
  11. +2,577 −1,563 package-lock.json
  12. +28 −29 package.json
  13. +0 −58 scripts/check-version.sh
  14. +1 −5 test/integration/regressions.test.js
  15. +1 −32 test/integration/routes.test.js
File renamed without changes.
15 changes: 4 additions & 11 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -122,18 +122,11 @@ jobs:

steps:
- uses: actions/checkout@v3
- run: git fetch --depth=1 --tags origin
- name: Install yq
run: sudo snap install yq
- name: Check Build Version
- name: Check version
id: get_version
run: ./scripts/check-version.sh
shell: bash
- name: Skipping release as version has not increased
if: steps.get_version.outputs.IS_NEW_VERSION != 'true'
shell: bash
run: |
echo "Skipping release as current version has already been published"
uses: digicatapult/check-version@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}

publish:
name: 'Publish package'
45 changes: 35 additions & 10 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -110,14 +110,39 @@ jobs:

steps:
- uses: actions/checkout@v3
- run: git fetch --depth=1 --tags origin
- name: Install yq
run: sudo snap install yq
- name: Check Build Version
- name: Check version
id: get_version
run: ./scripts/check-version.sh
shell: bash
- name: Error if version is not increased
shell: bash
run: |
exit $([[ "${{steps.get_version.outputs.IS_NEW_VERSION}}" = "true" ]] && echo 0 || echo 1)
uses: digicatapult/check-version@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}

docker-build:
name: 'Build docker image'
needs:
- lint
- tests
- check-version
- dependency-check
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

# Docker build
- name: Setup QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: all
- name: Setup Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2
with:
buildkitd-flags: '--debug'
- name: Publish multi-arch image
uses: docker/build-push-action@v4
with:
push: false
builder: ${{ steps.buildx.outputs.name }}
context: .
file: ./Dockerfile
platforms: linux/amd64, linux/arm64
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -5,7 +5,6 @@ node_modules/

# Test coverage
coverage
.nyc_output
coverage.json

# Logs
@@ -19,8 +18,6 @@ npm-debug.log*
!.vscode/extensions.json
.idea/

helm/**/*.tgz

# Misc
.DS_Store
.env
73 changes: 42 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Veritable API
# DSCP API

# DEPRECATED 20/02/24

## Description

A `Node.js` API to support communication to the [Substrate-based](https://www.substrate.io/) [`veritable-node`](https://github.com/digicatapult/dscp-node) (via [`polkadot-js/api`](https://www.npmjs.com/package/@polkadot/api)) and an [`IPFS`](https://ipfs.io/) node.
A `Node.js` API to support communication to the [Substrate-based](https://www.substrate.io/) [`dscp-node`](https://github.com/digicatapult/dscp-node) (via [`polkadot-js/api`](https://www.npmjs.com/package/@polkadot/api)) and an [`IPFS`](https://ipfs.io/) node.

## Getting started

@@ -12,12 +14,12 @@ First, ensure you're running the correct [version](.node-version) of `npm`, then
npm install
```

The API requires instances of [`veritable-node`](https://github.com/digicatapult/dscp-node) and [`IPFS`](https://ipfs.io/).
The API requires instances of [`dscp-node`](https://github.com/digicatapult/dscp-node) and [`IPFS`](https://ipfs.io/).
To bring them up locally:

### `veritable-node`
### `dscp-node`

Clone [veritable-node](https://github.com/digicatapult/dscp-node) and follow the README to setup and build a local node. Then run the following in its root directory:
Clone [dscp-node](https://github.com/digicatapult/dscp-node) and follow the README to setup and build a local node. Then run the following in its root directory:

```
./target/release/dscp-node --dev
@@ -46,33 +48,31 @@ npm run test:integration

## Authentication

`veritable-api` uses an [Auth0](https://auth0.com/) Machine to Machine API to issue a JSON Web Token for authentication on its endpoints. You will need to create your own Auth0 API, which can be done for free, and set the appropriate [environment variables](#configuration) (those prefixed with `AUTH`). Follow the start of this [tutorial](https://auth0.com/docs/quickstart/backend/nodejs#configure-auth0-apis) to create an API. Go [here](app/routes/auth.js) and [here](app/auth.js) to see where the environment variables are used.
`dscp-api` uses an [Auth0](https://auth0.com/) Machine to Machine API to issue a JSON Web Token for authentication on its endpoints. You will need to create your own Auth0 API, which can be done for free, and set the appropriate [environment variables](#configuration) (those prefixed with `AUTH`). Follow the start of this [tutorial](https://auth0.com/docs/quickstart/backend/nodejs#configure-auth0-apis) to create an API. Go [here](app/routes/auth.js) and [here](app/auth.js) to see where the environment variables are used.

## Configuration

The following environment variables are used by `veritable-api` and can be configured. Entries marked as `required` are needed when running `veritable-api` in production mode.

| variable | required | default | description |
| :------------------------------ | :------: | :--------------------: | :---------------------------------------------------------------------------------------------------------------------------- |
| PORT | N | `3001` | The port for the API to listen on |
| EXTERNAL_ORIGIN | N | | The origin from which the OpenAPI service is accessible. If not provided the value will default to `http://localhost:${PORT}` |
| EXTERNAL_PATH_PREFIX | N | | A path prefix from which this service is served |
| API_HOST | Y | - | The hostname of the `dscp-node` the API should connect to |
| API_PORT | N | `9944` | The port of the `dscp-node` the API should connect to |
| LOG_LEVEL | N | `info` | Logging level. Valid values are [`trace`, `debug`, `info`, `warn`, `error`, `fatal`] |
| USER_URI | Y | - | The Substrate `URI` representing the private key to use when making `dscp-node` transactions |
| IPFS_HOST | Y | - | Hostname of the `IPFS` node to use for metadata storage |
| IPFS_PORT | N | `15001` | Port of the `IPFS` node to use for metadata storage |
| METADATA_KEY_LENGTH | N | `32` | Fixed length of metadata keys |
| METADATA_VALUE_LITERAL_LENGTH | N | `32` | Fixed length of metadata LITERAL values |
| API_VERSION | N | `package.json version` | API version |
| API_MAJOR_VERSION | N | `v3` | API major version |
| FILE_UPLOAD_MAX_SIZE | N | `200 * 1024 * 1024` | The Maximum file upload size (bytes) |
| SUBSTRATE_STATUS_POLL_PERIOD_MS | N | `10 * 1000` | Number of ms between calls to check dscp-node status |
| SUBSTRATE_STATUS_TIMEOUT_MS | N | `2 * 1000` | Number of ms to wait for response to dscp-node health requests |
| IPFS_STATUS_POLL_PERIOD_MS | N | `10 * 1000` | Number of ms between calls to check ipfs status |
| IPFS_STATUS_TIMEOUT_MS | N | `2 * 1000` | Number of ms to wait for response to ipfs health requests |
| AUTH_TYPE | N | `NONE` | Authentication type for routes on the service. Valid values: [`NONE`, `JWT`, `EXTERNAL`] |
The following environment variables are used by `dcsp-api` and can be configured. Entries marked as `required` are needed when running `dscp-api` in production mode.

| variable | required | default | description |
| :------------------------------ | :------: | :--------------------: | :------------------------------------------------------------------------------------------- |
| PORT | N | `3001` | The port for the API to listen on |
| API_HOST | Y | - | The hostname of the `dscp-node` the API should connect to |
| API_PORT | N | `9944` | The port of the `dscp-node` the API should connect to |
| LOG_LEVEL | N | `info` | Logging level. Valid values are [`trace`, `debug`, `info`, `warn`, `error`, `fatal`] |
| USER_URI | Y | - | The Substrate `URI` representing the private key to use when making `dscp-node` transactions |
| IPFS_HOST | Y | - | Hostname of the `IPFS` node to use for metadata storage |
| IPFS_PORT | N | `15001` | Port of the `IPFS` node to use for metadata storage |
| METADATA_KEY_LENGTH | N | `32` | Fixed length of metadata keys |
| METADATA_VALUE_LITERAL_LENGTH | N | `32` | Fixed length of metadata LITERAL values |
| API_VERSION | N | `package.json version` | API version |
| API_MAJOR_VERSION | N | `v3` | API major version |
| FILE_UPLOAD_MAX_SIZE | N | `200 * 1024 * 1024` | The Maximum file upload size (bytes) |
| SUBSTRATE_STATUS_POLL_PERIOD_MS | N | `10 * 1000` | Number of ms between calls to check dscp-node status |
| SUBSTRATE_STATUS_TIMEOUT_MS | N | `2 * 1000` | Number of ms to wait for response to dscp-node health requests |
| IPFS_STATUS_POLL_PERIOD_MS | N | `10 * 1000` | Number of ms between calls to check ipfs status |
| IPFS_STATUS_TIMEOUT_MS | N | `2 * 1000` | Number of ms to wait for response to ipfs health requests |
| AUTH_TYPE | N | `NONE` | Authentication type for routes on the service. Valid values: [`NONE`, `JWT`, `EXTERNAL`] |

The following environment variables are additionally used when `AUTH_TYPE : 'JWT'`

@@ -96,7 +96,7 @@ npm start

### Authenticated endpoints

If `AUTH_TYPE` env is set to `JWT`, the rest of the endpoints in `veritable-api` require authentication in the form of a header `'Authorization: Bearer YOUR_ACCESS_TOKEN'`:
If `AUTH_TYPE` env is set to `JWT`, the rest of the endpoints in `dscp-api` require authentication in the form of a header `'Authorization: Bearer YOUR_ACCESS_TOKEN'`:

1. [GET /item/:id](#get-/item/:id)
2. [GET /item/:id/metadata/:metadataKey](#get-/item/:id/metadata/:metadataKey)
@@ -153,7 +153,18 @@ The `inputs` field is an array of token `id`s that identifies the tokens to be c
The `outputs` field is an array of objects that describe tokens to be created by running this process. To destroy tokens without creating any new ones simply pass an empty array. Each output must reference a `roles` object containing a (key, value) pair for each role associated with the new token. The value is the `AccountId` for the role. At minimum, a token requires the default `Owner` role to be set. The following role keys are accepted:

```json
["Owner", "Customer", "AdditiveManufacturer", "Laboratory", "Buyer", "Supplier", "Reviewer"]
[
"Owner",
"Customer",
"AdditiveManufacturer",
"Laboratory",
"Buyer",
"Supplier",
"Reviewer",
"Optimiser",
"MemberA",
"MemberB"
]
```

Each output must also reference a `metadata` object containing a (key, value) pair for each metadata item associated with the new token. The following metadata value types are accepted:
13 changes: 5 additions & 8 deletions app/api-v3/api-doc.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
import env from '../env.js'

const { PORT, API_VERSION, API_MAJOR_VERSION, EXTERNAL_ORIGIN, EXTERNAL_PATH_PREFIX } = env

let url = EXTERNAL_ORIGIN || `http://localhost:${PORT}`
if (EXTERNAL_PATH_PREFIX) {
url = `${url}/${EXTERNAL_PATH_PREFIX}`
}
url = `${url}/${API_MAJOR_VERSION}`
const { PORT, API_VERSION, API_MAJOR_VERSION } = env

const apiDoc = {
openapi: '3.0.3',
@@ -16,7 +10,7 @@ const apiDoc = {
},
servers: [
{
url,
url: `http://localhost:${PORT}/${API_MAJOR_VERSION}`,
},
],
components: {
@@ -168,6 +162,9 @@ const apiDoc = {
Buyer: { $ref: '#/components/schemas/AccountId' },
Supplier: { $ref: '#/components/schemas/AccountId' },
Reviewer: { $ref: '#/components/schemas/AccountId' },
Optimiser: { $ref: '#/components/schemas/AccountId' },
MemberA: { $ref: '#/components/schemas/AccountId' },
MemberB: { $ref: '#/components/schemas/AccountId' },
},
additionalProperties: false,
required: ['Owner'],
2 changes: 0 additions & 2 deletions app/env.js
Original file line number Diff line number Diff line change
@@ -41,8 +41,6 @@ const vars = envalid.cleanEnv(process.env, {
SUBSTRATE_STATUS_TIMEOUT_MS: envalid.num({ default: 2 * 1000 }),
IPFS_STATUS_POLL_PERIOD_MS: envalid.num({ default: 10 * 1000 }),
IPFS_STATUS_TIMEOUT_MS: envalid.num({ default: 2 * 1000 }),
EXTERNAL_ORIGIN: envalid.str({ default: '' }),
EXTERNAL_PATH_PREFIX: envalid.str({ default: '' }),
})

export default {
8 changes: 2 additions & 6 deletions app/server.js
Original file line number Diff line number Diff line change
@@ -22,7 +22,7 @@ import url from 'url'
const __filename = url.fileURLToPath(import.meta.url)
const __dirname = path.dirname(__filename)

const { PORT, API_VERSION, API_MAJOR_VERSION, AUTH_TYPE, EXTERNAL_PATH_PREFIX } = env
const { PORT, API_VERSION, API_MAJOR_VERSION, AUTH_TYPE } = env

export async function createHttpServer() {
const requestLogger = pinoHttp({ logger })
@@ -116,11 +116,7 @@ export async function createHttpServer() {
},
}

app.use(
EXTERNAL_PATH_PREFIX ? `/${EXTERNAL_PATH_PREFIX}/${API_MAJOR_VERSION}/swagger` : `/${API_MAJOR_VERSION}/swagger`,
swaggerUi.serve,
swaggerUi.setup(null, options)
)
app.use(`/${API_MAJOR_VERSION}/swagger`, swaggerUi.serve, swaggerUi.setup(null, options))

// Sorry - app.use checks arity
// eslint-disable-next-line no-unused-vars
23 changes: 7 additions & 16 deletions app/util/appUtil.js
Original file line number Diff line number Diff line change
@@ -55,11 +55,6 @@ function formatHash(filestoreResponse) {
}

export const processRoles = async (roles) => {
const defaultRole = await indexToRole(0)
if (!roles[defaultRole]) {
throw new Error(`Roles must include default ${defaultRole} role. Roles: ${JSON.stringify(roles)}`)
}

if (await containsInvalidMembershipRoles(roles)) {
throw new Error(`Request contains roles with account IDs not in the membership list`)
}
@@ -75,7 +70,7 @@ export const processRoles = async (roles) => {

export async function getMaxMetadataCount() {
await api.isReady
return api.consts.simpleNFT.maxMetadataCount.toNumber()
return api.consts.utxoNFT.maxMetadataCount.toNumber()
}

const validMetadataValueTypes = new Set(['LITERAL', 'TOKEN_ID', 'FILE', 'NONE'])
@@ -195,7 +190,7 @@ const downloadFile = async (dirHash) => {

export async function getLastTokenId() {
await api.isReady
const lastTokenId = await api.query.simpleNFT.lastToken()
const lastTokenId = await api.query.utxoNFT.lastToken()

return lastTokenId ? parseInt(lastTokenId, 10) : 0
}
@@ -245,7 +240,7 @@ export async function runProcess(process, inputs, outputs) {
logger.debug('Running Transaction inputs: %j outputs: %j', inputs, relevantOutputs)
return new Promise((resolve, reject) => {
let unsub = null
api.tx.simpleNFT
api.tx.utxoNFT
.runProcess(process, inputs, relevantOutputs)
.signAndSend(alice, (result) => {
logger.debug('result.status %s', JSON.stringify(result.status))
@@ -260,9 +255,8 @@ export async function runProcess(process, inputs, outputs) {
reject(ProcessExtrinsicError(errors[0]))
}

const tokens = result.events
.filter(({ event: { method } }) => method === 'Minted')
.map(({ event: { data } }) => data[0].toNumber())
const processRanEvent = result.events.find(({ event: { method } }) => method === 'ProcessRan')
const tokens = processRanEvent?.event?.data?.outputs?.map((x) => x.toNumber())

unsub()
resolve(tokens)
@@ -304,7 +298,7 @@ function transformItem({ originalId, createdAt, destroyedAt, ...rest }) {

export async function getItem(tokenId) {
await api.isReady
const itemRaw = (await api.query.simpleNFT.tokensById(tokenId)).toJSON()
const itemRaw = (await api.query.utxoNFT.tokensById(tokenId)).toJSON()

if (!itemRaw) {
return null
@@ -363,15 +357,12 @@ export const getReadableMetadataKeys = (metadata) => {

export const validateInputIds = async (accountIds) => {
await api.isReady
const userId = keyring.addFromUri(USER_URI).address

return await accountIds.reduce(async (acc, id) => {
const uptoNow = await acc
if (!uptoNow || !id || !Number.isInteger(id)) return false

const { roles, id: echoId, children } = await getItem(id)
const defaultRole = await indexToRole(0)
if (roles[defaultRole] !== userId) return false
const { id: echoId, children } = await getItem(id)

return children === null && echoId === id
}, Promise.resolve(true))
Loading