Skip to content

Latest commit

 

History

History
229 lines (154 loc) · 6.29 KB

08-13_stages.md

File metadata and controls

229 lines (154 loc) · 6.29 KB

Stages (Concurrency)

Key Value
Author(s) Jordan.Brockopp
Reviewers Neal.Coleman, David.May, Emmanuel.Meinen, Kelly.Merrick
Date August 13th, 2019
Status Complete

Background

Please provide a summary of the new feature, redesign or refactor:

This feature will enable the ability to execute multiple tasks, concurrently, for users.

By having some type of YAML attribute that can be provided in a pipeline, this will enable users to utilize functionality that already exists in other CI solutions.

It will also give users the flexibility to improve their build performance and throughput by running any and all tasks concurrently that can support it.

Please briefly answer the following questions:

  1. Why is this required?
  • provide compatible functionality with other CI solutions
  • enable users to optimize performance and throughput of builds
  • provide a fan-in, fan-out mechanism in Vela
  1. If this is a redesign or refactor, what issues exist in the current implementation?

N/A

  1. Are there any other workarounds, and if so, what are the drawbacks?

N/A

  1. Are there any related issues? Please provide them below if any exist.

Design

Please describe your solution to the proposal. This includes, but is not limited to:

  • new/updated endpoints or url paths
  • new/updated configuration variables (environment, flags, files, etc.)
  • performance and user experience tradeoffs
  • security concerns or assumptions
  • examples or (pseudo) code snippets
  • stages will always run in parallel
  • Each stage is required to have a unique name
  • A single stage can contain one or many steps
  • steps will always execute sequentially
  • You can use the needs label to specify one stage's dependency on another

Backwards Compatibility

  • stages declaration is optional
  • needs declaration is optional

All pipelines that exist in Vela today will not need to be modified for this new feature.

The plan is to keep the initial steps only syntax to allow simple use cases that don't require concurrent execution of processes.

To be more forward, using the stages and needs declarations should only be necessary when trying to execute processes in parallel.

Why

It is understood that this might appear as a barrier to entry for some, but the need to run processes in parallel is viewed as an advanced use-case.

So the barrier is necessary to mitigate the misuse or misunderstanding of the purpose of stages and needs.

There is also a hope that by explicitly defining the stages, this will help both the consumers and the producers of pipelines gain an elevated understanding for how the pipeline will be executed.

This should lead to an increased ability for all parties involved to discover, create and troubleshoot advanced use-cases of pipelines.

Sample

version: "1"

stages:

  backend:
    steps:
      - name: install
        image: golang:latest
        commands:
          - go get ./...
      - name: test
        image: golang:latest
        commands:
          - go test ./...
      - name: build
        image: golang:latest
        commands:
          - go build

  frontend:
    steps:
      - name: install
        image: npm:latest
        commands:
          - npm install
      - name: test
        image: npm:latest
        commands:
          - npm run test
      - name: build
        image: npm:latest
        commands:
          - npm run build

  publish_backend:
    needs: [ backend ]
    steps:
      - name: publish
        image: target/vela-docker:latest
        parameters:
          dockerfile: Dockerfile.backend
          repo: octocat/hello-world
          tags: [ backend ]

  publish_frontend:
    needs: [ frontend ]
    steps:
      - name: publish
        image: target/vela-docker:latest
        parameters:
          dockerfile: Dockerfile.frontend
          repo: octocat/hello-world
          tags: [ frontend ]

  notify:
    needs: [ publish_backend, publish_frontend ]
    steps:
      - name: notify
        image: target/vela-slack:latest
        parameters:
          webhook: https://api.slack.com

Implementation

Please briefly answer the following questions:

  1. Is this something you plan to implement yourself?

Yes

  1. What's the estimated time to completion?

2 weeks

Please provide all tasks (gists, issues, pull requests, etc.) completed to implement the design:

Questions

Please list any questions you may have:

N/A