Skip to content

Latest commit

 

History

History
751 lines (627 loc) · 21.8 KB

tasks.md

File metadata and controls

751 lines (627 loc) · 21.8 KB

Tasks

A Task (or a ClusterTask) is a collection of sequential steps you would want to run as part of your continuous integration flow. A task will run inside a pod on your cluster.

A Task declares:

A Task is available within a namespace, and ClusterTask is available across entire Kubernetes cluster.

ClusterTask

Similar to Task, but with a cluster scope.

In case of using a ClusterTask, the TaskRef kind should be added. The default kind is Task which represents a namespaced Task

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: demo-pipeline
  namespace: default
spec:
  tasks:
    - name: build-skaffold-web
      taskRef:
        name: build-push
        kind: ClusterTask
      params: ....

A Task functions exactly like a ClusterTask, and as such all references to Task below are also describing ClusterTask.

Syntax

To define a configuration file for a Task resource, you can specify the following fields:

  • Required:
    • apiVersion - Specifies the API version, for example tekton.dev/v1alpha1.
    • kind - Specify the Task resource object.
    • metadata - Specifies data to uniquely identify the Task resource object, for example a name.
    • spec - Specifies the configuration information for your Task resource object. Task steps must be defined through either of the following fields:
      • steps - Specifies one or more container images that you want to run in your Task.
  • Optional:
    • inputs - Specifies parameters and PipelineResources needed by your Task
    • outputs - Specifies PipelineResources created by your Task
    • volumes - Specifies one or more volumes that you want to make available to your Task's steps.
    • stepTemplate - Specifies a Container step definition to use as the basis for all steps within your Task.
    • sidecars - Specifies sidecar containers to run alongside steps.

The following example is a non-working sample where most of the possible configuration fields are used:

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: example-task-name
spec:
  inputs:
    resources:
      - name: workspace
        type: git
    params:
      - name: pathToDockerFile
        type: string
        description: The path to the dockerfile to build
        default: /workspace/workspace/Dockerfile
  outputs:
    resources:
      - name: builtImage
        type: image
  steps:
    - name: ubuntu-example
      image: ubuntu
      args: ["ubuntu-build-example", "SECRETS-example.md"]
    - image: gcr.io/example-builders/build-example
      command: ["echo"]
      args: ["$(inputs.params.pathToDockerFile)"]
    - name: dockerfile-pushexample
      image: gcr.io/example-builders/push-example
      args: ["push", "$(outputs.resources.builtImage.url)"]
      volumeMounts:
        - name: docker-socket-example
          mountPath: /var/run/docker.sock
  volumes:
    - name: example-volume
      emptyDir: {}

Steps

The steps field is required. You define one or more steps fields to define the body of a Task.

If multiple steps are defined, they will be executed in the same order as they are defined, if the Task is invoked by a TaskRun.

Each steps in a Task must specify a container image that adheres to the container contract. For each of the steps fields, or container images that you define:

  • The container images are run and evaluated in order, starting from the top of the configuration file.
  • Each container image runs until completion or until the first failure is detected.
  • The CPU, memory, and ephemeral storage resource requests will be set to zero if the container image does not have the largest resource request out of all container images in the Task. This ensures that the Pod that executes the Task will only request the resources necessary to execute any single container image in the Task, rather than requesting the sum of all of the container image's resource requests.

Step Script

To simplify executing scripts inside a container, a step can specify a script. If this field is present, the step cannot specify command or args.

When specified, a script gets invoked as if it were the contents of a file in the container. Scripts should start with a shebang line to declare what tool should be used to interpret the script. That tool must then also be available within the step's container.

This allows you to execute a Bash script, if the image includes bash:

steps:
- image: ubuntu  # contains bash
  script: |
    #!/usr/bin/env bash
    echo "Hello from Bash!"

...or to execute a Python script, if the image includes python:

steps:
- image: python  # contains python
  script: |
    #!/usr/bin/env python3
    print("Hello from Python!")

...or to execute a Node script, if the image includes node:

steps:
- image: node  # contains node
  script: |
    #!/usr/bin/env node
    console.log("Hello from Node!")

This also simplifies executing script files in the workspace:

steps:
- image: ubuntu
  script: |
    #!/usr/bin/env bash
    /workspace/my-script.sh  # provided by an input resource

...or in the container image:

steps:
- image: my-image  # contains /bin/my-binary
  script: |
    #!/usr/bin/env bash
    /bin/my-binary

Inputs

A Task can declare the inputs it needs, which can be either or both of:

Parameters

Tasks can declare input parameters that must be supplied to the task during a TaskRun. Some example use-cases of this include:

  • A Task that needs to know what compilation flags to use when building an application.
  • A Task that needs to know what to name a built artifact.

Parameters name are limited to alpha-numeric characters, - and _ and can only start with alpha characters and _. For example, fooIs-Bar_ is a valid parameter name, barIsBa$ or 0banana are not.

Each declared parameter has a type field, assumed to be string if not provided by the user. The other possible type is array — useful, for instance, when a dynamic number of compilation flags need to be supplied to a task building an application. When the actual parameter value is supplied, its parsed type is validated against the type field.

Usage

The following example shows how Tasks can be parameterized, and these parameters can be passed to the Task from a TaskRun.

Input parameters in the form of $(inputs.params.foo) are replaced inside of the steps (see also variable substitution).

The following Task declares an input parameter called 'flags', and uses it in the steps.args list.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: task-with-parameters
spec:
  inputs:
    params:
      - name: flags
        type: array
      - name: someURL
        type: string
  steps:
    - name: build
      image: my-builder
      args: ["build", "$(inputs.params.flags)", "url=$(inputs.params.someURL)"]

The following TaskRun supplies a dynamic number of strings within the flags parameter:

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: run-with-parameters
spec:
  taskRef:
    name: task-with-parameters
  inputs:
    params:
      - name: flags
        value: 
          - "--set"
          - "arg1=foo"
          - "--randomflag"
          - "--someotherflag"
      - name: someURL
        value: "http://google.com"

Input resources

Use input PipelineResources field to provide your Task with data or context that is needed by your Task. See the using resources docs.

Outputs

Task definitions can include inputs and outputs PipelineResource declarations. If specific set of resources are only declared in output then a copy of resource to be uploaded or shared for next Task is expected to be present under the path /workspace/output/resource_name/.

outputs:
  resources:
    name: storage-gcs
    type: gcs
steps:
  - image: objectuser/run-java-jar #https://hub.docker.com/r/objectuser/run-java-jar/
    command: [jar]
    args:
      ["-cvf", "-o", "/workspace/output/storage-gcs/", "projectname.war", "*"]
    env:
      - name: "FOO"
        value: "world"

note: if the task is relying on output resource functionality then the containers in the task steps field cannot mount anything in the path /workspace/output.

In the following example Task tar-artifact resource is used both as input and output so input resource is downloaded into directory customworkspace(as specified in targetPath). Step untar extracts tar file into tar-scratch-space directory , edit-tar adds a new file and last step tar-it-up creates new tar file and places in /workspace/customworkspace/ directory. After execution of the Task steps, (new) tar file in directory /workspace/customworkspace will be uploaded to the bucket defined in tar-artifact resource definition.

inputs:
  resources:
    name: tar-artifact
    targetPath: customworkspace
outputs:
  resources:
    name: tar-artifact
steps:
 - name: untar
    image: ubuntu
    command: ["/bin/bash"]
    args: ['-c', 'mkdir -p /workspace/tar-scratch-space/ && tar -xvf /workspace/customworkspace/rules_docker-master.tar -C /workspace/tar-scratch-space/']
 - name: edit-tar
    image: ubuntu
    command: ["/bin/bash"]
    args: ['-c', 'echo crazy > /workspace/tar-scratch-space/rules_docker-master/crazy.txt']
 - name: tar-it-up
   image: ubuntu
   command: ["/bin/bash"]
   args: ['-c', 'cd /workspace/tar-scratch-space/ && tar -cvf /workspace/customworkspace/rules_docker-master.tar rules_docker-master']

Volumes

Specifies one or more volumes that you want to make available to your Task, including all the steps. Add volumes to complement the volumes that are implicitly created for input resources and output resources.

For example, use volumes to accomplish one of the following common tasks:

  • Mount a Kubernetes secret.
  • Create an emptyDir volume to act as a cache for use across multiple build steps. Consider using a persistent volume for inter-build caching.
  • Mount Kubernetes configmap as volume source.
  • Mount a host's Docker socket to use a Dockerfile for container image builds. Note: Building a container image using docker build on-cluster is very unsafe. Use kaniko instead. This is used only for the purposes of demonstration.

Step Template

Specifies a Container configuration that will be used as the basis for all steps in your Task. Configuration in an individual step will override or merge with the step template's configuration.

In the example below, the Task specifies a stepTemplate with the environment variable FOO set to bar. The first step will use that value for FOO, but in the second step, FOO is overridden and set to baz.

stepTemplate:
  env:
    - name: "FOO"
      value: "bar"
steps:
  - image: ubuntu
    command: [echo]
    args: ["FOO is ${FOO}"]
  - image: ubuntu
    command: [echo]
    args: ["FOO is ${FOO}"]
    env:
      - name: "FOO"
        value: "baz"

Sidecars

Specifies a list of Containers to run alongside your Steps. These containers can provide auxiliary functions like Docker in Docker or running a mock API server for your app to hit during tests.

Sidecars are started before your Task's steps are executed and are torn down after all steps have completed. For further information about a sidecar's lifecycle see the TaskRun doc.

In the example below, a Docker in Docker sidecar is run so that a step can use it to build a docker image:

steps:
  - image: docker
    name: client
    script: |
        #!/usr/bin/env bash
        cat > Dockerfile << EOF
        FROM ubuntu
        RUN apt-get update
        ENTRYPOINT ["echo", "hello"]
        EOF
        docker build -t hello . && docker run hello
        docker images
    volumeMounts:
      - mountPath: /var/run/
        name: dind-socket
sidecars:
  - image: docker:18.05-dind
    name: server
    securityContext:
      privileged: true
    volumeMounts:
      - mountPath: /var/lib/docker
        name: dind-storage
      - mountPath: /var/run/
        name: dind-socket
volumes:
  - name: dind-storage
    emptyDir: {}
  - name: dind-socket
    emptyDir: {}

Note: There is a known bug with Tekton's existing sidecar implementation. Tekton uses a specific image, called "nop", to stop sidecars. The "nop" image is configurable using a flag of the Tekton controller. If the configured "nop" image contains the command that the sidecar was running before the sidecar was stopped then the sidecar will actually keep running, causing the TaskRun's Pod to remain running, and eventually causing the TaskRun to timeout rather then exit successfully. Issue tektoncd#1347 has been created to track this bug.

Variable Substitution

Tasks support string replacement using values from all inputs and outputs.

Input parameters can be referenced in the Task spec using the variable substitution syntax below, where <name> is the name of the parameter:

$(inputs.params.<name>)

Param values from resources can also be accessed using variable substitution

Variable Substitution with Parameters of Type Array

Referenced parameters of type array will expand to insert the array elements in the reference string's spot.

So, with the following parameter:

inputs:
    params:
      - name: array-param
        value:
          - "some"
          - "array"
          - "elements"

then command: ["first", "$(inputs.params.array-param)", "last"] will become command: ["first", "some", "array", "elements", "last"]

Note that array parameters must be referenced in a completely isolated string within a larger string array. Any other attempt to reference an array is invalid and will throw an error.

For instance, if build-args is a declared parameter of type array, then this is an invalid step because the string isn't isolated:

 - name: build-step
      image: gcr.io/cloud-builders/some-image
      args: ["build", "additionalArg $(inputs.params.build-args)"]

Similarly, referencing build-args in a non-array field is also invalid:

 - name: build-step
      image: "$(inputs.params.build-args)"
      args: ["build", "args"]

A valid reference to the build-args parameter is isolated and in an eligible field (args, in this case):

 - name: build-step
      image: gcr.io/cloud-builders/some-image
      args: ["build", "$(inputs.params.build-args)", "additonalArg"]

Variable Substitution within Volumes

Task volume names and different types of volumes can be parameterized. Current support includes for widely used types of volumes like configmap, secret and PersistentVolumeClaim. Here is an example on how to use this in Task definitions.

Examples

Use these code snippets to help you understand how to define your Tasks.

Tip: See the collection of simple examples for additional code samples.

Example Task

For example, a Task to encapsulate a Dockerfile build might look something like this:

Note: Building a container image using docker build on-cluster is very unsafe. Use kaniko instead. This is used only for the purposes of demonstration.

spec:
  inputs:
    resources:
      - name: workspace
        type: git
    params:
      # These may be overridden, but provide sensible defaults.
      - name: directory
        type: string
        description: The directory containing the build context.
        default: /workspace
      - name: dockerfileName
        type: string
        description: The name of the Dockerfile
        default: Dockerfile
  outputs:
    resources:
      - name: builtImage
        type: image
  steps:
    - name: dockerfile-build
      image: gcr.io/cloud-builders/docker
      workingDir: "$(inputs.params.directory)"
      args:
        [
          "build",
          "--no-cache",
          "--tag",
          "$(outputs.resources.image)",
          "--file",
          "$(inputs.params.dockerfileName)",
          ".",
        ]
      volumeMounts:
        - name: docker-socket
          mountPath: /var/run/docker.sock

    - name: dockerfile-push
      image: gcr.io/cloud-builders/docker
      args: ["push", "$(outputs.resources.image)"]
      volumeMounts:
        - name: docker-socket
          mountPath: /var/run/docker.sock

  # As an implementation detail, this Task mounts the host's daemon socket.
  volumes:
    - name: docker-socket
      hostPath:
        path: /var/run/docker.sock
        type: Socket

Using an extra volume

Mounting multiple volumes:

spec:
  steps:
    - image: ubuntu
      script: |
        #!/usr/bin/env bash
        curl https://foo.com > /var/my-volume
      volumeMounts:
        - name: my-volume
          mountPath: /var/my-volume

    - image: ubuntu
      script: |
        #!/usr/bin/env bash
        cat /etc/my-volume
      volumeMounts:
        - name: my-volume
          mountPath: /etc/my-volume

  volumes:
    - name: my-volume
      emptyDir: {}

Using Kubernetes Configmap as Volume Source

spec:
  inputs:
    params:
      - name: CFGNAME
        type: string
        description: Name of config map
      - name: volumeName
        type: string
        description: Name of volume
  steps:
    - image: ubuntu
      script: |
        #!/usr/bin/env bash
        cat /var/configmap/test
      volumeMounts:
        - name: "$(inputs.params.volumeName)"
          mountPath: /var/configmap

  volumes:
    - name: "$(inputs.params.volumeName)"
      configMap:
        name: "$(inputs.params.CFGNAME)"

Using secret as environment source

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: goreleaser
spec:
  inputs:
    params:
    - name: package
      type: string
      description: base package to build in
    - name: github-token-secret
      type: string
      description: name of the secret holding the github-token
      default: github-token
    resources:
    - name: source
      type: git
      targetPath: src/$(inputs.params.package)
  steps:
  - name: release
    image: goreleaser/goreleaser
    workingdir: /workspace/src/$(inputs.params.package)
    command:
    - goreleaser
    args:
    - release
    env:
    - name: GOPATH
      value: /workspace
    - name: GITHUB_TOKEN
      valueFrom:
        secretKeyRef:
          name: $(inputs.params.github-token-secret)
          key: bot-token

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License.

Debugging

In software, we do things not because they are easy, but because we think they will be. Lots of things can go wrong when writing a Task. This section contains some tips on how to debug one.

Inspecting the Filesystem

One common problem when writing Tasks is not understanding where files are on disk. For the most part, these all live somewhere under /workspace, but the exact layout can be tricky. To see where things are before your task runs, you can add a step like this:

- name: build-and-push-1
  image: ubuntu
  command:
  - /bin/bash
  args:
  - -c
  - |
    set -ex
    find /workspace

This step will output the name of every file under /workspace to your build logs.

To see the contents of every file, you can use a similar step:

- name: build-and-push-1
  image: ubuntu
  command:
  - /bin/bash
  args:
  - -c
  - |
    set -ex
    find /workspace | xargs cat

These steps are useful both before and after your Task steps!

Inspecting the pod

One task will map to one Pod, to check arbitrary thing in Pod, the best way is to login the pod, add a step at the position you want to pause the task, then checking.

- name: pause
  image: docker
  args: ["sleep", "6000"]