Pipelines#

A pipeline is a user-defined workflow for executing a set of pre-commit and/or post-commit jobs.

A pipeline coordinates validation, workflows, and container steps before and after a commit—similar to GitHub Actions and GitLab pipelines—adapted for EDA's transaction model.

A PipelineDefinition custom resource defines a pipeline. When you commit a transaction that matches a definition, EDA runs the pipeline.

Key concepts#

PipelineDefinition structure#

A PipelineDefinition resource has these main sections:

apiVersion: core.eda.nokia.com/v1
kind: PipelineDefinition
metadata:
  name: deployimage-pipeline
  namespace: eda-system
spec:
  enabled: true
  triggers:
    events:
      resourceTrigger:
        - type: [Update]
          gvk: {group: core.eda.nokia.com, version: v1, kind: TopoNode}
          fields: [spec.version, spec.nodeProfile]
  transaction:
    inputs:
      - name: nodes
        gvk: { group: core.eda.nokia.com, version: v1, kind: TopoNode}
        extract: "metadata.name"
        description: "get nodes field from toponodes"
        required: true
  jobs:
    - name: pre-check-interfaces
      workflow:
        group: interfaces.eda.nokia.com
        version: v1
        kind: CheckInterfaces
      with:
        - name: spec
          value:
            nodes: ${{ transaction.inputs.nodes }}
    - name: pre-check-bgp
      workflow:
        group: protocols.eda.nokia.com
        version: v2
        kind: CheckDefaultBgpPeers
      with:
        - name: spec
          value:
            nodes: ${{ transaction.inputs.nodes }}
    - name: pre-check-isl-ping
      workflow:
        group: fabrics.eda.nokia.com
        version: v1
        kind: IslPing
      with:
        - name: spec
          value:
            count: 10
            timeoutSeconds: 10
            addressFamily: "DualStack"
    - name: pre-check-system-ping
      workflow:
        group: routing.eda.nokia.com
        version: v1
        kind: SystemPing
      with:
        - name: spec
          value:
            nodes: ${{ transaction.inputs.nodes }}
            count: 10
            timeoutSeconds: 10
            addressFamily: "DualStack"
    - name: put-maintenance
      needs: [pre-check-system-ping, pre-check-isl-ping, pre-check-bgp, pre-check-interfaces]
      runsIn: eda-toolbox
      env:
        - name: NODES
          value: ${{ transaction.inputs.nodes }}
      steps:
        - id: resetMode
          run: |
            edactl -n $EDAPL_TRANSACTION_NAMESPACE patch toponodes.core.eda.nokia.com $NODES \
              -p '[{"op": "replace", "path": "/spec/npp/mode", "value": "maintenance"}]'

      # === commit barrier ===

    - name: reboot-and-wait
      needs: [commit]
      uses: wait-for-node-pipeline
      with:
        - name: namespace
          value: "eda"
        - name: nodeSelector
          value: ""
        - name: nodes
          value: ${{ transaction.inputs.nodes }}
        - name: waitFor
          value: 10
    - name: post-check-interfaces
      needs: [reboot-and-wait]
      compareWith: pre-check-interfaces
      if: ${{ needs.reboot-and-wait.outputs.synced == 'true' }}
      workflow:
        group: interfaces.eda.nokia.com
        version: v1
        kind: CheckInterfaces
      allowFailure: true
      with:
        - name: spec
          value:
            nodes: ${{ transaction.inputs.nodes }}
    - name: post-check-isl-ping
      needs: [reboot-and-wait, post-check-interfaces]
      compareWith: pre-check-isl-ping
      workflow:
        group: fabrics.eda.nokia.com
        version: v1
        kind: IslPing
      allowFailure: true
      with:
        - name: spec
          value:
            count: 10
            timeoutSeconds: 10
            addressFamily: "DualStack"
    - name: post-check-system-ping
      needs: [reboot-and-wait, post-check-interfaces, post-check-bgp]
      compareWith: pre-check-system-ping
      workflow:
        group: routing.eda.nokia.com
        version: v1
        kind: SystemPing
      allowFailure: true
      with:
        - name: spec
          value:
            nodes: ${{ transaction.inputs.nodes }}
            count: 10
            timeoutSeconds: 10
            addressFamily: "DualStack"
    - name: post-check-bgp
      needs: [reboot-and-wait, post-check-isl-ping]
      compareWith: pre-check-bgp
      workflow:
        group: protocols.eda.nokia.com
        version: v2
        kind: CheckDefaultBgpPeers
      allowFailure: true
      with:
        - name: spec
          value:
            nodes: ${{ transaction.inputs.nodes }}
    - name: remove-maintenance
      needs: [put-maintenance, post-check-system-ping, post-check-isl-ping, post-check-bgp, post-check-interfaces]
      runsIn: eda-toolbox
      if: ${{ always() && needs.put-maintenance.outcome == 'Success' }}
      env:
        - name: NODES
          value: ${{ transaction.inputs.nodes }}
      steps:
        - id: resetMode
          run: |
            edactl -n $EDAPL_TRANSACTION_NAMESPACE patch toponodes.core.eda.nokia.com $NODES \
               -p '[{"op": "replace", "path": "/spec/npp/mode", "value": "normal"}]'

Pipeline triggers#

Triggers define when a PipelineDefinition runs. A trigger can be a resource trigger (resourceTrigger) or a workflow call workflowCall.

Resource trigger (resourceTrigger)#

A resource trigger runs a pipeline when a transaction changes matching custom resources.

Within one trigger, field and label conditions are combined with AND. Multiple triggers on the same definition are combined with OR.

Example — match when either field changes (OR):

triggers:
  events:
    resourceTrigger:
      - type: [Update]
        gvk: { group: core.eda.nokia.com, version: v1, kind: TopoNode }
        fields: [spec.version]
      - type: [Update]
        gvk: { group: core.eda.nokia.com, version: v1, kind: TopoNode }
        fields: [spec.profile]

Example — match when both fields change (AND):

triggers:
  events:
    resourceTrigger:
      - type: [Create, Update]
        gvk: { group: core.eda.nokia.com, version: v1, kind: TopoNode }
        fields: [spec.version, spec.profile]

One transaction can match multiple PipelineDefinition resources, even across namespaces. All matches run under one TransactionPipelineRunner. While a definition is running, a new run of the same definition triggered by another transaction is queued in the WaitingToStart state until the current run finishes. By default, a queued run fails if it waits for more than 60 minutes.

Reusable pipeline (workflowCall)#

With a workflowCall trigger, another pipeline invokes this definition with uses. Reusable pipelines can be nested up to 10 levels deep. If a referenced PipelineDefinition is missing, the caller is marked invalid with a missing dependencies error, and is automatically re-validated when the missing definition is created.

The workflowCall type defines the contract between caller and callee:

• inputs: parameters the caller must/may provide, types (String, Number, Boolean), required, isArray, default
• outputs: values the callee produces, each with a type/value expression.

spec:
  enabled: true
  triggers:
    workflowCall:
      inputs:
        - name: namespace
          description: 'namespace'
          required: true
          inputType: String
        - name: nodeSelector
          description: 'Node selector'
          required: true
          inputType: String
        - name: nodes
          description: 'nodes whose BGP peers to check'
          required: true
          inputType: String
          isArray: true
        - name: waitFor
          description: 'wait for'
          required: true
          inputType: Number
        - name: status
          description: 'serialized status to compare with'
          required: false
          inputType: String
      outputs:
        - name: synced
          outputType: Boolean
          outputValue: ${{ jobs.wait-for-node.outputs.waitStatus }}

Transaction inputs#

Under transaction.inputs, define named values to extract from custom resources in the triggering transaction. Reference them in jobs with ${{ transaction.inputs.<name> }}.

Each input specifies:

When multiple CRs match the input's GVK, EDA collects all extracted values into a list.

Jobs#

A PipelineDefinition is made up of one or more jobs, which run in parallel by default. To run jobs sequentially, you can define dependencies on other jobs using the jobs.<name>.needs keyword. An EDA built-in job named commit exists, allowing jobs to execute themselves pre- and post-commit.

Declare a list of jobs under spec.jobs. Each item in the list requires a unique name field.

Job types#

A job uses only one of workflow, runsIn/container, or uses—they are mutually exclusive. A job that sets uses cannot also define steps or outputs.

Job fields#

  jobs:
    - name: pre-check-interfaces
      workflow:
        group: interfaces.eda.nokia.com
        version: v1
        kind: CheckInterfaces
      with:
        - name: spec
          value:
            nodes: ${{ transaction.inputs.nodes }}

  jobs:
    - name: put-maintenance
      runsIn: eda-toolbox
      env:
        - name: NODES
          value: ${{ transaction.inputs.nodes }}
      steps:
        - id: resetMode
          run: |
            edactl -n $EDAPL_TRANSACTION_NAMESPACE patch toponodes.core.eda.nokia.com $NODES \
              -p '[{"op": "replace", "path": "/spec/npp/mode", "value": "maintenance"}]'

  jobs:
    - name: reboot-and-wait
      uses: wait-for-node-pipeline
      with:
        - name: namespace
          value: "eda"
        - name: nodeSelector
          value: ""
        - name: nodes
          value: ${{ transaction.inputs.nodes }}
        - name: waitFor
          value: 10

runsIn job steps#

Outputs#

Pipelines can pass named string values:

• from one step to a later step in the same job
• from one job to downstream jobs.

Write step outputs to $EDAPL_OUTPUT as name=value lines. Reference them later with ${{ steps.<id>.outputs.<key> }}. A step can only read outputs of earlier steps—not its own, and not one that has yet to occur.

steps:
  - id: myUpstreamStep
    run: |
      echo "tag=1.2.3" >> $EDAPL_OUTPUT

Reference the previous step with ${{ steps.<step_id>.outputs.<key> }} in run, env or if

steps:
  - id: myUpstreamStep
    run: |
      echo "tag=1.2.3" >> $EDAPL_OUTPUT
  - id: myDownstreamStep
    run: |
      echo "deploying ${{ steps.myUpstreamStep.outputs.tag }}"
      echo "also accessible through deploying $ENV_TAG"
    env:
      - name: ENV_TAG
        value: "${{ steps.myUpstreamStep.outputs.tag }}"

A job exposes outputs to downstream jobs by mapping named job outputs to step outputs under:

jobs:
  - name: build
    runsIn: eda-toolbox
    outputs:
      - name: tag
        value: ${{ steps.myUpstreamStep.outputs.tag }}
    steps:
      - id: myUpstreamStep
        run: |
          echo "tag=1.2.3" >> $EDAPL_OUTPUT  

Phases and commit flow#

For resource-triggered pipelines, EDA partitions jobs automatically into pre-commit and post-commit phases around a commit barrier.

Execution flow:

Add commit to needs when a job must run after the commit:

- name: reboot-and-wait
  needs: [commit]
  workflow: { group: components.eda.nokia.com, version: v2, kind: Reboot }
  with:
    - name: spec
      value:
        nodes: ${{ transaction.inputs.nodes }}
        drains:
          skip: true
        prompts:
          skip: true

If no job lists commit in its needs field, the pipeline has no commit barrier: the transaction commit does not depend on the pipeline, and all jobs are considered post-commit.

Pre/post checks and scratchpads#

Run the same workflow before and after the commit to validate a change—for example, pre-check-interfaces and post-check-interfaces.

Pre-check workflows can store results in a scratchpad (arbitrary data keyed by workflow ID). Set compareWith on the post-check job to the pre-check job ID. Both jobs must use the same workflow GVK. EDA passes the pre-check workflow ID to the post-check run so it can read that scratchpad and compare results.

- name: post-check-interfaces
  needs: [reboot-and-wait]
  compareWith: pre-check-interfaces
  workflow: { group: interfaces.eda.nokia.com, version: v1, kind: CheckInterfaces }

Environment variables#

A list of name/value of variables that are available to the steps of all jobs in the PipelineDefinition. You can also set variables that are only available to the steps of a single job or to a single step.

When more than one environment variable is defined with the same name, EDA uses the most specific variable. For example, an environment variable defined in a step will override job and PipelineDefinition environment variables with the same name, while the step executes. An environment variable defined for a job will override a PipelineDefinition global variable with the same name, while the job executes.

Set variables at three levels (most specific wins):

Names must start with a letter or _, contain only letters, digits, and _, and must not use the reserved prefix EDAPL_. The prefix check is case-sensitive; lowercase names such as edapl_custom are valid.

Default variables#

Expressions#

Use ${{ ... }} in env, run, with, if, and outputs.

Operators#

Status functions#

If conditions#

You can use the jobs.<name>.if or jobs.<name>.steps[*].if conditional to evaluate a boolean expression. If the boolean condition evaluates to false, the job or step is skipped (not failed), and the rest of the pipeline keeps going.

When no if statement is defined, a job or step behaves as success() and depends on prior (dependent) jobs or all prior steps to be succeeded.

if: ${{ needs.<job_name>.result == 'failure' }}

if: ${{ success() && needs.<job_name>.result == 'failure' }}

if: ${{ always() && needs.<job_name>.result == 'failure' }}

Skipping pipelines#

You can skip pipelines to make changes without waiting for pipeline completion.

• From the UI, select Commit without pipeline from the following operations:

• Transactions basket

• deleting a resource confirmation prompt
• schema form confirmation prompt

• merge request confirmation prompt

• When reverting a transaction from the UI, select Revert without pipeline.

• In the API, set skipPipeline: true on /apps or /transaction POST requests (the default is false).

• In edactl, add the --skip-pipeline option, which is available on the edactl apply, replace, create, delete, and patch commands, as well as on edactl mr create, edactl mr merge, and edactl mr dry-run.

Pipelines in edactl#

You can monitor pipeline runs using edactl, the Nokia EDA CLI:

• edactl pipeline: lists all pipeline runners, including their runner ID, the transaction ID that triggered them, and their status.

• edactl pipeline <runner-id>: shows the details of a single pipeline runner, including the jobs of each pipeline, partitioned into their pre-commit and post-commit phases.

Failure scenarios#

If Pipeline Engine or ConfigEngine fails, the outcome depends on the active phase.

Pipeline state#

You can monitor runs in the UI, CLI, or API.

The PipelineRun status includes pipeline state, per-job status, phase status, timestamps, and references to workflows and child pipelines.

Working with pipelines in the UI#

From the Main navigation panel, click Pipelines to view pipeline runs.

Table: Pipelines page

The Pipelines page displays a datagrid with the following information:

From the Pipelines page, double-click a runner to display the summary for that run or click the transaction ID to go to the transaction that triggered the pipeline.

The Runner Summary page#

The Runner Summary page includes a Runner Summary panel and a Jobs panel.

The Runner Summary panel shows run state, commit result, and links to phase and job details for the selected runner.

The Jobs panel lists all jobs per phase and whether each job succeeded, failed, or was skipped. Click the expand icon to display all jobs per phase.