Triggering a YAML Pipeline in Azure DevOps Automatically After Another Pipeline Completes

Introduction

In modern CI/CD workflows, it’s common to have interconnected pipelines where the successful completion of one pipeline triggers the execution of another. Azure DevOps makes this seamless with YAML pipelines. This guide explains how to set up automatic pipeline triggering in Azure DevOps using YAML.

This use case is beneficial in the following scenarios:

  • Dependency management: When an upstream component (such as a library) changes, downstream dependencies must be rebuilt and revalidated.
  • Multi-stage deployments: In situations where different application components are built and deployed separately, you can ensure that dependent components are updated only after the core components have been successfully built and tested.
  • Cross-project integration: This feature allows you to trigger pipelines across different projects within Azure DevOps, enabling better coordination between teams or components.
  • Sequential workflows: In cases where certain processes must occur in a specific order, pipeline triggers ensure that each step is executed only after the previous one has been completed successfully.

Prerequisites for setting up Azure DevOps pipelines

  1. Azure DevOps Organization: Ensure you have access to an Azure DevOps organization and a project.
  2. Two Pipelines: Create two YAML-based pipelines:
    • Source Pipeline: The pipeline whose completion will trigger another pipeline.
    • Target Pipeline: The pipeline is triggered upon the source pipeline’s completion.
  3. Permissions: Ensure appropriate permissions to access and edit both pipelines.

Step 1: Define the Source Pipeline

The source pipeline is your primary pipeline. Configure it as usual for building, testing, or deploying your application. No specific modifications are needed in the YAML to trigger another pipeline.

Example:

# First deployment YAML pipeline
steps:
- bash: echo "The First deployment pipeline runs first"

Step 2: Configure the Target Pipeline to Be Triggered

To automatically trigger the target pipeline when the source pipeline completes:

  1. Open the YAML file for the target pipeline.
  2. Add the resources block to reference the source pipeline.
  3. Define the trigger under the resources block.

Example:

# app-ci YAML pipeline
# We are setting up a pipeline resource that references the security-lib-ci
# pipeline and setting up a pipeline completion trigger so that our app-ci
# pipeline runs when a run of the FirstDeployment-ci pipeline completes
resources:
  pipelines:
  - pipeline: FirstDeployment-ci # Name of the pipeline resource.
    source: FirstDeployment-ci # The name of the pipeline referenced by this pipeline resource.
    trigger: true # Run app-ci pipeline when any run of security-lib-ci completes

steps:
- bash: echo "app-ci runs after security-lib-ci completes"
Explanation:

pipeline: The alias used to reference the source pipeline.
project: The name of the project where the source pipeline exists.
source: The name of the source pipeline.
trigger.branches.include: Specifies which branches in the source pipeline should trigger the target pipeline. Replace main with your branch name if different.

Step 3: Add Steps to the Target Pipeline

Once triggered, the target pipeline will execute its tasks as defined in its YAML file.

Example:

trigger: none

resources:
  pipelines:
    - pipeline: sourcePipeline
      project: "MyProject"
      source: "SourcePipeline"
      trigger:
        branches:
          include:
            - main

pool:
  vmImage: 'ubuntu-latest'

steps:
- script: echo "Triggered by the completion of the source pipeline"
Notes:
  • trigger: none: Ensures the target pipeline is not triggered by code changes directly but only by the source pipeline.
  • Customize the steps section as per the requirements of the target pipeline.

Step 4: Verify the Setup

  1. Commit and push the YAML files for both pipelines.
  2. Trigger the source pipeline manually or by committing changes to the monitored branch (e.g., main).
  3. Upon completion of the source pipeline, check the target pipeline for automatic execution.
  4. Review the logs in Azure DevOps to confirm the workflows.

Advanced Configuration

Conditional Execution

If the target pipeline should run only under specific conditions (e.g., based on the result of the source pipeline), you can use pipeline variables:

jobs:
- job: TriggeredJob
  condition: eq(dependencies.sourcePipeline.result, 'Succeeded')
  steps:
  - script: echo "Executed because the source pipeline succeeded"

Multi-Stage Pipelines

For larger projects, consider combining multiple stages into a single YAML file to reduce complexity and dependencies.

Common Issues and Troubleshooting

  1. Pipeline Not Triggering: Verify the project and pipeline names in the resources block.
  2. Permissions: Ensure the source pipeline has permission to trigger the target pipeline. Check the service connection and user access.
  3. Branch Filtering: Confirm the branch filters match your source pipeline’s trigger branch.
  4. Circular Dependency: Avoid circular triggers where two pipelines trigger each other indefinitely.

Conclusion

Azure DevOps YAML pipelines offer flexible automation for complex CI/CD workflows. The proper configuration enables seamless, automated pipeline orchestration.