Authoring Azure DevOps Pipelines
A short blog on the key concepts and components that make up a pipeline. And understanding how to get started, what tools to use and what links to follow.
- A DevOps project and a blank YAML file in the repo.
- VS Code installed.
- Azure Pipelines extension installed (Ref. https://marketplace.visualstudio.com/items?itemName=ms-azure-devops.azure-pipelines)
- Some patience with code 🤣
- A trigger tells a Pipeline to run.
- A pipeline is made up of one or more stages. A pipeline can deploy to one or more environments.
- A stage is a way of organizing jobs in a pipeline and each stage can have one or more jobs.
- Each job runs on one agent. A job can also be agentless.
- Each agent runs a job that contains one or more steps.
- A step can be a task or script and is the smallest building block of a pipeline.
- A task is a pre-packaged script that performs an action, such as invoking a REST API or publishing a build artifact.
- An artifact is a collection of files or packages published by a run.
Getting Started – Creating a Pipeline:
Step 1: Clone the DevOps repo (where you have the YAML file created) to your system and open VS Code.
and ensure the pipeline (YAML file) is visible:
Step 2: Choosing Triggers & Pool
Triggers to run a pipeline automatically. Azure Pipelines supports many types of triggers. Based on your pipeline’s type, select the appropriate trigger from the list below:
- Scheduled triggers are independent of the repository and allow you to run a pipeline according to a schedule.
- Pipeline triggers in YAML pipelines and build completion triggers in classic build pipelines allow you to trigger one pipeline upon the completion of another.
- Continuous deployment triggers help you start classic releases after a classic build or YAML pipeline completes.
- Pull request release triggers are used to deploy a pull request directly using classic releases.
Agent pool is needed to build your code or deploy your software using Azure Pipelines (you need at least one agent). As you add more code and people, you’ll eventually need more.
When your pipeline runs, the system begins one or more jobs. An agent is a computing infrastructure with installed agent software that runs one job at a time.
These can be of two types: Microsoft-hosted agents and Self-hosted agents
More details here: Azure Pipelines Agents – Azure Pipelines | Microsoft Docs
Step 3: Using Resources for Azure Pipelines
Azure DevOps Pipelines provide you with the ability to share inputs, secure your data and create connections. Resources help in a few ways:
- Ways to share something such as a secure file or password across pipelines.
- Examples of using resources for sharing are variable groups, secure files, and service connections. In all cases, you’re using a resource as a way for a pipeline to access and consume something.
- A tool for enhancing security through access checks and other restrictions.
- For example, you can limit a service connection to only run on one pipeline. You could also make sure that a repository can only be accessed from a pipeline after a manual approach check.
- Ways to improve traceability for your pipeline and make it easier to troubleshoot environments.
- For example, you can see the number of the last run that was deployed to an environment.
Step 4: Understanding what’s needed – ie. multiple stages or multiple steps with tasks or a mix of both.
Like in the below example, we have different stages and each stage has a task related to it.
Option 1: Single-Stage Single Job Pipeline.
For single jobs, you don’t have to specify the stages, instead, you can straightway define the steps and then the task under it.
In the below code, I am trying to deploy multiple Azure Resources, starting with APIM and then networking component for a subnet and will gradually peer it with the main subnet. Then will deploy some random resources like KeyVault and SQL.
In the below example, the top 4 inputs (green block) remains the same for any type of pipeline/number of the jobs but the inputs below that would change depending on the tasks.
Option 2: Single-stage, multiple jobs:
In this type of pipeline, the first part remains the same but has multiple tasks as shown below:
Option 3: Single-stage, multiple jobs:
This can be useful when deploying in multiple environments, like non-prod state followed by prod etc.
And we can also have control, checks and approvals between two stages.
Step 5: The complete Pipeline !!
In the last few steps we covered:
- What tool to use (VS Code/Extension)
- How Pipeline would run (triggers)
- Where Pipeline will run (Agent)
- What options do we have to run the code or execute steps
Now, the last step is: What all we can do?
Using VS Code + the extension that we discussed earlier, we have an option of Intellisense
As you type the ‘task’ using IntelliSense, you will have all the options for tasks that you can do using the pipeline.
So, let’s take an example of ARM Deployment:
I choose the template deployment option:
And then supply the inputs.
Using IntelliSense – I will get all the options that are needed:
Though the tool does not informs you about what’s required or optional, but you can use the task name and perhaps search GitHub or MS Docs to see what’s needed.
For example, for AzureResourceGroupDeployment@2 task you can use MS Docs and check what’s required to run the task:
Once you have completed the pipeline, you can run it and any issues should be reported in the Azure DevOps screen with an error.