Continuous Integration And Deployment With Azure DevOps Pipelines

6.6K

I have talked about Azure DevOps before and it made me realize there is so much more to tell about this tool. Well, it’s actually a system with tools. The previous time I told you about Artifacts. But let me also tell you about continuous integration and deployment with Azure DevOps Pipelines. It’s a great tool for CI/CD, which stands for continuous integration and delivery.

Azure DevOps Pipelines Explained

As usual, first a bit of boring theory which gives you some background information about what we are looking at. Azure DevOps is the agile solution or tool made by Microsoft. It is a Swiss army knife with all kinds of tools you can use to boost your productivity. I am not affiliated with Microsoft or Azure DevOps, so I am allowed to say that not everything is perfect.

But the Pipelines are a great tool if you want to automate the process of distributing your website. And not only distribute, but you can also use it to run unit tests each time you are pushing code to your code repository. You can do a lot with Pipelines.

Imagine you have a website and you are constantly updating it with new features. Publishing a website by hand is a hassle and sometimes takes a long time. Wouldn’t it be great if you could automate this? With a few clicks, you can make Azure DevOps Pipelines do this for you.

This article focuses mostly on testing and distributing an API to a host.

Preparation

I have created a very simple minimal API with .NET 7. At the end of this article, I want a full CI/CD pipeline that builds the code (checks for build errors), runs unit tests, and publishes the code to a host. I will be using Azure Cloud. It is also possible to use a different cloud host or a not-cloud host.

You can download the project here:

This solution has three projects: The API, a business layer with “all” the logic, and an xUnit test project that tests the logic of the business. You can run the unit tests or run the API. Note that one unit test will fail, this is intended.

You must put this code in an Azure DevOps GIT or a GitHub repository. If you don’t, the pipeline cannot reach your code. The code in this article is stored in an Azure DevOps GIT repo.

Creating A Pipeline

When your code is in a repository (Azure DevOps Git or GitHub) it’s pretty easy to create a pipeline. It all starts with creating one. This happens in Azure DevOps. When you have your project selected you see a menu on the left side. One of the options is “Pipelines”. Guess which one you’ll have to click…?

To create and execute a pipeline we need to follow a few steps:

• Tell Azure DevOps where to get the code
• Create tasks that should be executed in a particular order
• Send the compiled code to Azure Cloud (or other hosts)

There are two ways of achieving this: Through the use of YAML or the classic editor. For this article, I am going to use the classic editor.

Creating a pipeline is just a one-time job. When you have set up a pipeline and all works well, you don’t have to change it anymore.

Hit the Create Pipeline button to see the next screen.

Locate Your Code

Azure DevOps wants to know where you have stored your code. Seems logical since the code is the starting point of this whole endeavor. The 3 biggest parties are represented: Azure DevOps Git, Bitbucket, and GitHub. But these options will generate a pipeline using YAML, and this article is all about the classical editor.

Select the Use the classic editor link at the bottom.

Select The Source

You still need to select the location of your code. In this screen, you have the same options as the previous window, but this time we really need to select one. My code is in the Azure Repos Git. If your code is located somewhere else, select the one that fits your needs.

If you want to select a GitHub repo, you need to do some extra steps.

The Azure Repos Git has a few options that are basically set to the correct values. The Team Project is the project that has the Git repository. Since a project can have multiple Git repositories you need to select the correct one at Repository. And since a repository can have multiple branches, select the correct branch in the last dropdown.

After you have selected the correct values, press the button Continue.

Templates

If you have worked with Microsoft tools before you might already know that Microsoft loves templates. The next screen shows us a lot of templates. These are basically presets of a whole pipeline with a lot of tasks. It makes our lives way easier.

Most templates are about building and testing the code. Then they do specific tasks depending on your code and goal. There are templates for Android, Docker, Maven, Python, and much, much more.

I want to publish the API to an Azure Web App, so I could select the template Azure Web App For ASP.NET. However, I am going to create an empty job and add the tasks one by one.

Setting Up The Pipeline

Now the real work begins. A pipeline is divided into agents and each agent has tasks. These tasks make the pipeline do things like building the code, running unit tests, and publishing the code. By default, you get one agent, which is usually enough.

We start by setting up the Pipeline itself. It’s recommended to give it a good, useful name. I will call mine PipelineDemo-CICD.

The agent pool is where you can set the location of where the pipeline should run, or actually the agents. There are two options: Azure Pipelines and Default (no agents). It is possible to add more options: Your own pools. You can run the agents on your own machine if you want. These are called self-hosted agents and they are pretty easy to set up. But for now, keep the Azure Pipelines.

The Agent Specification is basically the virtual machine that is being used to build, run, and test your code. The default value is windows-2019, but I set it to windows-latest.

Agent Jobs

Yes, I skip the Get sources. This contains settings like where your code is located, the team project, and more things we already set.

You can have multiple agent jobs. Each with its own purpose, but I will be using just one job. You don’t have to change anything here, except for the Display name maybe.

Behind the name of the job (left side) is a big plus sign. If you press this one you can add tasks to the job.

Adding Tasks

Tasks can be all kinds of things. Build code, run unit tests, publish compiled code, archive files, send email, run Azure functions, and much more.

Some of the tasks I am going to add are usable for most .NET projects. Some are specifically for my project/solution. The idea is that you get some feeling about how to work with Azure DevOps Pipelines.

.NET Core

The first task is a task called .NET Core. Actually, I am going to use this one three times. Find the task (it’s usually on top of the list) and drag it to Agent job 1.

It looks scary, but it isn’t. Each task has different options and settings. It depends on the type of the task. The .NET Core task has a few settings, but the most important one is the Command. With this, you can determine what the .NET Core task will do. The options should look familiar.

Restoring

Restoring the solution is the very first task. It will install all dependencies and tools needed for your solution – and its projects – to work. Let’s set the command of the .NET Core task to Restore.

Building

When all is set and restored, it is time to build the solution. Building the solution is exactly the same as building it in Visual Studio.

Add another .NET Core task to Agent job 1 and make sure you place it under the dotnet restore task. This time choose the command build.

You will see a setting for Path to project(s). Here you can define which projects you want to build. You can specify specific paths if you want. Or you can use a wildcard to just build all projects. I will use the wildcard and put **/*.csproj in the textbox.

Testing

If the build is successful we want to make sure all our unit tests work. As a good developer you make unit tests for all the code you make, right? (…)

Add another .NET Core task and choose test as the command. Here you can define the path to the project(s) you want to use, but I want to run all the projects that contain unit tests. So I keep it empty.

Azure Web App Deploy

The idea is to publish the API to an Azure Cloud Web App. Azure DevOps needs to know your cloud subscription. First, let’s add the task Azure Web App Deploy to the agent.

Select your Azure subscription from the dropdown. If you don’t see your subscription, please follow this article written by Microsoft.

Even if you have connected your Azure Cloud subscription or used it before in Azure DevOps, you might need to authorize the pipeline. Press the button Authorize behind the selected Azure subscription. This opens a popup where you need to log in with your Microsoft Account. Make sure it’s the same account as the one that is connected to the selected Azure subscription.

After that, you can select the App Type. Here you select if the app should be hosted on a Windows or Linux environment. This should match the same environment as the app you registered on Azure Cloud. In my case that’s Windows.

The last required setting is the App name. Expand this one and select the Azure Cloud app you want to use to publish the code to. My app name on Azure Cloud is PipelineDemoCICD.

All the required fields are now correctly set. But we are not done yet. Do you see the setting Package or folder? The value is pointing to a zip file, which we don’t have yet.

Zipping Web App

We need to add another .NET Core task, but this time with the command publish. This one needs an argument:

--output $(Build.ArtifactStagingDirectory)

It’s to make sure the file(s) we want to upload to Azure Cloud are stored in a directory we know because we need to enter this directory in the Azure Web App Deploy task. It’s a good idea to copy the $(Build.ArtifactStagingDirectory).

Make sure the checkbox Zip published projects is checked. Leave other settings as is.

Go back to the Azure Web App Deploy and enter the following value in the Package or folder field:

$(Build.ArtifactStagingDirectory)/**/*.zip

This task will try to get the zip file from the directory and send it to the Azure Cloud, where it will be unzipped.

Testing The Pipeline

The pipeline can now be tested. Let’s run it!

Somewhere on the top of the page, you see Save & queue. You click this. It will save the pipeline and run it.

You can add a comment if you want, but I usually keep it empty. Who’s gonna look at it anyway? Click on Save and run.

You will now see the summary of the pipeline. Under jobs, you see Agent Job 1. Click this.

Failed

It might take some time, but it will start at some point… And it will fail. The nice thing is that Azure DevOps will send you an e-mail when the pipeline has failed or succeeded. So you don’t have to wait for it.

The dotnet test task will fail. When you click on it you will see a log. After some investigation, you will see that it failed because of an error:

##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1

This is correct because I made an error in one of the unit tests. I will come back to this later. We can say for sure this pipeline is working… This far. To test if the code is published to the cloud we can disable the task for testing.

Go back to editing the pipeline, right-click on the dotnet test task, and select Disable selected task(s). When you have done this, save & queue the pipeline again.

If will now succeed and the API is published.

Triggers

While editing the pipeline you are in the Tasks-tab by default. but there are more items to edit in the pipeline. One of them is the tab Triggers.

A trigger is nothing more than a function that will be called when a certain action occurs. It could be a schedule, build completion, or when the code in the repository changes. And the last one is very interesting.

Normally, when you work with Git you have multiple branches. A development branch and a main branch. Some companies also have a test branch. How it usually works is that you (the developer) work on the development branch. And at the end of the sprint (when working agile) someone pushes all the code from the development branch to the test or main branch.

Activation Continuous Integration

Most teams have a trigger on the main branch; as soon as the code in the main branch is changed, execute the pipeline. To activate you simply have to check the box Enable continuous integration. This will enable the whole continuous integration and deployment with Azure DevOps.

This will allow you to include, or exclude, certain branches. Each branch included will be looked after by the trigger.

That’s it! Don’t forget to save it by clicking on the arrow behind Save & queue and selecting Save.

Make sure you enable the test-task in the pipeline. Then go to the pipeline-page and select the pipeline you just added/edited. Here you see all the runs. Keep an eye on it.

Pushing New Code

Remember the unit test that fails? Open the solution in Visual Studio and fix the unit test. The fix is pretty easy. Change the GetSingle method in the MovieService class:

public Movie GetSingle(int id)
{
    if (id <= 0) throw new ArgumentException(null, nameof(id));

    return movies.SingleOrDefault(x => x.Id == id);
}

Commit and push the fix to Git and look at the list of pipeline runs. You might want to refresh the page…

(Yeah, I had to test a bit…)

And now you have a fully working CI/CD working in Azure DevOps.

Conclusion On Continuous Integration And Deployment With Azure DevOps Pipelines

Continuous integration and deployment with Azure DevOps is a great way of automating building, testing, and publishing. That doesn’t mean you don’t have to test your own builds and unit tests before you push your code to Git. Always do this before pushing code, no matter what.

A pipeline can do so much more. Ever heard of SonarCube? Integrate this in your Azure DevOps pipeline and it will check your code on so many levels. From naming conventions to spaces used between keywords.

I have written a website scraper that gathers a lot of information on different websites. The annoying thing about websites is that the HTML might change and my crawler stops working or gives errors. To avoid this I made a pipeline that checks all the websites I am crawling by using unit tests. Every 2 days the pipeline is triggered by a schedule and it will send me an e-mail when it fails.

It’s free till a certain level, so go ahead and play around with it. You can’t break it.