In your organization, you may have an Azure environment where all deployed services need to be private for security-related requirements. In this context, it will not be possible to deploy from azure devops or gthub actions using Microsoft-hosted devops agents because such agents will not be able to enter a private environment.
For more information about , please refer to the following link : https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml.
Several solutions were considered by devops teams to deploy in a private environment from github action or azure devops:
Among these solutions, we can enumerate the following :
- Deploying private agents on virtual machines hosted within a virtual network, utilizing these machines for deployment within the private environment
- Open a firewall to permit entry for DevOps agents (Microsoft-hosted DevOps agents) into the private network
- With GitHub, you have the capability to push images into GitHub and subsequently import them into a private Azure Container Registry. These images can then be utilized for deployment within the private environment using the container deployment option
- running a self-hosted agent in Docker using Azure Container Instance
In this article, we will explore the final solution, which appears to be the most effective as it eliminates the need for maintenance
For more information about running a self-hosted agent in Docker , please refer to the following link:
https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops
I will need to deploy the following services :
- An azure Virtual Network
- A Devops Subnet with service delegation to Microsoft.ContainerInstance/containerGroups.
- A user managed identity
- An azure container registry
- An azure container instance
- An azure devops organization
- And finally, I should have authorization to create an agent pool and a personal access token (PAT)
Setup terraform
The first step is to set up your Terraform environment. For more information about Terraform, please refer to the following link, where you can find all the information to begin using Terraform, including the Azure provider, which enables deployment in Azure:
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
The following configuration requires the Azurerm provider to enable deployments to Azure using a 3.0.0 version or later
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">=3.0.0"
}
}
}
provider "azurerm" {
features {}
}
Resource group and virtual network
The following code creates a ressource group and a virtual network
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/resource_group
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/virtual_network
Here you can find all documentation related to azure rm provider :
resource "azurerm_resource_group" "resource_group" {
location = var.resource_group_location
name = var.resource_group_name
}
resource "azurerm_virtual_network" "virtual_network" {
address_space = [var.virtual_network_address_space]
location = var.resource_group_location
name = var.virtual_network_name
resource_group_name = var.resource_group_name
depends_on = [
azurerm_resource_group.resource_group,
]
}
Let us explain the terraform code defined above
The Terraform code is used to deploy two resources: a Resource Group and a Virtual Network.
azurerm_resource_group defines an Azure resource group. The specified attributes are:
location: The geographic location where the resource group will be created. The value is set by the var.resource_group_location variable.
name: the name of the resource group. The value is set by the variable var.resource_group_name.
azurerm_virtual_network defines an Azure virtual network. The specified attributes are:
address_space: the address space for the virtual network. This is an array of CIDR addresses. The value is set by the var.virtual_network_address_space variable.
location: the geographic location where the virtual network will be created. The value is the same as the resource group, defined by var.resource_group_location.
name: the name of the virtual network. The value is set by the var.virtual_network_name variable.
resource_group_name: the name of the resource group to which the virtual network will be associated. The value is set by the variable var.resource_group_name.
depends_on: Specifies a dependency between this resource and the previously created resource group, ensuring that the resource group is created before the virtual network is created.
For more information about terraform please refer to the following links :
DevOps Subnet
I will need a subnet delegated to Microsoft.ContainerInstance/containerGroups because the azure container will be deployed in, here the idea is to have the devops agents inside the virtual network
resource "azurerm_subnet" "devops_subnet" {
address_prefixes = [var.subnet_address_space]
name = var.subnet_name
resource_group_name = var.resource_group_name
virtual_network_name = azurerm_virtual_network.virtual_network.name
delegation {
name = "delegation"
service_delegation {
actions = ["Microsoft.Network/virtualNetworks/subnets/action"]
name = "Microsoft.ContainerInstance/containerGroups"
}
}
depends_on = [
azurerm_virtual_network.virtual_network
]
}
User Managed Identity
In this section, I will create a user managed identity, which will have the acrpull role to be able to deploy images from the container registry to the container instance.
And this user managed identity will be the identity of the container instance
resource "azurerm_user_assigned_identity" "user_assigned_identity" {
resource_group_name = var.resource_group_name
location = var.resource_group_location
name = "${var.resource_group_name}-identity"
tags = (merge(var.tags, tomap({
type = "user_assigned_identity"
})
))
depends_on = [azurerm_resource_group.resource_group]
}
Azure Container Registry Module
resource "azurerm_container_registry" "container_registry" {
name = var.registryName
resource_group_name = var.resource_group_name
location = var.resource_group_location
sku = "Standard"
admin_enabled = false
tags = (merge(var.tags, tomap({
type = "container_registry"
})
))
}
resource "azurerm_role_assignment" "role_assignment" {
scope = azurerm_container_registry.container_registry.id
role_definition_name = "acrpull"
principal_id = var.user_assigned_identity_principal_id
depends_on = [azurerm_container_registry.container_registry]
}
This code defines two Azure resources:
azurerm_container_registry creates an Azure container registry with the specified parameters, including name, resource group, location, service level (SKU), and tags.
azurerm_role_assignment assigns a role to a user identity specified in the container registry. The “acrpull” role is assigned to the user identity to allow container images to be pulled from the registry.
Azure Container Instance Module
resource "azurerm_container_group" "container_group" {
name = var.containerGroupName
resource_group_name = var.resource_group_name
location = var.resource_group_location
os_type = "Linux"
ip_address_type = "Private"
subnet_ids = [var.subnetId]
identity {
type = "UserAssigned"
identity_ids = [
var.user_assigned_identity_id
]
}
dynamic "container" {
for_each = var.containers
content {
name = container.value.name
image = "${container.value.image}:${var.build_number}"
cpu = container.value.cpuCores
memory = container.value.memoryInGb
ports {
port = container.value.port
protocol = "TCP"
}
secure_environment_variables = {
"AZP_URL" = var.AZP_URL
"AZP_TOKEN" = var.AZP_TOKEN
"AZP_POOL" = container.value.AZP_POOL
"AZP_AGENT_NAME" = container.value.AZP_AGENT_NAME
}
}
}
tags = (merge(var.tags, tomap({
type = "container_group"
})
))
image_registry_credential {
server = var.registryLoginServer
user_assigned_identity_id = var.user_assigned_identity_id
}
}
This Terraform code is used to create an Azure Container Group with devops agent containers inside.
resource “azurerm_container_group” “container_group”: This is the definition of a Terraform resource of type “azurerm_container_group” with the logical name “container_group”.
name: The name of the container group.
resource_group_name: The name of the resource group that the container group will be associated with.
location: The geographic location where the container group will be created.
os_type: The container operating system type. In this case it is Linux.
ip_address_type: The type of IP address assigned to containers. In this case it’s “Private”, meaning they will get private IP addresses.
subnet_ids: The IDs of the subnets where the containers will be deployed.
identity: This section defines the identity used by the container group. In this case, it is a User Assigned Identity
dynamic “container”: This part of the code dynamically creates containers inside the container group based on the var.containers variable. Each container is defined by a JSON object containing the following attributes:
name: The name of the container.
image: The Docker image used for the container, with variable interpolation to include the version number (var.build_number).
cpu: The number of CPU cores allocated to the container.
memory: The amount of memory allocated to the container.
ports: The ports exposed by the container, with the port number and protocol (TCP).
secure_environment_variables: The secure environment variables defined for the container.
tags: The tags associated with the container group.
image_registry_credential: The credentials used to access a private Docker registry.
Testing
create resource group
$subscriptionName="Visual Studio Enterprise"
az account set --subscription $subscriptionName
# create resource group
$resourceGroupName="rg-datasynchro-iac"
$resourceGroupLocation="westeurope"
$storageAccountName ="stdatasynchroiac"
# create resource group
./powershell/resourceGroup.ps1 -resourceGroupName $resourceGroupName `
-resourceGroupLocation $resourceGroupLocation
create terraform backend storage account
$subscriptionName="Visual Studio Enterprise"
az account set --subscription $subscriptionName
$resourceGroupName="rg-datasynchro-iac"
$resourceGroupLocation="westeurope"
$storageAccountName ="stdatasynchroiac"
./powershell/storageAccount.ps1 -resourceGroupName $resourceGroupName ` -
-resourceGroupLocation $resourceGroupLocation `
-storageAccountName $storageAccountName
create azure container registry
$subscriptionName="Visual Studio Enterprise"
az account set --subscription $subscriptionName
terraform init -var-file="dev.tfvars"
terraform plan -target="module.container_registry" -var-file="dev.tfvars"
terraform apply -target="module.container_registry" -var-file="dev.tfvars" --auto-approve
build and push docker image to azure container registry
$registryName="lortcslogcorner"
$tag="1.0.2"
$imageName="dockeragent"
az acr login --name $registryName
docker build . -t "${registryName}.azurecr.io/${imageName}:${tag}"
docker push "${registryName}.azurecr.io/${imageName}:${tag}"
Deploy azure container instance
$devopsOrg="https://dev.azure.com/logcornerworkshop"
$personalAccessToken="{{REPLACE_WITH_YOUR_PERSONAL_ACCESS_TOKEN}}"
$poolName="DOCKER-AGENTS"
terraform init -var-file="dev.tfvars"
terraform plan -target="module.container_instance" -var-file="dev.tfvars" -var "build_number=$tag" -var "AZP_TOKEN=$personalAccessToken" -var "AZP_URL=$devopsOrg" -var "AZP_POOL=$poolName"
terraform apply -target="module.container_instance" -var-file="dev.tfvars" -var "build_number=$tag" -var "AZP_TOKEN=$personalAccessToken" -var "AZP_URL=$devopsOrg" -var "AZP_POOL=$poolName" --auto-approve
Source code
The full code is available here :