Simplifying Event-Driven Architectures: KEDA Scaling with Managed Identity in Azure Container Apps

Introduction

I’ve been using Azure Container Apps (ACA) for some time, It has been a perfect fit for the use cases where you want to avoid managing a whole Kubernetes environment, or don’t want to use a shared one. But, not everything can be perfect; one limitation that some of us found, was that it wasn’t possible to have KEDA authenticate using Managed Identities (MI).

There were ways to get around this limitation, as I described on this GitHub issue and one of my previous blog posts; but you ended up having to write a lot of glue code to build a small microservice, this adds operational overhead and will make the architecture more complex. This is fun to put together; but most of us are busy, so a simpler architecture would be ideal.

The Microsoft Azure Team to the Rescue

After a long wait, the team in charge of this service announced they were working on this feature. Finally, we were seeing the light at the end of the tunnel. Then, on 2024-06-25 we finally got the news we were waiting for: the feature was being rolled out with the API version 2024-02-02-preview.

Testing the New Feature

Although there are many use cases and Azure services to which this can be applied, I will to focus on the azure-pipelines use case. This allows you to scale your self-hosted Azure DevOps agents based on the pool’s queue.

Before this feature, the rules section of the Azure Container Apps ARM template would look like this.

"rules": [
    {
        "name": "azure-pipelines",
        "type": "azure-pipelines",
        "metadata": {
            "organizationURLFromEnv": "AZP_URL",
            "personalAccessTokenFromEnv": "AZP_TOKEN",
            "poolID": "${adoPoolId}",
            "poolName": "${azPool}",
        },
        "auth": [
            {
                "secretRef": "organizationurl",
                "triggerParameter": "organizationURL"
            },
            {
                "secretRef": "azptoken",
                "triggerParameter": "personalAccessToken"
            }
        ]
    }
]

As you might have guessed, KEDA authenticates with Azure DevOps using the Organization URL, the Personal Access Token (PAT) or Bearer Token (depending on your implementation), and the poolID or poolName.

Now, with the new feature on API version 2024-02-02-preview, we need to specify the Managed Identity we want to use, the Organization URL, and either the poolID or poolName, and under the hood, it will obtain the Bearer Token to authenticate with Azure DevOps, as shown in the code below.

"rules": [
    {
        "name": "azure-pipelines",
        "type": "azure-pipelines",
        "identity": "${managedIdentity}",
        "metadata": {
            "poolID": "${adoPoolId}",
            "poolName": "${azPool}",
        },
        "auth": [
            {
                "secretRef": "organizationurl",
                "triggerParameter": "organizationURL"
            }
        ]
    }
]

After applying these changes (I used OpenTofu and the AzAPI provider), we can test. I’m lazy, So I created 10 jobs using the following command.

for i in {1..10}; do az pipelines run  --organization "https://dev.azure.com/AwesomeORG" \
    --branch "main" --name "adoagents-test" --project "AwesomeProject" ; sleep 3; done

Since I’m using Azure Container Apps Jobs, we have to go to the Monitoring > Execution History section to see if the jobs started.

And if we go to the Agent Pool section in Azure DevOps, we can see that the jobs executed successfully.

Conclusions

As you just saw, it works perfectly after making a few modifications. One of the main benefits of this new feature is that the overall architecture is simplified, as you no longer need glue code or additional microservices to renew Bearer Tokens and update the secrets, or to rely on on Personal Access Tokens which are insecure and cumbersome to manage.

I used the azure-pipelines integration as an example, but this feature has been proven to work with other services such as Service Bus, Azure Storage Queues and more, this is great news because this feature will help to simplify Event Driven Architectures and make them more secure.

For additional information and examples on how to configure this for other services, please refer to the following documentation.

• Managed identities in Azure Container Apps.
• Set scaling rules in Azure Container Apps.
• KEDA Azure Pipelines.
• KEDA Azure Service Bus.
• KEDA Azure Storage Queue

In case you have any comments or questions, just let me know. You can always find me on LinkedIn.