In today’s cloud-native world, building resilient systems is more important than ever. As teams adopt microservices and rely on various downstream dependencies, it becomes crucial to handle failures gracefully. One common challenge when working with Azure Functions is how to temporarily pause message processing. In this post, we’ll explore how to implement a circuit breaker pattern that allows you to programmatically disable and re-enable function triggers, helping your system recover smoothly without losing data or overwhelming dependent services.

If you’re using Azure Functions at scale, you’ve likely encountered scenarios where you need to temporarily pause processing:

A critical downstream service is experiencing an outageYou’ve hit API rate limits with an external serviceYou’re performing maintenance on dependent systemsYou need to recover from an incident without processing new events

Manually disabling function triggers through the Azure Portal is cumbersome and doesn’t scale well. Automating this process provides more reliability and helps prevent cascading failures across your architecture.

The Solution

The solution is based on an Azure Function App called called FuncTriggerManager which provides a circuit breaker pattern implementation for other Azure Function Apps. It allows you to programmatically enable or disable function triggers.

Let’s walk through the flow at a high level:

Function Triggered by Message
An Azure Function is triggered by a message from a Service Bus queue or topic.Processing and API Call
The function processes the message and attempts to forward it to a downstream API.Failure Detected
If the API responds with an HTTP 503 (Service Unavailable), the function logs the error and sends a control message to an Azure Storage Queue.Trigger Manager Disables Function
A separate Function App (FuncTriggerManager) listens to this queue, disables the affected Function App, and places another message on the queue with a scheduled delay.Function Re-enabled
After the delay, the manager function re-enables the original Function App, allowing message processing to resume once the downstream service is likely to have recovered.

Key Features

Queue-based control: Send simple messages to enable/disable specific functionsAutomatic re-enablement: Set a time period after which functions automatically resumeMinimal permissions: Uses managed identity with least privilege accessSimple integration: Minimal code changes needed in your existing functionsHow it works

The solution consists of an Azure Function with a queue trigger that:

Monitors a designated storage queue for control messagesParses instructions about which function to enable/disableUses Azure Resource Manager APIs to modify the function’s settingsIf configured, automatically re-enables functions after a specified period

Here’s what the control message looks like:

{ "FunctionAppName": "YourFunctionAppName", "FunctionName": "SpecificFunctionName", "RessourceGroupName": "YourResourceGroupName", "DisableFunction": true, "DisablePeriodMinutes": 30}

When this message is processed, the specified function will be disabled for 30 minutes, then automatically re-enabled. If you set DisablePeriodMinutes to 0 or omit it, the function will remain disabled until explicitly enabled.

Full source code of the solution can be found here https://github.com/connectedcircuits/funcappshortcct

Implementation Details

Under the hood, FuncTriggerManager uses the Azure Management APIs to modify a special application setting called AzureWebJobs..Disabled that controls whether a specific function trigger is enabled. This is the same mechanism used by the Azure Portal when you manually disable a function.

The FuncTriggerManager service authenticates using its managed identity with the following permissions applied to the Function App you wish to control.

Reader access to the resource group of function apps to controlWebsite Contributor permissions on target function apps to controlTesting with a Sample Function App

I have included a sample Function App so you can follow how to send a control message onto the storage queue. This sample Function App is triggered when a message is placed onto a Service Bus queue. It then sends the message to a HTTP endpoint which represents an API. If it returns a status code of 5xx a control message is placed onto the Storage Account queue to disable its trigger.

The test function simulates a real-world scenario where you might need to temporarily disable message processing triggered messages from a Service Bus queue.

The source code for sample simulator can be found here – https://github.com/connectedcircuits/FuncSbProxy-funcappshortcct-

Getting Started

To implement this pattern in your environment:

Deploy the FuncTriggerManager function app with managed identitySet up the required app settings:StorageConnection: Connection string to your Azure StorageQueueName: Name of the queue to monitorAzureSubscriptionId: Your Azure subscription IDDevelop your Function App service and copy the DisableFuncMessenger.cs (as in the sample FuncSbProxy)Setup the required app setting your Function App serviceStorageConnection: Connection string to your Azure StorageQueueName: Name of the queue to monitorResourceGroupName: Name of the Azure Resource where this function app will be deployed to.DisableFuncPeriodMin: Time in minutes you want disable the trigger for.Real-World Use Cases

Here are some scenarios where I’ve found this pattern particularly useful:

Scenario 1: Downstream Service Outage

When an external API reports a 503 Service Unavailable error, your error handling code can place a message in the circuit breaker queue to disable the corresponding function until the service recovers.

Scenario 2: Rate Limit Protection

If your function processes data that interacts with rate-limited APIs, you can temporarily disable processing when approaching limits and schedule re-enablement when your quota resets.

Conclusion

The circuit breaker pattern is essential for building resilient systems, and having an automated way to implement it for Azure Functions has been invaluable in our production environments. FuncTriggerManager provides a simple yet powerful approach to managing function trigger states programmatically.

By incorporating this pattern into your Azure architecture, you can improve resilience, reduce cascading failures, and gain more control over your serverless workflows during incidents and maintenance windows.

Note: Remember to secure your circuit breaker queue appropriately as it provides control over your function apps’ operation.

Enjoy…