Azure Serverless Components and Cost
Recently I came across this scenario related to Azure Serverless components, specially Azure Functions and Azure Sql Database (Serverless) and how its usage directly or indirectly affect the cost.
In this scenario following are key things.
Initial Scenario
Azure Functions (Function App) represent the service as part of a large application and it has a certain amount of function but for simplicity let's assume that it is CURD endpoint to create person.
Azure Sql Database is data store where the information store. This is being configured as Serverless General purpose to save the cost. If it is not being used or say idle for 1 hr, it will go to pause mode. In this situation it will cost nothing. Due to this if it used after it went to pause mode, it consume some time to come into running state and this is fine as some of the services in overall application architecture is not heavily used.
- The diagram display above is the initial point and everything is working fine.
New Requirement - Publish message from person service
First change that is so obvious and specially in large application with Microservice architecture that one service needs data of other service. Now person data created in person service required by other service and after some discussion solution comes to use Azure Service Bus to publish newly created person and change in person as a event.
Due to introduction of service bus, now cost is involved related to Azure Service Bus. This is also fine as it is required component for overall architecture and it is anticipated during initial analysis.
Flow is changed like after successful insert/update to database and it will also publish message to service bus and then consumer service will process that message.
New Requirement - Disaster recovery option for published message
Now this is something new at that time and after going through certain official document related to this from microsoft, it is clear that Service bus support Disaster recovery for message at Zone level but not region level. Region-level only metadata will replicated if there is replicated service bus running in another region. In simple words, if you have any topic or subscription, it is going to replicate at region level but not the message it self. Also not all message required to be replicated but in our scenario it is not the case. It is ok if the message process two times but it is not ok if it is not processed at all.
After some investigation and discussion, Outbox or say transactional outbox pattern seems to be the solution. Along with this Inbox pattern as well, to record message that successfully processed but this is not main discussion point over here.
Before going into detail about cost, Outbox is one of the essential pattern for durable message and make event driven arcihtecture more reliable.
Now flow is changed like this.
Instead of publish message directly to service bus, as part of database insert or update and in same transaction publish message store in data store and in this case Azure Sql Database and that table called "Outbox".
Now question comes which component will process the message? If it is on-premise application then we can think of some kind of background process and in Azure Function case, we can think of timer trigger. Consider that timer trigger configured to run every 5 minutes and what it will do?
- Check if there is any message that is not processed. If that found then it will process otherwise not. Simple right?
Now above point is key thing in terms of overall cost increase for database and that is what I explain next.
As soon as the time trigger introduce and configure to run every 5 minutes, now it goes to the database every 5 minutes to check. What it does?
- Now to save the cost, the database needs to be idle for 1 hr and due to this checking it is never going to idle mode and it will cost for minimum resource configuration and there is definitely 1 query every 5 minutes even if there is not the message.
In my scenario cost for DB, that is anticipated as per usage around 25 Euro and it suddenly goes upto 170 Euro.
Summary
If there is serverless components does not mean that it will always cost less. It all depends on usage and this point needs to consider.
During the design and analysis phase, cost needs to consider for every critical pattern that has the possibility to affect cost intensive components like storage or compute unit.
It is not good either to assume that the outbox is wrong and it all depends on usage and there is always a trade-off and needs to choose between one over another.