How to build cloud computing cost forecasting
The flexibility of the cloud is one of its huge selling points. It brings all kinds of benefits, but also some challenges. From a cloud cost management perspective, the biggest challenge is to be able to predict costs when infrastructure is in a process of change. The more rapidly this change occurs, the more challenging it is to predict costs. The good news is that this challenge has a solution.
Start by working out where you are now
You need a baseline from which to track changes and the obvious one is where you are now. Hopefully, you are already on top of your cloud cost spending and have clear visibility of what spend belongs to what project, product or service. If not, then you need to fix that before going any further. You could take this exercise as an opportunity to address any obvious red flags in your billing. That basically means anything which suggests that people are not managing their cloud spend as economically as they could.
Deal with any cloud cost optimization issues
Once you have checked that people are not wasting cloud resources, it’s strongly recommended to take a good, hard look at your billing data and see if there are any signs that you need to improve your underlying cloud infrastructure. For example, excessive data transfer costs may be a sign that you need to rework your apps to reduce the extent to which data is transferred between regions or to and from the internet. Calculate your ec2, lambda, data transfer or s3 cloud cost to have a baseline.
There are two reasons for this. First of all, you will get a far better return from addressing major issues such as costly weaknesses in your cloud infrastructure than you will from finessing your forecasting. Secondly, you want and need an accurate baseline from which to track changes, so you need to sort out any obvious issues you have in the present (or at least the major ones) before you try to predict what is going to happen in the future.
Analyze your historical usage data
Your historical usage data will, quite literally, show you how your cloud usage has developed over time. Even though it may (and probably will) reflect your cloud learning curve and all the inefficiency that implies (as you learn how to use the cloud), you will usually still get a good idea of the general direction of travel and that can often generate a lot of insights into the future of your cloud usage and therefore your cloud spend.
At the very least, you should be able to see the seasonal trends in your cloud usage (in other words the periods of highest and lowest demand) and use these to inform your estimates for the same periods going forward. You may be able to take this a step further and see what services, or at least which types of services, are popular at what times of the year and also if there are any services, or types of services, which are increasing or decreasing in popularity.
In addition to analyzing usage, it’s also helpful to look at how people are paying for what they consume. For example, are they using On-Demand Instances, Reserved Instances (or a Compute Savings Plan) or Spot Instances? Is there a reason for this spending pattern? For example, have your staff noticed that Spot Instances tend to be particularly economical at certain times of the year, or are people just following habits, and, if the latter, could those habits be improved?
Keep checking your estimates against your invoices
Forecasting billing in data centers is a bit like driving down a major highway. You may come across the odd bend or turn, but they will probably be few and far between and usually indicated well in advance. Forecasting billing in the cloud is more like driving in a strange city. Circumstances will probably change so regularly that you could simply stop noticing the fact unless you actually stop and check.
When you are auditing your invoices, you need to be very clear about whether any inaccuracy was caused because you did not predict usage correctly (and if so what was the issue) or if it was simply a reflection of the fact that cloud platforms not only have extremely intricate billing models but that these change frequently. This means that you could forecast your cloud cost usage with slam dunk accuracy and still be wrong with your billing costs.
If you really want to finesse your billing forecasting, you could look for trends in how often cloud-platform providers update their billing for certain services and incorporate this into your estimates. This is, however, likely to be too much for most companies. The more pragmatic approach is just to stay alert to this practice and be ready to take action as soon as it becomes appropriate.