AI is increasing costs related to the type of cloud resource it uses. Below I take a look at why AI is increasing resource costs and how you can manage it.
Structural Costs > Reserved vs On-Demand
AI’s dramatically variable resource demand will increase resource costs.
Historically, an organization’s server rooms anchored its IT strategy. Servers were provisioned with dedicated hardware with fixed capacity. Annual budgets were allocated to provision new storage, compute or network capabilities.
This changed when cloud providers began offering their infrastructure for rent. The centralized hardware resources are faster to provision and today you can provision a large scale system in seconds.
In general, cloud resources are made available to customers one of two ways:
- Reserved. Reserved resources work similarly to the old server room days: your data center resources are reserved for your use, have a static specification and generate fixed costs.
- On-Demand. On-Demand resources are flexible, with customers only paying for what they consume. With the flexibility of on-demand comes cost volatility – if your traffic suddenly increases, so does your bill.
How is AI changing things?
The scale of resources that AI consumes, along with the highly variable nature of that consumption, is requiring more flexibility than traditional systems, and providing this flexibility introduces more cost as more on-demand resources are needed.
In addition, the “what” that is being provisioned has changed:
- Traditional teams would choose and pay for cloud instances described with infrastructure terms: CPUs, RAM, Storage.
- Teams working with AI technology are provisioning “Throughput”, which is a measure of how fast an AI model can process a request.
What can you do about it?
While reserved capacity continues to cost less than on-demand capacity, the large scale and highly variable nature of what resources AI will consume makes it difficult to project what capacity to reserve. Here’s a summary of what you can do to control costs:
- Deploy AI Efficiently. The very best way to control costs is to ensure your AI is deployed efficiently and doesn’t consume more than it needs of either reserved or on-demand capacity.
- Educate Users. Explain the costs of using AI so that users only trigger expensive AI activities when they can produce corresponding value.
- Constantly Revise Allocations. Monitoring and revising the capacity you have reserved and the capacity that is consumed on-demand is a critical activity to ensure that the right balance exists.
- Stay Informed of Offerings. As new business models emerge and vendors compete to offer more effective ways to address the large-scale needs of AI, it’s important to stay aware of the changes in offerings so that you can take advantage of newer, cheaper options.
In Summary: Continuously rebalance reserved vs on-demand capacity to help control costs.
Hi
Support this Work
You can purchase the book on Amazon:
You can listen to the podcasts for free through Patreon, Apple Podcasts or YouTube Music: