A recent study on cloud costs has sparked provocative headlines, such as InfoWorld’s “Was moving to the cloud a mistake?” Clearly, this one was intended to grab attention. With the exponential growth in cloud adoption, who wouldn’t want to find out if it were all for naught? Fortunately, the news is less dire than suggested. Rather, it is evidence of an increasing maturity across firms already well along in their cloud journey.
The pandemic accelerated digital transformation and the shift to the cloud. The word has gotten out about the performance, elasticity, and simplicity it offers. In the current dynamic business environment, that’s the right prescription. Gartner® predicts that “by 2025, there will be more expenditure on cloud-based services than on traditional, non-cloud IT… By 2023, 40% of all enterprise workloads will be deployed in cloud infrastructure and platform services, which is an increase from 20% in 2020”. 1
Image source: Wanclouds
The survey that’s generating all the buzz asked IT leaders what direction they were getting from their CFO or other C-suite executives with regard to cloud spending. The headlines point to an aggregated 81% of respondents being asked to stay the same or reduce spending. Yet only 14% were asked to reduce cloud spending “dramatically.” And, 18% were being asked to increase spending. The most common response was to stay the course with no additional spending. In today’s current business environment, that shouldn’t come as a surprise.
Another survey sheds further light: 42% of CIOs and CTOs surveyed identified appropriate resource utilization as their top cloud challenge. Other challenges include justifying budget increases (34%) and adhering to allocated cloud budget (31%). As investment is expected to grow, better governance of costs is imperative. That result suggests that the issues are not with cloud itself but rather with the overwhelming popularity of the new services.
Slow into the turn; accelerate out
As I learned in driver’s education, you’ve got to slow down into a turn and accelerate out of it. That’s what these companies are gearing up to do. Now that they’ve clearly entered into the cloud turn, they will slow down growth in spending in order to be better positioned to accelerate once they’re headed in the right direction.
Remember the old adage “faster is slower,” particularly if you hit the wall going into the turn.
Address cloud cost concerns with good financial governance
Those concerned with rising bills refer to the “cloud cost paradox” of higher-than-expected costs with a move to the cloud. A whole new industry has emerged to promote a collaborative, cross-disciplinary practice to manage cloud computing costs. At the center of Financial Operations is the need for effective financial governance, a practice that Snowflake takes very seriously. Financial governance is one of the pillars of innovation presented on Snowflake’s Data Cloud World Tour.
Snowflake starts with an advantage out of the box with the cost benefits of the storage compression, the elasticity of compute clusters and their ability to be resized or turned off for greater efficiency, and Snowflake’s commitment to continuous performance improvement. Snowflake passes performance improvements directly to customers, without the need for expensive upgrades or migrations. These enhancements just appear within a customer’s deployment.
In addition, Snowflake offers a portfolio of tools to provide greater visibility into usage, to enable granular usage controls, and to facilitate optimization of consumption, and therefore spending. Yes, the resources available in the cloud might be near-unlimited but we know that customer budgets are not.
Let’s take a quick look at these three pillars of financial governance:
Visibility. With resource monitors, Snowflake customers have visibility into usage with dashboards that drill down by specific resource or action such as warehouse, database, data transfers, other procedures such as pipeline usage or materialized views, and of course query history. It’s not just logs, but account usage views that are already analytics and visualization ready. Administrators can look at hour-by-hour consumption, query volumes by hour, longest running queries, most data consuming queries, and even credit consumption by user, to name a few metrics. Armed with this information, they can see if there are certain users consuming more credits than they should. Or they can see if resources should be resized for better performance.
Visibility into usage is the first step to getting a handle on costs. Watch this space for more features to enable granular usage attribution and even chargeback policies.
Control. Good financial governance is not only looking in the rearview mirror. Snowflake provides tools for more proactive control. Administrators can establish usage quotas. Resource monitors can send alerts when those quotas are reached, or even automatically suspend the account. If that approach is too drastic, administrators can set up suspension warnings when approaching quotas. Today, alerts are at the account or resource level but are soon to be even more granular.
Other features also limit the potential for overspending. An auto-suspend policy will turn virtual warehouses off quickly after they go idle, so costs don’t accrue when resources are not running. Auto-resume will turn them back on if necessary. Administrators can also set time thresholds to limit long-running queries. Session timeouts or scope statement timeouts limit errant use. And, a review of access history could identify unused tables which could be dropped to reduce costs.
Optimization. With visibility into usage metrics like length of query or cloud services usage, administrators can identify anomalies and rectify inefficiencies. If a query is running too long, should the resources be resized? Or should the query be restructured for greater efficiency? If a query is being executed too often or takes too long, maybe there is an opportunity to materialize the results as a table.
For more tips on how to visualize, control, and optimize spending, take a look at Snowflake’s ebook that outlines 7 Best Practices for Optimizing Your Snowflake Investment, or watch the webinar 10 Ways to Save: How to Manage Costs and Optimize Resources on Snowflake.
As with drivers, companies have realized that when entering a turn or undertaking your cloud data transformation, you can’t just accelerate. Slow into the turn by putting financial governance in place so that you can accelerate out. The reflective pause that cloud adoption surveys have revealed is evidence that too fast actually can be slow. The laws of driving—and business—still apply.
1 Gartner, How Cloud Adoption Will Increase Opex Budgets, ID G00765092, May 2022 GARTNER is the registered trademark and service mark of Gartner Inc., and/or its affiliates and has been used herein with permission. All rights reserved.
Originally posted on November 16, 2022 @ 12:28 am
