The cloud holds a large – and growing – percentage of enterprise data. While much of this growth comes from unstructured information, such as documents and media files, enterprises also still depend on structured data, stored in databases.
The global market for databases grew by 12.8% last year, according to analysts at Gartner, and is now worth some US$102bn annually. The bulk of that growth, though, is now in the cloud.
Gartner calculates that 61% of the overall database market is already in the cloud, with 91% of growth coming from cloud technologies.
Adoption trends: gathering clouds
This rise of cloud databases comes hand in hand with wider enterprise cloud adoption.
According to Adrian Bradley, head of cloud at consultancy firm KPMG, cloud take-up among enterprises is growing at 25% to 30% globally. On-premise technology spending is rising more slowly, at 17-20%. Over time, an ever-higher percentage of workloads will be in the cloud.
One reason is that more enterprises are adopting a “cloud-first” or “cloud-preferred” strategy. In this scenario, database teams may have little choice but to move their operations to the cloud too.
But this no longer means accepting cloud as a second-best option.
According to Henry Cook, a research director at Gartner, covering database technologies, it is no longer the case that cloud databases are a second-rate option. “You’ll hear the phrase ‘cloud first’,” he says.
“Vendors used to develop on the ground and then have a cloud version. It’s usually the other way around now – the cloud version leads the way with new features, and then they will retrofit those to the on-prem version. It’s the cloud that’s setting the pace.”
When it comes to performance, cloud technology now matches – or can even beat – on-premise options.
And cloud databases have the added advantage of tighter integration with analysis and business intelligence tools, and a shorter pathway to workloads such as artificial intelligence (AI) training.
Recent developments from some of the most significant database players underscore this.
Google Cloud, for example, recently announced that it will offer Oracle Cloud Infrastructure database services on its cloud infrastructure, under the label Oracle Database@Google Cloud.
And Microsoft announced further integration of its Microsoft Azure Database for PostgreSQL with its AI capabilities, including Azure’s OpenAI service. This allows enterprises to connect to the most advanced AI models, according to Microsoft, but will keep their data in their own database “for privacy and compliance reasons”.
Microsoft and Oracle are the largest, but certainly not the only, database suppliers with cloud offerings. Others include Amazon Web Services, IBM, SAP, Nutanix and Snowflake, as well as several open source offerings, especially those based around PostgreSQL. To complicate matters further, most of the hyperscalers offer both open source and proprietary technology.
Workloads and markets
As a core technology, databases – and cloud databases – are used to support business functions across a range of industries. However, some sectors are adopting cloud relational database technology more quickly.
First among these is the IT and telecoms sector, which analyst firm Grand View Research found accounted for just under 18% of the market in 2022. Other significant users include banking and financial services, healthcare, government, manufacturing, retail, and media and entertainment.
This shows how widely used relational database technology is. The largest users of cloud databases largely mirror the sectors making the most use of database technology overall.
Added to this are organisations that are using databases in the cloud indirectly, usually through software as a service (SaaS). Firms using business applications, such as Salesforce’s customer relationship management (CRM) software, or even SAP’s cloud-based enter prise resource planning (ERP) software, will be using cloud databases, albeit not databases they manage or configure.
“If you’re using Workday or Salesforce for CRM, for example, you don’t have that much choice in terms of cloud,” says Andrew Brust, founder and CEO of technology advisory Blue Badge Insights. Even conventional, on-premise enterprise software suppliers are now largely cloud-based.
“Take SAP as an example, they are pretty much a SaaS company,” he says. “The new deployments are cloud-based, so the whole database then becomes transparent.”
And successful use of SaaS adds impetus to move other technology, such as customised databases or bespoke applications running on them, to the cloud.
CIOs, Gartner’s Cook suggests, are also attracted to technology that is “cloud neutral”. Platforms such as Microsoft’s SQL server have an edge in compatibility between on-premise and cloud versions, easing cloud migration or even eventual repatriation. But other enterprises are more concerned about the ability to move between clouds.
“Some of the market wants to nail their colours to the mast of a particular hyperscaler, and they’re happy with that because it’s all compatible,” he says. “Others deliberately don’t want to do that. They want to use products that either span clouds or can interoperate, where you can choose which cloud you want to run on.”
In some highly regulated markets, being able to run on multiple clouds can even be a requirement, he adds.
Use cases and benefits
The primary use cases for cloud databases, then, are rarely application-specific. Instead, uptake is being driven by factors such as scalability, reliability and the pace of innovation from cloud providers.
Cost, however, should not be the main factor, says Cook. “There is an element of cost saving, but we say you shouldn’t go to the cloud with cost saving as your main objective,” he cautions. “You may make savings, but it depends on the circumstances.”
You shouldn’t go to the cloud with cost saving as your main objective. You may make savings, but it depends on the circumstances Henry Cook, Gartner
Rather, organisations are moving their databases for their high-availability capabilities, improved resilience and business continuity, and support for redundant operations across geographies. “One [advantage] is geographic redundancy, so that your data, if possible, can be in multiple cloud regions for disaster recovery purposes, or just for high availability,” says Blue Badge Insights’ Brust.
And, as with other cloud technology deployments, firms are moving their databases to reduce their IT management overheads.
Moving to the cloud taps into the suppliers’ cycle of new features and upgrades. Firms no longer need to manage that process, nor interrupt production systems to roll out new versions.
“For management, upkeep, maintenance, tuning and optimisation, most of those burdens shift from the customer to the vendor,” says Brust. “That has its own savings in terms of personnel, risk, complexity, friction and getting new projects rolling.”
The cloud also offers scalability in a way that cannot be matched with on-premise systems. This suits both fast-growing organisations and those with variable and possibly unpredictable workloads.
The cloud offers an easy and quick route to add capacity, whether that is for a single database instance or an entire data warehouse or data lakehouse. Large-scale analytics projects are now, for the most part, cloud-based for that reason. And companies can also scale back down. “You can set up a new test environment in minutes, start using it, test it, and move into production,” says Cook. “When you’re done, you can break the system down again and throw it away because you don’t need it anymore.”
Above all, organisations are adopting cloud databases because that is where the industry is focused, both in terms of newer technologies, including serverless, containers and no-code or NoSQL, but also open formats that promise enterprises the ability to migrate between database platforms or, indeed, run their data across multiple platforms.
Barriers to adoption
Nonetheless, there are some reasons for firms to be wary of cloud databases.
Some database applications are still well-suited to on-premise operations. These include stable and predictable workloads, as well as systems running high volumes of lower-value transactions. In these cases, predictable costs favour on-premise, especially where firms can optimise their systems to maximise utilisation.
“We do occasionally see systems being repatriated back down to on-prem, usually because the workload is well understood,” notes Cook. “By doing that, you have a fixed cost.”
And although cloud data egress costs are more transparent than they were, they can still tip the balance towards on-prem if there are large data movements.
Lastly, data sovereignty, security and compliance remain a barrier to cloud database adoption in some sectors. Recent cases around data residency and SaaS applications underscore the risk.
But even in regulated sectors, firms are starting to mix technologies, with on-premise databases for the regulated and sensitive data, and cloud for other, less sensitive applications, including analytics and AI.
“Financial services and healthcare are still more accustomed to keeping things on-premise,” says Brust. “Even there, with virtual private cloud technology, there are ways to hybridise … there are paths to compliance in the cloud, even for those industries.”