Why are we Excited About Google BigQuery Omni?
Love changes, changes everything as the classic 80’s song goes, and Google has shown some multi-cloud love recently, especially with the recent announcement of Google BigQuery Omni - its cloud spanning data processing platform.
A quick intro to Google BigQuery
First, a bit of background...
Released in 2011, the original BigQuery is Google Cloud’s Enterprise Data warehouse. It’s what you use when you’ve collected a large volume of data in GCP, and need to ask questions of it.
When I say large, I mean how big do you want to go? BigQuery can handle anything from gigabyte to petabyte scale… we’re not yet in the exabyte query world… at least not yet!
But with Omni, the limitation of your data having to reside in GCP is lifted, and this might just be the start of something remarkable… and inevitable.
BigQuery Omni: A refreshing approach to the cloud market
Inevitable… why’s that?
Whenever we get multiple services offering solutions into the market, inevitably a company comes along and creates a meta/overarching-service; a service that brings together many of the services from the separate vendors.
This has been done in many other markets. Take AppleTV for example, unifying all of your streaming services under one searchable interface. Or what about GrubHub, bringing together multiple restaurants’ menus into one ordering app. Or even any of the travel websites like Expedia or Trivago showing you all of the best deals on flights from competing airlines and hotels.
All of these overarching-services have one thing in common: they make the focus of the service the end-user, presenting information in a way that is best suited to them.
It’s a very different experience from those trying to tie the user into their services, and (usually) theirs alone - this is not the type of lock-in we like.
So after Google coming out with Anthos, and now BigQuery Omni, it is an interesting and refreshing approach in the Cloud market.
A multi-cloud masterstroke?
It may be a masterstroke from Google, since one of the many questions we get asked by customers is “which is the best Cloud for XYZ type of data”, or “how do we do multi-cloud?”.
There is no simple answer to either of these questions. However, that could well be changing now (at least for the latter question).
Omni goes a long way to creating the nirvana of a cogent, multi-cloud data architecture. This would be the system that carries the pulses of innovation throughout the body of your company. For a system like this to work, it needs to be resilient and flexible to adapt to the changing demands placed up the body... allowing you to focus more on the value you are trying to bring, rather than getting bogged down in how you are going to do it.
This multi-cloud approach is an easy way for companies to start to take advantage of the power and ease-of-use of BigQuery, no matter which cloud they are on currently; it's undoubtedly a door opener for Google, and I believe will lead to companies adopting GCP more readily.
But what about costs?
One of the big factors in cloud pricing is the egress cost of data - this is especially a limiting factor for multi-cloud deployments.
However, in a way that is reminiscent of how Map-Reduce functions in Hadoop limits ‘off-node data transfer’ (for performance reasons), BigQuery Omni only egresses the results of the data computation, not the data itself - you’ll not be charged a princely sum - keeping your costs down significantly.
Also, imagine the compliance benefits:
- Geographic Regulatory Compliance - the data stays in whichever data centre/availability zone it was originally deposited in - no matter which cloud you are on, as Omni will run on all three of the major CSP’s platforms.
- Data Lineage - since you won’t be shunting data all over the place from cloud to cloud - and by having one, unified data warehousing and analytics platform, you’ll not be in ‘ETL hell’ - your data lineage will become significantly simpler - so when you have to prove GDPR compliance, and when you get GDPR requests to remove a person's PII data from your systems, it will be significantly simpler - less ‘‘data-spaghetti’, less Technical Debt
It’s another case of the coming together of tools, to build new tools, as I talked about in my previous blog on the future of augmented reality.
The next phase of computing
In the next 20 years, we’ll have more societal change due to technology than in the past 200 years. The next phases of computing and IT will be driven by the results obtained through the uses of Machine Learning - and with tools such as BigQuery Omni appearing, we’re entering what I like to call the era of ‘cloud scale ML for all’.
The move to the Cloud frees companies from legacy data-technology restrictions; to a cloud native, unified, ML-enabled data architecture, that gives us faster innovation, new products, and better decisions to outperform the competition.
That’s how Cloud should be - and that’s why, when it comes to your data, we’re excited to be Gettin' Omni Wit’ It