Becoming a Cloud Data Platform Driven Organization - Part 2

By: Ron L'Esteve   |   Updated: 2022-06-20   |   Comments   |   Related: 1 | 2 | > Cloud Strategy

Problem

Becoming a Cloud Data driven organization is a long-term strategic initiative which delivers value and significant ROI to organizations. In Part 1 of this series, you learned about the various challenges of on-premises environments and how a Cloud Platform can solve many of these challenges. You also learned about the value of a good Cloud Adoption Plan and how organizations can go about creating one. Customers are interested in learning more about the benefits of becoming a Cloud Data Platform driven organization.

Solution

This article will explore the benefits of adopting a cloud data platform by beginning to define the guiding principles of a Cloud Data Lakehouse, along with what a foundational data journey into the cloud might look like. As you gain a deeper understanding of what a foundational data journey might include, the topic on Data and AI Maturity will provide deeper insights on how to continue the journey and achieve maturity as a Data and AI driven organization in the cloud. The various phases of Gartner's Data Maturity model, a model which has been adopted and presented by numerous cloud providers, will be discussed towards end of the article. Finally, the section on Total Cost of Ownership will introduce various methods for estimating both tangible and intangible value and cost when considering moving to the cloud.

Data Lakehouse Guiding Principles

Frequently, many organizations begin their first MVP project in the cloud by implementing a foundational Cloud Data Lakehouse Platform. The platform would be Hybrid in nature since it would be leveraging connections to existing on-premises source systems in the ELT pipelines. This foundational platform is typically built with the concept of 're-usability' deeply engrained within the architecture.

As an example, this re-usability can manifest itself in the form of dynamically parameterized ELT pipelines. As part of the platform buildout, data integrity will be critical. Secure and well governed access to data will be critical requirements for an enterprise platform which can be scaled and managed. Reporting and data analysis will be critical components of the foundational MVP platform to deliver instant value to business stakeholders that will continue to support and promote the scalability of the platform as more value is delivered to the business through continued growth.

With cloud storage being relatively cheap, data ranging from real-time streaming data to historical data can be stored within Cloud Data Lakes to enable robust reporting and advanced analytics business use cases. Most importantly, customers of this new Data Lakehouse platform would be seeking ease of use. With re-usability as a core platform component, this begins to support 'ease of use' within the platform. As the platform begins to scale, report users and analysts can upskill to learn how best to use the Cloud Platform tools that are available at their disposal. By doing so, business insights can be derived much faster as customers of the platform begin to self-serve their data acquisition, analysis, and reporting needs.

The following principles, shown in the figure below, will guide this process and identify why it is critical to begin “breaking ground” on a Data Lakehouse early in the company's data management journey. These pillars create a foundation for best-in-class future state data management. A Data Lakehouse foundation will enable scalable growth as new systems go live with established data integrity procedures and seamless cross-functional reporting integration.

Data Journey

A foundational data journey into the Cloud would include the design and implementation of a Cloud Data Platform that is securely built and managed by Infrastructure as Code (IaC). A foundational platform would be expected to have robust security and monitoring as foundational characteristics. Once built, the platform would enable and support a variety of Data related disciplines including but not limited to Data Engineering, Data Analytics, Data Science, and Data Governance. Most of the cloud platforms offer tools for each Data component that was previously mentioned. Once data in ingested using a cloud ELT technology, it can further be analyzed and visualized into reports and dashboards, and subsequently shared using Cloud reporting tools. As the foundational platform matures, advanced analytics and governance roadmaps can be introduced to continue efforts to add business value and securely govern data assets.

Data and AI Maturity

While contemplating their cloud data journey and how best to embark on it, Gartner's Data Maturity model presented itself in many forms. The general idea of this model is shown in the figure below. Descriptive Analytics typically answers the question around 'What happened?' and can be answered by having access to both raw and cleansed data which can be accessible by reporting and data querying tools in a variety of languages. As an organizations analytics maturity advances, they will have access to even more historical and/or dimensionally modeled data that is co-located in either a Cloud Data Lake, Data warehouse, or combination of both to address the Diagnostic Analytics question of “Why did it happen?'. As the capabilities of the cloud platform and organizational skillsets mature even further, with AI, ML and other forms of Advanced Analytics, Predictive Analytics can be achieved by answering the question 'What will happen?'. Finally, at the peak of analytics maturity, an organization will be able to tap into Prescriptive Analytics by answering the question 'How can we make it happen?'. Notice from the figure below that Prescriptive Analytics yields the highest competitive advantage and value but is also the most difficult maturity to achieve. As organizations build their foundational platform to begin answering descriptive and diagnostics questions, they can gradually ascend into predictive and prescriptive analytics over time.

Total Cost of Ownership

Cloud ownership is a hefty responsibility which organizations must deal with after their Cloud Platforms are built and delivered. Total Cost of Ownership (TCO) is the sum of all costs involved in the purchase, operation, and maintenance of a given asset during its lifetime which helps with determining the value and Return on Investment (ROI) of a product. For a cloud platform, TCO includes cloud infrastructure related adoption, provisioning, and operation costs. When considering the benefits of adopting a cloud platform, organizations would be interested in weighing the costs and benefits of an on-premises vs. cloud vs. multi-cloud vs hybrid environment.

There are numerous TCO calculators which are available across cloud providers and 3^rd party vendors which empower customers to simply input their current on-premises workloads including server, database, storage, and networking detail to estimate current on-premises costs and compare them with estimated cloud costs through Pricing Calculators. This on-prem and cloud cost comparison, coupled with the estimated migration costs is often more than enough to demonstrate the anticipated year over year cost savings and value from moving workloads to the cloud. It is also important to consider the intangible benefits of the cloud which include innovation as a result of the various out-of-box services within the cloud and elasticity from the ability to dynamically scale up compute for peak workloads and then downscaling to keep costs under control.

While the cloud frequently offers lower TCO, it is important to keep in mind that a cloud computing model might not always be cheaper than an on-premises model and therefore a wise decision must also be based on the opportunity cost, business value, and potentially larger anticipated ROI that the cloud platform brings with it.

Next Steps

• Read more about Gartner's Data and Analytics Maturity Model by Digital.AI
• Explore Microsoft Azure's Total Cost of Ownership (TCO) Calculator
• Read more about How to calculate your Cloud TCO
• Read more about How to Manage Public Cloud Costs on Amazon Web Services and Microsoft Azure by Gartner

About the author

Ron L'Esteve is a trusted information technology thought leader and professional Author residing in Illinois. He brings over 20 years of IT experience and is well-known for his impactful books and article publications on Data & AI Architecture, Engineering, and Cloud Leadership. Ron completed his Master’s in Business Administration and Finance from Loyola University in Chicago. Ron brings deep tec

This author pledges the content of this article is based on professional experience and not AI generated.

View all my tips