Why a Data Marketplace is Needed to Share and Deliver Data Products

We’ve all heard that data is the lifeblood of today’s digital business. But how do we get the right high-quality data into the right hands in a timely manner across complex organizations? How do we ensure it meets internal and external policy compliance requirements for appropriate use while accelerating modern business initiatives?

If you are not currently thinking through the challenges involved and working to solve them, you may be missing an opportunity to accelerate the value that data products can bring to your organization. Let’s dig a bit deeper into this topic and explore why a data marketplace can fast-track your data strategy.

First, let’s level set. There are varying definitions of data products in the market. We believe the best definition of a data product is:

A container or “product” that provides business value and encapsulates all the data-related functionality needed for achieving the business objectives. This includes — but is not limited to — reports, machine learning models, user interfaces, governance policies, data pipelines, data schemas and access mechanisms.

Enterprises see data products as an effective way to reuse and build upon, share and democratize, govern, and audit data assets across different use cases as Figure 1 illustrates.

Figure 1. A diagram depicting the business value of data products.

Let’s take a look at the most essential factors to scale data products.

Adoption – The most important factor for data products to effectively be leveraged across the enterprise is adoption across domains, across business and technology functions, and across use cases. The biggest bottleneck to adoption success is trust and transparency. This is especially the case within enterprises that are traditionally decentralized and seemingly self-sufficient. The temptation to build within silos persists in these organizations. Breaking down silos is key to adoption and building trust in data sharing. Uncovering hidden data sources and building literacy can help accelerate data products that benefit the organization more widely and democratize their use to propel business.

Data Literacy – To ensure that data products are genuinely trustworthy and can be effectively leveraged by business and data consumers across the community, transparency in understanding (or data literacy) is crucial. If consumers lack understanding of a data product, it’s creation and transformation, its data contract (including terms of usage and service level agreements and metrics, such as quality and observability) – and its user base with feedback for additional context, the data products run the risk of being expensive or irrelevant to manage and maintain in comparison to its utility.

What is Most Critical for Enabling Data Product Trust and Transparency?

A key driver of trust and transparency is enabling a single environment for data sharing. This approach enables collaboration between data producers (or data owners) and data consumers, fostering a thorough understanding and trust their data products. It facilitates the definition of appropriate use in alignment with enterprise strategy. As enterprise applications, these are often referred to as a data marketplace or exchange, and they provide the means to connect data consumers to high-quality data using automated tools.

A data marketplace delivers value to data consumers by enabling the following capabilities:

1. Discovery – The ability to find and understand the right data products needed for their business use-cases.
2. Understanding – The ability to know how these data products were built by inspecting the underlying metadata, lineage, quality and so on, verifying if it meets the needs and trustworthiness for their consumption and use requirements.
3. Collaboration – The ability to interact efficiently with data producers and request extensions or guarantees, such as the frequency of refresh for the data product. This includes the ability to rate assets as more relevant and reliable, raising data literacy in the process, and eliminating less relevant products to increase efficiency and ROI.
4. Subscription and Consumption – The ability to choose the right delivery targets and have the data delivered in a safe, reliable and secure manner.
5.  Iteration – The ability to evolve and either receive new versions of the data product or update the data contract with data producers.

-------------------------------------------------------------------------------------------------------------------------------------------------------

Did you know? Data Products Help Scale AI!

According to McKinsey, “Data products are the secret sauce for scaling AI. They help deliver data-intensive applications as much as 90 percent faster, at 30 percent lower cost, and with a reduced risk and data governance burden. A data product delivers a high-quality, ready-to-use set of data in a way that people and applications across the organization can easily access and consume.”¹
--------------------------------------------------------------------------------------------------------------------------------------------------------

Data producers also benefit through greater data sharing efficiencies and alignment to organizational strategy, such as policy compliance and ensuring fitness for purpose.

1. Building Blocks for Data Products – A data producer doesn’t always want to start from scratch and is looking at which data management building blocks (publishing mechanisms, data catalogs, data quality libraries, data integration templates, etc.) can be leveraged.
2. Specifying Data Contracts – Data producers are looking at easy ways to expose the terms of a data contract such as delivery mechanisms, certified usage and purposes, policies of usage, etc., to help ensure data product use is aligned to enterprise strategy.
3. Data Access Management – A critical concern for data producers and owners is to define and enforce data access management policies for data prior to consumption, incorporating the context of the type and location of use, and other attributes such as sensitivity. A reusable and scalable manner to identify consumers who are subscribing (or checking out) data products while applying fine-grained controls to limit risk exposure is crucial.
4. Insights and Consumer Feedback – Insights and analytics are essential for data producers to be able to understand consumer demographics, likes and dislikes, and requests for iterative extensions and improvements.
5. Reduced Overhead for More Efficiency – Often data producers, and by extension data stewards, are bombarded by requests for access to data. Streamlining operations, including the automation of data product provisioning, empowers data producers to support the organization, while enabling data consumers to achieve faster business outcomes by economizing the “last mile” of data delivery.

In summary, it boils down to a simple yet comprehensive approach to provide a “handshake” between data producers or owners, and data consumers. Imagine an Amazon.com-like “shopping experience” for data products, complete with ratings and reviews, to expedite recommendations for the next-best appropriate data products, leveraging automation to drive greater value. 

Informatica Intelligent Data Management Cloud (IDMC) contains end-to-end data management services which exchange metadata and data intelligence between its components in a functionally scalable manner. A critical business-facing component to encapsulate these services into a data product presentation portal and data sharing collaborative experience is the Cloud Data Marketplace (CDMP).

Figure 2. Lifecycle of a data product

As shown in Figure 2, CDMP provides a portal which enables both data producers and consumers to be able to identify, design, build, deploy and iterate its data products in a composable, extensible manner.

Leveraging reusable components enables data consumers to discover, understand, check out, access – and provide feedback to data producers – in an intuitive manner that an enterprise user can more easily follow. This is accomplished through automation and a business-friendly interface, irrespective of data maturity.

As an example, CDMP is like Airbnb. Similar to how Airbnb orchestrates the exchange between a guest and housing provider, CDMP aggregates and collects data products from various ecosystem data exchanges and data storages, orchestrating exchanges based on data consumer needs. The seamless experience, trust and ability to easily leverage a contract between parties without often requiring direct interaction is a main reason for Airbnb’s adoption and success. These are the facets that CDMP offers today to enterprises with its data product adoption.

CDMP includes key characteristics, as depicted in Figure 3:

Figure 3. The data products capabilities to scale.

Several data marketplaces and data exchanges exist today in the enterprise ecosystem. Therefore, it is essential to understand the value of CDMP from Informatica and how it can complement and abstract other data marketplaces and exchanges.

Illustrated in Figure 4, CDMP is an ecosystem, data storage and exchange-agnostic data marketplace which acts as the metadata layer irrespective of where the underlying physical data needs to be leveraged and exchanged. Even more critical is that it can orchestrate and leverage other data exchanges, such as Snowflake Data Exchange or AWS Data Exchange. In the blueprint diagram below, CDMP acts as the metadata and data product portal managing transparency and self-serve governance. It then orchestrates the data exchanges to physically manage and exchange the data on consumer checkout. This is also where data access management enables robust, policy-based access in the cloud using a single pane of glass.

Figure 4. A governed, quality-checked “Data Products Sharing” blueprint.

No matter what your key business initiative — from powering AI with trusted data to analytic insights from data sources, to improving customer experience and beyond — data products are the fuel to drive better business outcomes for today’s digital businesses. 

We invite you to learn more about Informatica Cloud Data Marketplace and how we can help empower your organization by accelerating trusted data product delivery at scale.

Download “Enable Smarter Business Decisions with Trusted Data,” from Informatica and Accenture. Hear from Intelligent Business Strategies CEO, Mike Ferguson and Informatica’s Ian Stahl in our webinar, “How a Data Marketplace Helps CDOs Drive Data Value, Innovation and Revenue.”

_{¹Six signature moves led by the C-suite can build organizations that will outperform in the age of digital and AI.}