Why a Data Marketplace Will Help You Deliver Data Mesh

Data mesh aims to solve issues that arise from traditional centralized data architectures: bottlenecks in data access, slow delivery speeds and scalability challenges. But many organizations are yet to realize the potential of this exciting operating model.

Adopting a data marketplace could deliver your organization capabilities that are critical to implementing data mesh.

Data mesh is an operating model for managing and accessing data within large, distributed environments, first conceived by Zhamak Dehghani of Thoughtworks in 2018.¹ It emphasizes decentralized data ownership and architecture to enable autonomy and accountability in individual domains (business units).

There are four principles to data mesh:

1)    Domain-driven data ownership

Each domain within an organization owns, maintains and shares its data products, which are accessible and usable by other domains. This approach helps streamline data management by giving responsibilities and agency to those who are closest to individual data assets.

2)    Data as a product

Product design and development principles are applied to data management to meet the needs of data consumers. Data products need to be reliable, reusable and well-documented. Data products should be designed for consumption, requiring a product-thinking approach to data management.

3)    Self-serve data platforms

To support various domains in managing their data as products, data mesh promotes a self-serve data infrastructure. This platform provides the tools and capabilities required for data sourcing, profiling, cataloguing and provisioning, which empowers domain owners to easily and effectively manage their data products.

4)    Federated computational governance

Data mesh introduces a federated computational governance model. Guidelines, standards and compliance mechanisms, such as delivery and availability SLAs, can be commonly agreed upon while delegating enforcement to various domain owners to perform independently. Standardization allows for increasing automation of governance policies, quality standards and security. This approach alleviates overhead from the domain owners, while retaining the flexibility to accommodate individual producer-consumer requirements.

Organizations adopt data mesh for better business outcomes and competitive advantages through lowering the friction and cost required to get utility and value from their data assets.

For large and complex organizations that deal with vast volumes of data across diverse business units, the key advantages of data mesh include:

Improved Data Quality – Since the ownership of data is shifted closer to its source, domain teams have both context and control, leading to better quality and more relevant data products.

Increased Agility – Data mesh’s decentralized approach allows quicker adaptation to change and faster delivery of valuable data products. Domains can innovate and respond to needs without being distracted by centralized bureaucracy.

Enhanced Collaboration – Data mesh fosters a culture where data is more easily shared and innovative uses of data are encouraged. Treating data as a product means it’s designed to be readily accessed and consumed by other domains.

Reduced Time-to-Value – The autonomy granted to individual domains to manage their own data speeds up processes from generating data to extracting and sharing insights. Faster decision-making cycles directly impact business operations and efficiency.

Scalability – Distributing data ownership and management across domains avoids the scalability limits of traditional, centralized data lakes. Each domain can scale independently to align better with organizational growth and diverse data needs.

Risk Mitigation – Splitting data into domain-centric models can mitigate risks by containing issues and drawing on expertise within specific domains.

A data marketplace is a platform where data producers can share data products with data consumers, facilitating access to diverse, high-quality and compliant data.

A marketplace is an integral component in the successful implementation of a data mesh architecture. Don’t plan for data mesh without one!

Here’s how data marketplaces can help with each principle of data mesh:

Visibility across domains

A data marketplace allows you to see all the data products managed by the producers for a given domain. You can easily see who owns the domain and its purpose.

When data products are created across many different domains, the marketplace is a critical mechanism for consumers to search for data across the entire data mesh.

Data products come to life

A data marketplace provides the clear interface data consumers need to understand and compare the characteristics of data products and how they can and should be used. Allowing consumers to see who else is using a data product and collaborate with them takes the burden off the data producer, which is a great incentive for them to want to share their data.

With a marketplace, producers can enrich a data product with deeper information around data quality, business definitions and policy information, with more granular lineage, and by integrating it with other data sources. If no data products match what a consumer wants, the marketplace can be a mechanism to request new products.

Simplified self-service

Simply finding the right data at the right time could be worth millions of dollars in value for your business. At the same time, many executives believe the culture in their organization today is an impediment to becoming data-driven. Sustainable data-centric culture depends on the ready availability of data.

A data marketplace can create a sense of community around data by bringing together data producers and data consumers from diverse domain-oriented teams in one place. Producers can set clear expectations and consumers can find, understand, trust and access data in a self-service, decentralized way.

Empowering governance

Data mesh aims to balance speed of innovation with adherence to security and governance in a decentralized, federated organizational structure. Consumers always need to understand how data should be used, especially when it's related to sensitive data, like PII.

A data marketplace provides the interface to support this contract between the producer and the consumer and even to enforce data protection policies at a granular level when data is accessed. Decentralized producers assign terms of use for specific data products, including information on the purpose of the data product, who owns it, expectations for its maintenance, end of life and so on.

A new demo shows how Informatica Cloud Data Marketplace (CDMP) supports all four principles of data mesh, including self-service, managing domains, treating data as a product and federated computational governance.

In the video, you’ll hear Amy McNee, who leads the Global Solution Architecture Team at Informatica, talk through CDMP features and functions that could make data mesh a success in your organization:

• CDMP provides visibility into data products built across many domain-oriented teams with an intuitive, self-service interface.
• Consumers can use search to find the data products that will meet their needs.
• Data products are assigned data quality scores, so consumers are clear on what they're getting.
• The categorization and terms of use are clear for each data product.
• Consumers can give feedback such as rating the data sources.
•  A built-in chat mechanism helps them collaborate.
• With the Informatica Cloud Data Access Management capability embedded in the marketplace, policies can be set across data products in hybrid or multi-cloud architectures without having to go to each individual environment.
• Consumers use checkout functionality to request access to data, providing a reason why they need the data and specifying how and when they want that data product delivered.

Watch the demo video today to see Informatica Cloud Data Marketplace in action.

Learn more about the customers mentioned in the video by visiting our customer success stories.

¹Data mesh was first proposed in the article, “How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh”,