What Is Data Mesh?
Data mesh is a solution architecture that focuses on building business-oriented data products. It accounts for the ubiquitous nature of data. It also advocates for putting data at the core of product design. Under this architecture, data product owners / domain owners define data product requirements up front. This helps organizations create domain-oriented self-service data platforms to accelerate data delivery.
Leveraging your data helps you make informed decisions, drive tangible business outcomes and stay competitive. For organizations looking to maximize the value of their data, implementing data mesh can help. But how do you know if data mesh is the right architectural approach for your company? Whether it’s a fit depends on your prior investments, culture and operating model.
Data mesh allows you to stay agile and adapt to changing business requirements as they grow. From a technical standpoint, it automates data governance, quality and data management processes. This automation minimizes human intervention. Data mesh also ensures that data infrastructure can be provisioned rapidly.
Implementing data mesh also enables business data users to better discover, understand and trust their data. It distributes data and promotes sharing across departments and with business partners. This plays a key role in the success of your analytics program. But to be effective, data mesh requires a shift in organizational structure. Your culture needs to change internally to align with the methodology of a data product mindset. Embracing this shift ultimately helps to inform decisions based on your data.
What Is Data Mesh Architecture?
Data mesh architecture introduces a shift in the enablement of data analytics within the enterprise. It is built upon the following design principles:
- Focuses on being domain-driven. It decentralizes analytical data ownership by placing it on the areas of the business closest to the data. These departments may be the source of the data or the main data consumers.
- Treats data as a product. It enables the creation of accessible, discoverable, trustworthy and secure data products. This gives the data products demonstrable business value of their own.
- Provides federated computational governance. It ensures secure, trusted and reusable data — regardless of who “owns” it within the organization.
- Uses self-service infrastructure. It provides a new type of self-service platform. This empowers departmental teams to manage the data lifecycle, from acquisition to democratization.
What Are Three Key Considerations for Data Mesh?
Accountability: Data mesh architecture places accountability as close to the data source as possible instead of on your data scientists. It does so via a behavioral framework. And it relies on business domain owners to be accountable for specific domains, or subject areas, of data. This helps establish effective data governance, which requires domains to be clearly defined and intentionally managed.
Methodology: Data mesh methodology focuses on the decentralization of data. This often means using a federated data governance program. Under this model, a central data governance team provides enterprise standards for distributed entities within the organization to follow.
Technology: Data mesh technology aligns with its self-serve data infrastructure principle. This delivers data products that follow data-focused policies and fit into the federated model described above.
For a more in-depth description of these three areas of consideration for data mesh, check out this blog post, “Data Mesh and Data Fabric – A Comparison.”
Why Does Data Mesh Matter?
There are several issues that data mesh can help companies address, including:
Data latency: Data mesh helps reduce the delay in provisioning data. It also enables you to address a lack of access to the right information at the right time. And it prevents outdated data and varying definitions of data assets that can lead to non-utilization and a lack of trust in data.
Productivity issues: Data mesh removes hinderances to productivity and innovation. This is due to a federated approach to governance and compliance. It also eliminates dependence on subject matter experts and internal tribal knowledge.
Data landscape complexity: Data mesh addresses the complexity of your data. Data needs to be curated, managed and protected for the right use. Data mesh recognizes that data infrastructure at large organizations is very diverse and changes frequently.
Business and IT gap: Data mesh helps drive alignment between business and IT teams. It does so by assigning ownership of domain-oriented data products to business experts. More and more, business users are consuming data and seeking more business context for data. Various business functions request data in different ways to support their decision-making needs. Data mesh helps meet these business demands by balancing centralization and decentralization.
Lack of adaptive governance: Data mesh eliminates the top-down data governance that fails to democratize data across the organization. Governance with data mesh facilitates data sharing and the effective use of data within your organization.
Internal culture: Data mesh helps drive a cultural shift. Implementing data mesh helps eliminate the barrier to leveraging data as a strategic asset due to current mindsets as well as organizational and operational models.
Data teams disconnected from domain teams: Data mesh helps address the lack of domain expertise in centralized data teams, as they often don’t fully understand the business context.
What Are the Benefits of Data Mesh?
Distributed data mesh architecture links your strategic business objectives to your data products. As a result, it delivers fit-for-use data products that meet your business needs. And it scales the delivery of your data products by decentralizing data ownership and providing domain-specific expertise.
Data mesh also improves agility by reducing complexity. It breaks down monolithic, centralized data architectures. This prevents bottlenecks, which makes the whole development process more efficient. Data mesh also enables a flexible data governance operating model with federated governance. This allows you to enhance the model to meet your unique needs.
What Is Data Mesh vs. Data Fabric?
Data fabric and data mesh each represent a different architectural approach. Both can be beneficial, depending on your business needs and drivers. They both aim to improve an organization’s ability to leverage data to make more informed decisions and drive tangible business outcomes. You can leverage one or more of these approaches to maximize data based on prior investments, culture and operating models.
Specifically, data fabric as a design concept and architecture addresses the complexity of data management. It minimizes the disruption to data consumers. It also ensures that any data on any platform can be effectively combined, accessed, shared and governed.
Data fabric architecture is enabled by:
- AI/machine learning (ML)-driven augmentation and automation
- An intelligent metadata foundation
- A strong technology backbone (i.e., cloud-native, microservices-based, API-driven, interoperable and elastic)
Implementing data fabric helps you democratize your data. It fully leverages data assets by optimizing access to distributed data across a multitude of systems and platforms. Data fabric is agnostic across deployment platforms, data processing models, data delivery methods, locations and architectural approaches. It enables you to use data as a strategic asset by reducing complexity.
- Integrates and connects all your organization’s data to facilitate frictionless data sharing
- Accelerates self-service data discovery and analytics by making trusted data accessible to all users
- Optimizes data management costs and initiatives via intelligent automation of data management tasks
- Delivers real-time insights by optimizing the data lifecycle and accelerating data-driven application development
One main difference between the two is that data mesh treats data as a product, whereas it’s more of an afterthought with data fabric. Data mesh advocates product-centric thinking for data as a core design principle. As a result, data is maintained and provisioned like any other product in the organization with data mesh, which is not true with data fabric.
For more on the difference between data mesh and data fabric, read these blog posts: “Data Fabric vs. Data Mesh: 3 Key Differences, How They Help and Proven Benefits” and “Data Mesh and Data Fabric – A Comparison.”
What Data Mesh Challenges Might You Face?
As with any new approach, you should beware of potential pitfalls before you begin implementation. Here are a few best practices to ensure success with data mesh:
- Align your business use case and value proposition. Start with a business domain where agility is critical and business domain engagement is vital. Align your business KPIs with your data KPIs.
- Balance centralization and decentralization of your data, as both approaches bring benefits.
- Ensure data governance acts as a foundation for your data mesh architecture.
- Leverage existing data management investments, best practices and experiences within the organization.
- Prevent the creation of additional silos with a decentralized data approach. Ensure domains don’t become isolated from data teams.
- Standardize data infrastructure across domains, if necessary.
- Define roles and responsibilities clearly across your domain and data teams.
How To Get Started with Data Mesh
Informatica is uniquely positioned to support your data mesh architecture with our Intelligent Data Management Cloud™ (IDMC). We can help you future-proof your investments in data and analytics. Explore more about how data mesh transforms the business value of data.