Which Data Catalog Features Do I Really Need? Essential Data Catalog Capabilities for Enterprise Business Intelligence

Certain essential data catalog features and capabilities provide an all-important foundation for metadata management as the data landscape rapidly grows and changes. The new battlefield of our time is to achieve competitive advantage by mining business intelligence from big data found across today’s data lakes and warehouses. Yet despite making a commitment to becoming data-driven, many organizations seem to be falling short in their efforts.¹

In a recent survey of executives,² only 26.5% reported that they have a data-driven organization and culture. Organizations need to turn their data assets into data intelligence that fuels data-driven decision making for growth and profits. And what is the first step in any data-driven digital transformation initiative? To govern and share your data with trust by connecting data consumers with the business intelligence they need.

• In other words, take inventory of your big data and assess its value. Then maximize that value by democratizing data use across your organization, connecting data producers and consumers more efficiently through automation.

Organizations today find it challenging to democratize data use. Data that’s diverse and distributed across many different departmental silos and applications isn’t easy to find, let alone access. And you can’t drive data analytics and reliable business outcomes with data that’s difficult to find, understand and trust. Some of the data may reside on premises, while some of it is in cloud data warehouses and data lakes. Not knowing the origin of your data and who owns it — or whether you can trust it — makes it hard to know what data you have to support value creation initiatives that drive growth, innovation, efficiencies and more.

Another complication is the lack of visibility into the movement of data across the data supply chain. With the increase in the number of data sources, types and formats, the data landscape becomes even more complex. The importance of incorporating the modern cloud data catalog capabilities of data lineage and impact analysis cannot be overstated. Consider the following challenges that can occur when data cataloging and data lineage are not leveraged effectively:

1. Limited visibility. Distributed data makes it virtually impossible for an organization to get a complete view without automated data cataloging. Siloed data tends to present only a fragmented picture of business activities. As a result, you end up missing valuable business insights hidden within the data.
2. It affects data integrity. Data silos create data fragmentation, which results in low assurance of data quality. And such data weakness causes low confidence in trustworthiness — even costly damages — from unreliable results. Take the case of Canadian power company, TransAlta. A simple cut and paste mistake made while using spreadsheets to store, analyze and move their data cost them $24 million.
3. Risk of data exfiltration. A self-service analytics environment can be at high risk for accidental data exfiltration. Events like this represent a loss of credibility for the enterprise. This happened to the Federal Deposit Insurance Corporation (FDIC). An employee downloaded data for 44,000 FDIC customers onto a personal storage device by mistake, resulting in a major cyber breach. A data cataloging and lineage solution connected to a data marketplace can provide greater control over appropriate data use.
4. Increased costs. Data has a financial cost, too. There are infrastructure costs related to storing data. If you want to move data, you incur migration costs. It takes time and effort to collect and use data, too. So, it's wise to consider how data redundancy, maintenance and duplication will demand more resources.
5. Requires validation of data pipelines. This means you need to ensure that all approved data assets are from authorized data sources. You also need to ensure that your data pipelines are not transferring unauthorized data. There is a growing need to know the origin (provenance) and data lineage of sensitive and personal data assets for data residency policies, regional mandates such as privacy laws, and more.
6. It hinders collaboration. Data silos emerge out of the silos formed from organizational separations. And as you build on top of each layer of separation, you create both tribal boundaries and technical incompatibilities. Meaningful collaboration that can improve business intelligence is a challenge when you disconnect data producers and data consumers.

So, how do you address these issues?

Build a single source of truth for your organization’s data. A data catalog helps you address all the above data challenges and more. Data catalog tools allow an organization to create and maintain an inventory of data assets through automation. They do this through the discovery, metadata tagging, inventory and organization of your datasets. A data catalog also enables both business and technical context. It works in tandem with a business glossary to help further define the context of data.

Context can be useful for data consumers—such as data engineers or data stewards—as they find and understand relevant datasets. It can also help business teams leveraging data intelligence for value creation activities. A data catalog organizes details such as technical metadata into a simple and consumable format. This allows business users and decision makers to synthesize this info across many data dictionaries. Business users can then trust and access clean, high-quality data across applications.

Learn how data intelligence can help harness your value creation opportunities and drive better business outcomes from trusted data. Data may be the world’s most valuable resource, but it’s what you do with the raw data that counts. Successful data-driven organizations know that the first step is to discover raw data and catalog it. The next step is to curate and enrich it to confirm that it’s fit for purpose. And finally, democratize it for their knowledge workers through data sharing tools.

Use intelligent data catalog features to build trust in data

AI-powered data catalog capabilities help you capture, curate and activate data intelligence

Before embarking on an evaluation of modern cloud data cataloging solutions, you must determine what you want to do. Data catalog features and capabilities offer options that can drive opportunities to create value. They can automate the acceleration of data intelligence or democratize data sharing for an organization. But data is only meaningful to decision makers if you can enrich it with context, which comes from people and insights from good metadata management. Connecting trusted data to its context and sharing between data producers and consumers is important. It can be the difference between making right or wrong decisions with data. For example, when using the imperial versus metric systems and using the wrong unit definition to hang a shelf might not seem to be a big problem. Unfortunately for NASA, that gap in understanding cost $125 million in 1999.

Organizations want to gain competitive advantage with better business intelligence for more informed decision-making. They also want to automate routine and non-routine tasks. Embracing Artificial Intelligence (AI) and machine learning (ML) models lets them harness the power of the cloud.

These five capabilities help ensure you can make the most of your enterprise data.

• Automated data Intelligence. Automating processes including metadata-driven insights avoids the manual tasks that consume valuable time and resources when searching for answers. Automated technologies leverage data usage and queries to link or assign business context to data assets at scale.
• Data democratization connectivity. An accessible data catalog that links data intelligence with data delivery for data consumers enables data transparency to allow even non-technical users to find, access and use data. Connecting data producers and consumers enables faster, more reliable collaboration throughout an enterprise by delivering trusted data while minimizing unnecessary risk exposure.
• Data discovery and data lineage analysis. Two key pillars of a modern data catalog, these two pillars help build trust and confidence in the data that you use to derive business intelligence from big data by uncovering unknown data sources and tracing data movement to understand its impact.
• Data governance. Comprehensive capabilities that automate data stewardship can ensure trustworthy data available for users by aligning business and technical users around data purpose. Automated governance helps improve data quality and govern appropriate data exposure. Only those who are qualified data consumers should be able to access and use relevant data sets.
• Metadata curation. Organizations that adopt multi-cloud environments need to connect to multiple databases with a “” approach. Their success also depends on comprehensive visibility and data access. 

They need a data catalog that can connect to and extract metadata from legacy infrastructure as well as modern environments, whether on premises or in the cloud. This includes data warehouses and lakes, ETL and BI tools.

A modern cloud data catalog will help you find, prepare, understand and trust big data. A machine learning-based discovery engine scans cloud data stores, BI tools, ETL, third-party data assets and more. Then you can curate and prepare your data with automated domain discovery and recommendations. Business and IT users can easily discover, understand, trust and access relevant data and apply data-driven insights. Intelligent data catalog features include:

• Semantic search
• End-to-end data lineage
• Domain discovery
• Integrated data quality

Data cataloging tools integrate with the Informatica Intelligent Data Management Cloud™ to provide critical capabilities across multi-cloud environments including cloud data warehouses, data lakes and lakehouses.

Learn how Informatica has been helping organizations achieve:

• Self-service analytics. Generali, the third-largest insurance provider in the world, had two goals for their data. The first objective was to organize and inventory their data. Establishing a data catalog allowed employees to automate the discovery and inventory of data assets. Their second goal was to democratize and improve access to strengthen their data-driven analysis capabilities. Informatica data governance has helped improve data quality and provide better access to trusted data.
• Data governance. L.A. Care Health Plan is one of the largest publicly operated health plans in the United States. They have worked with Informatica to increase their data governance maturity in several key ways. They can now keep data semantics consistent across their organization. They have been able to agree on enterprise-wide definitions and terms. And they have developed business context around data for reporting and analytics use.
• Cloud modernization. A major broadcasting company  modernized public broadcasting. They leveraged customer insights to build loyalty and improve audience satisfaction. Informatica data cataloging helped increase the value of customer data by making it easier to locate, understand, trust and re-use.
• A 360-degree view of the business. AIA Singapore wanted to start a new phase in their data governance journey. They began by gaining a better understanding of their business terms, data lineage and the quality of data at the source. They then integrated their data governance with cataloging and data quality to build a complete solution.

Modern organizations collect huge volumes of data from many sources (such as ERP, CRM and streaming devices). It’s harder than ever to manage and generate data in real time with different qualities. A siloed, departmental data catalog can help, but intelligent, scalable, enterprise-class data cataloging allows you to discover, understand and trust data across:

• Cloud and on-premises platforms during modernization
• The widest range of data ecosystems and sources, and
• Business use cases from diverse disciplines needed by data consumers

Organizations are more productive when they use data catalog tools to enable predictive data intelligence. They can also save time over manual efforts: scanning tens of millions of records, inventorying hundreds of data sources and collating thousands of business terms. Organizations can also use intelligent data cataloging to quickly visualize end-to-end data lineage.

Whether you want to gain end-to-end visibility, find critical data with a simple search or share trusted data to empower your organization, Informatica can help. Watch this video to learn more.

What is a data catalog?

¹IDC Infographic, sponsored by Informatica, “Delivering Data Value by Activating Data Intelligence,” (Doc# US49588722, September 2022)

²https://hbr.org/2022/02/why-becoming-a-data-driven-organization-is-so-hard