What’s New With Informatica, AWS, and re:Invent

Seven years ago, Amazon Web Services held the first re:Invent in Las Vegas. I remember it well – I was there as a speaker, talking about the first big data products Informatica had released dealing with hierarchical data processing at scale. I can’t overstate how significantly things have changed since then – for Informatica, for AWS, and for the industry as a whole.

Informatica’s data engineering evolution

In 2011, we released two big data products focused on key problems related to processing data at scale. Today, Informatica has the industry’s broadest end-to-end data engineering capabilities: discovery, ingestion, integration, streaming, quality, and data preparation. As I’ve written before (What’s the Most Challenging Part of AI? The Answer in 2019 May Surprise You), driving transformative insights from your data requires the full capabilities of a comprehensive data management solution.

From our original product offerings, we’ve innovated to deliver the full set of capabilities you need to be successful. Whether you’re modernizing your enterprise data warehouse to Amazon Redshift, building a cloud data lake on Amazon Simple Storage Service (Amazon S3), or managing a complex on-premises and multi-cloud data environment, you need data engineering solutions that span the entire data supply chain.

In 2011, processing was done leveraging on-premises MapReduce, facilitating concurrent processing by splitting petabytes of data into smaller chunks and processing them in parallel on Hadoop commodity servers. Informatica bet on Spark and Structured Spark technology in 2015 and now in 2019 delivers multi-cloud serverless processing.

Recognizing that “big data” has become synonymous with on-premises Hadoop, we’ve shifted our portfolio to focus on data management in the cloud with a suite of industry-leading data engineering solutions.

AWS—and the industry’s—big data disruption

At that first re:Invent, one of the companies I stood onstage with was MapR—which no longer exists. The age of managing on-premises Hadoop clusters is rapidly fading. Instead, the industry is moving to serverless technologies. And there’s tremendous growth in cloud-native platforms like Amazon EMR and Databricks.

Seven years ago, AWS had just released Amazon EMR, its cloud-native Hadoop distribution. Today, Amazon EMR dominates the Hadoop-as-a-Service market (with some estimates putting market share at close to 90%) and Hadoop in the cloud is now the dominant pattern.

And while 6,000 people attended the first re:Invent, the event now is now projected to attract 65,000 attendees—and tickets sell out every year. We’re excited to be there this year with our partner Deloitte (come by and chat with our experts at booth #1305).

What the AWS-Informatica partnership means for you

We’ve had a great history of innovating together with AWS—and look forward to many more years of partnership. As part of writing this blog post, I went back through my Twitter feed and discovered a host of moments from past shows—times when I was interviewed on theCUBE at re:Invent, or when AWS CEO Andy Jassy gave a shout-out to Informatica from the main stage, or when Informatica was recognized as a “Think Big” partner.

But what really brings the partnership to life are the customers and their stories. In the lead-up to re:Invent, Informatica was finishing up its Informatica World Tour events. And it was great to see the joint Informatica-AWS customers there to share their stories. We had companies like T. Rowe Price, Freddie Mac, and Sage present — and this comes on the heels of Informatica World presentations by Avis Budget Group and Community Technology Alliance, who are using solutions from AWS and Informatica for transformative business results.

What comes through from all of these customer stories is the theme of modernizing your analytics environment. That is the main customer challenge—and it’s where our partnership with AWS can bring the most value to you, whether you’re looking to modernize and replatform, launch a departmental agile analytics initiative, take a cloud-first approach, or undertake a lift-and-shift data center replacement.

I invite you to take a moment to learn more about our offerings for AWS. And if you’re going to re:Invent, stop by booth #1305. You can book a meeting now to meet 1:1 with one of our experts and learn about the latest AI-powered technology to accelerate AWS migration at scale.