With Informatica PowerCenter 9.6, you get the data warehouse of your dreams in 30 days or less by aligning business and IT as co-development partners
One of the largest pharmaceutical firms in the world had a data integration problem. A big one. Its data on critical drug trials was fragmented, delayed, located in multiple systems, and required extensive manual manipulation before becoming useful. Incompatible legacy systems impeded data integration and prevented executives from getting an accurate view of the business. Any time data was moved or transformed, simple “stare and compare” methods were employed to ensure validity. Furthermore, on-going post-implementation management and monitoring of operations was manual, and thus a risk.
This had the potential to be disastrous. Revenues for a new, patented drug can be as much as $1 million per day, translating to enormous losses for every day a trial is delayed or unnecessarily extended.
Clearly something needed to be done. The pharmaceutical firm needed an integrated data solution—and fast. And it couldn’t afford to fail. Yet as the volume, velocity, and complexity of data grows, the more ominous the statistics become. A recent survey of 300 businesses from small to enterprise found that 100 percent of Big Data integration projects exceed schedule, 100 percent go over budget, and half fail entirely.
100 percent of big data integration projects exceed schedule, 100 percent go over budget, and half fail entirely.
Here are the biggest problems:
- IT is overburdened. At any given time, IT is being bombarded with requests for projects. Getting IT’s attention can be hard, and you often end up at the end of a long queue. When projects need quick turnaround, many business users despair—and turn to building “shadow” IT operations within the business unit. These, however, raise additional challenges for the company as a whole. One healthcare company found it had more than 30,000 data marts filled with sensitive patient information, thanks to shadow IT initiatives.
- The business doesn’t know upfront precisely what it needs. Many times, when the initial requirements for a data integration project are collected, business users are unsure of what they want—and many times, unsure of what’s possible, technically. Not until they see the first working prototype do they begin to understand themselves what their true requirements are. This isn’t to blame business users—it’s part of the creative process of developing something new. But few companies have taken this reality into account.
- Iterations and course corrections are non-existent or few and far between. A related but separate issue is that a lot of time passes before IT shows business users what they’re doing. As a result, IT can go too far down the wrong path before checking in and validating its work with a business counterpart. Mistakes and wrong turns are discovered too late in the process, inevitably producing delays and cost overruns (see Figure A).
The solution: Infuse agility throughout the data integration process.
Agile software development is a well-known method of developing software that depends on iterative and incremental collaborative efforts between cross-functional teams. In data integration, this means
“CIOs and Big Data: What Your IT Team Wants You to Know,” Infochimps, January 2013. http://www.infochimps.com/resources/report-cios-big-data-what-your-it-team-wants-you-to-know-6/