Pilot projects turn business owners into big data advocates

Organizations need to limit expectations around the size and scope of big data pilots for maximum effectiveness.


“At its core, a big data pilot is a sanctioned Shadow IT project and, as such, enjoys the efficient buy-in, decision-making, and execution afforded by a decentralized project.”

The explosive growth of data, while continuing to make headlines, is no longer news. It is within reach of most organizations. The challenge is finding an efficient and effective way to enter the big data arena because it is no longer just early adopters taking advantage of its value. A smart place to start is with a clearly defined big data pilot, led and owned by the business, enabled by technology, and governed by you.

An increasing number of organizations are moving past early adoption stage, according to IDG Enterprise’s 2014 Big Data Survey.1 More than 20 percent of respondents say their big data projects have improved both the quality and speed of decision making at their organization.

Meanwhile, 49 percent expect that big data usage will be widespread across their organization in three years. And another 26 percent expect mainstream use at one or more business unit, department, or division.

While big data initiatives need executive support, a big data pilot will falter if there is an attempt at inclusivity across the organization. At its core, a big data pilot is a sanctioned Shadow IT project and, as such, enjoys the efficient buy-in, decision-making, and execution afforded by a decentralized project.

Think big, start small

Your organization’s pilot needs a line of business manager with clear objectives to own the pilot and build out a cross-functional team of willing and energetic participants. Identifying a business case that resonates across the organization will help the pilot gather momentum. And stakeholders who are passionate about the project will not only see it through but will also stand to gain from it, even if it does not succeed as conceived.

Although by definition limited in scope, a pilot needs to be framed so it can be easily operationalized. This is where you come in. You need to ensure that data governance is built into the pilot from its inception. From a strategic perspective, make recommendations that will help marry the outcomes to your more holistic data governance efforts. And on a more tactical level, ensure it plugs into your canonical data model.

Stay focused

Having a self-contained pilot with clear boundaries from the outset is key. That way, participants will be less concerned with the uncertainties of big data and more focused on the defined outcome with tangible—and measurable—value.

Make sure your organization’s pilot keeps the following high-level goals in mind when defining requirements:

  • Define your data and make sure it is accessible, safe, and familiar
  • Limit the length of the pilot
  • Ensure everyone is committed and understands potential risks
  • Incorporate company data privacy and security policies into the plan
  • Respect existing rules governing compliance and regulation
  • Gauge whether your model can be repeatable and applied to other business use cases if successful
  • Have contingency plans in effect should any core business process be jeopardized
  • Capture how you plan to measure ROI—qualitatively, quantitatively, or both
  • Identify existing processes and actions that you hope to automate
  • Build a financial projection: Factor costs incurred by the pilot as well anticipated savings moving forward
  • Build an exit strategy into your pilot that identifies the point where diminishing returns outweigh moving forward

Be risk-aware, not risk-adverse

Being innovative requires some degree of risk. To realistically gauge the impact of big data, you need to apply what Ernst & Young refers to as a “risk lens” to the business. They point out that, “Companies that succeed in turning risk into results will create competitive advantage through more efficient deployment of scarce resources, better decision-making, and reduced exposure to negative events.”2

Building fail-safes into the pilot is one way to mitigate risk. For instance, working in short, iterative cycles can help you narrow down the source of any problems, should they arise. It can also ensure your pilot generates quick results.

As your organization gains exposure to successful pilots, you will benefit from more ambitious efforts that will have larger implications. Being smart about governance from the beginning will encourage scalability while permitting the agility that comes from a clearly defined and operationalized model.

For details on how to grow your big data efforts while lowering costs and minimizing risk, read “The Safe On-Ramp to Big Data.”

Related content


Fannie Mae

Informatica helped Fannie Mae ensure clean and correct data is collected and integrated from 100+ data sources


Measure the success of a merger by the quality of your data

Follow these 4 steps to ensure you use best practices and emerge from a merger or acquisition with a trusted version of your data.


Rise of the machines: The Internet of Things

Are devices that track our every move poised to unlock new potential in humankind or are they just downright invasive?