Working with Big Data? 3 Must Have Tools to Get it Right

A recent TDWI study showed 20% of the 416 companies that responded plan to implement a big data strategy in the next 12 months (see Figure 1 below). This means that analytic solutions will need to be able to handle new types of data sources to generate business value.

Companies will be looking to technology vendors to not only accommodate the traditional data source types but will expect it to be done better, faster, and cheaper than historical methods.

You need to know your data and have a plan when it comes to Big Data initiatives. Equally as important is having the right technology tools to ensure success. With qualifying data being very specialized in Big Data initiatives, there are three classes of technology tools organizations must have to do Big Data right:

1. Data preparation tools

When evaluating data preparation tools think about how your data needs to be prepared, extracted, transformed, stored, and integrated based on the uniqueness of data sources being applied to Big Data initiatives. Standardization and quality components must also remain flexible and allow organizations to scale across non-standard data sources to implement Master Data Management strategies that support Big Data

2. Data storage tools

Begin exploring new data storage options beyond traditional databases that are better suited to support Big Data. Consider whether traditional DBMS is sufficient or would your strategy be better served with a Hadoop implementation? Or, whether in-memory is an option for your analytics solution. 

This does not mean that an implementation must be one or the other. In fact, different methodologies for data storage will and should be used in Big Data implementations. You need to consider the pros or cons of how the data will be used and which technique facilitates better usage.

3. Data query and visualization tools

Data query and visualization tools have changed tremendously as data volumes and types of data have increased. Vendors must provide technology for organizations to quickly visualize voluminous amounts of data at the same or faster speed of traditional BI and query tools. Whereas query tools require the user to know what they are looking for and ask questions of the data; visualization tools allow the user to easily explore complex data to get fast answers. This can be accomplished through various techniques such as data exploration, reporting and dash-boarding, and well as advanced analytics.

The bottom line is that technology tools such as the ones mentioned above must enable your overall Big Data strategy to ensure success.

*TDWI Best Practices Report, Forth Quarter 2013: Managing Big Data