Big Data is definitely one of the bigger trends pushing through BI and the IT world. You’ll hear this idea get tossed around a lot, especially in the BI and datawarehouse worlds. My interpretation is that it means scenarios and projects that generate a lot of relatively unstructured, freeform data. With the internet, social networks, and personal devices, the amount of data that companies and organizations are capturing is staggering and is only growing year over year.
One of the most popular technologies for managing this structurely is Apache Hadoop. This is an open source framework for networking back end servers together to act like a massive data source. The advantages are the scalability, reliability (if one unit goes down its processing is pushed over to other units), and cost. The software is open source, and its designed to work on cheap “whitebox” servers, so hardware costs are reduced. It hasn’t stopped Oracle from offering a big data appliance, based on Sun servers and Hadoop, to push itself as the big stack vendor for this. While Hadoop is a leader in the market, Greenplum and NoSQL are gaining some market share. Microsoft is also readying technology to compete. There are also some commercial versions of Hadoop getting released that offer more support to often overwhelmed IT backend administrators. The amount of increasing data and need to administer the systems is definitely straining staffs.
One issue is that the data is stored unstructured, so an additional layer needs to be implemented to allow for more traditional SQL queries. Of course, companies want to query and report on this information. Hive is another open source tool used to allow these queries to run against the database. Companies like Cloudera are offering connection abilities to Hive to allow that to be seen as a standard data source for BI vendors like Microstrategy. Another innovative player in this space is 1010Data, which utilizes a type of in-memory type technology to scale,store, and deliver information to data consumers.
Copyright 2011-2022 BusinessIntelligenceBase.com