Data is powerful: Used right, companies, scientists, research institutions, or even the average person can make informed choices based on data discoveries. With enough information parsed through the right tools, medical professionals can track trends in the spread of illnesses, companies can predict customer needs and wants, or environmental scientists can discover new correlations between human actions and environmental effects — the possibilities may well be endless.
However, to achieve these goals, the data sets processed need to contain a vast amount of data — not just terabytes of information, but peta- or exabytes; the sort of data that is usually too unwieldy for traditional relational databases and data processing systems to handle.
It is just not a matter of databases, either: When handling massive amounts of information, concerns are not limited to the storage of it but also its analysis, visualization, searching, sharing, and more. These issues are condensed into the “Three Vs” of managing big data: Volume, velocity, and variety. (more…)