What's the Big Deal with Big Data?

Introduction

“Big Data” has become almost a buzzword in the computing world in recent years, so much so that that the term has become nearly synonymous with information technology and computer science. Still, even though the term big data has become inextricably linked to the IT world, not everyone has a concrete understanding of what big data actually is and may wonder what the big deal with big data is all about.

Defining Big Data

Various analysts have both tried to define and disparage the term big data. IBM data analysts have tried to define big data in the four dimensions of volume, veracity, variety and velocity. However, it is not such a simple thing to define something as wide scaled as big data is such narrow parameters. Those who dismiss the term say it I not about the amount of data itself, but what and how operators put the data to use. As is the case with most things, the actual facts tend to reside somewhere in the middle.

Some say the term does not need to be defined, as the word “big” is intuitive enough for most people to understand the significance. However, because it is so all-encompassing, understand all that big data is, and can be, will benefit from some clarification.

Big data is perhaps best defined as the merging of multiple trends in data storage, management, growth, AI, cloud and mobile computing and analytic systems and tools. In short, big data is the convergence of everything that makes up the computing universe. This also includes the transition away from disk-style data and memory storage and processing for both on-site storage and cloud-computing utilities. Big data is both relational and NoSQL; both DBMS and Hadoop and Spark; and oriented towards both commercial and open-source software applications.

The critical takeaway here is to understand that these transitions are not necessarily replacing the old tech with new, but adding to the existing technology. For example, relational databases are not, nor becoming, obsolete and should still be used as a primary component for a big-data platform strategy.

Big data is not just the tech itself, by how it is implemented by database manages and IT professionals. Big data is the shifting away from using mainly, or only, internally-supplied data for assimilating and integrating information from various sources. Big data is the adding of analytical data to transactional information, combining structured and unstructured data and compounding persistent and transient data.

Data does not need to have a formula applied to see if it qualifies as “big,” nor does it have to be more than SQL to qualify as big data. Big data can be as simple as one database operator for a national trucking company performing logistical analytics or it can be as complex as a team bringing together every bit of needed information for the first manned mission to Mars.

Big data is simply about how DBAs can administer, analyze, augment, manage and integrate and all of their data to make better decisions to expand their productivity. This may also include integrating other relevant data from social media and other sources. Transactional database systems are designed to record a business’s daily transactions and these can produce a large amount of data that is becoming invaluable to large company’s for tracking and processing their customers, clients, suppliers, producers and shippers.

Big Data Tools on a Small Budget

Database administration and automation tools will be indispensable for business to succeed as the amount of information database managers are required to handle continues to increase. Like big data itself, every tool that a database manager uses in the management and administration of assimilating and integrating data are big-data tools. Managers simply need to focus on what is most important when working within their budget and add only the tools they truly need, or need the most at present.

As applications are often transitory, many DBAs are often skeptical of industry trends that may be hot today and forgotten tomorrow. However, as processing applications, like many new AI programs that just don’t pan out, are not always around for the long term, data, in some form, will always be relevant.

Conclusion

Clearly, big data is vitally important in today’s digital world. DBAs can, and should, embrace the ascension of big data as a way to not only improve their data quality and their ability to manage it, but as a way to improve their database infrastructure and ultimately their company’s efficiency and long-term viability. Someday big data may be supplanted by something someone has yet to think up. However, for now, whether it is being used by small companies to expand their reach around the world or taking humans to other planets, the big deal with big data is it is important for businesses of all sizes.