A philosopher in ancient Greece has a thought about the nature of reality and writes it down. A rice farmer during China’s Tang Dynasty records the number of rows in his rice fields. An eighteenth-century writer documents her experiences travelling to a foreign country in a book. In each instance, data was created. And for centuries, not much changed. Then, innovations in data computing and storage during the late 1800s and early 1900s laid the foundation for some changes—some big changes.
In just a few short decades—a blink of an eye on the scale of human existence—our ability to store and process data has skyrocketed. Today, it’s increasing exponentially. Mobile phones, smart watches, Google searches, GPS systems in vehicles, parking meters, social media accounts—all these things are leaving behind a stream of “data exhaust,” so much so that virtually every action we take today creates data. To put things in perspective, we’re now creating more data in roughly a year than had been previously created in all human existence.
This is big data. But what does this mean for our society? How will we store it, understand it, and act on it? And will we use this tool for good?
Where does big data come from?
In the past, creating data was an active process. In other words, when you were writing something down, recording a song, or taking a picture of your pet turtle, you knew you were creating data. And even today, there are still ways in which we actively contribute to our society’s collective database of information—be it submitting a paper for an online class or blogging about your family vacation to Alaska. But, increasingly, sources of data are becoming almost invisible to us. Our lives are being measured in ways we don’t even realize.
If you wear a smartwatch, for example, information is constantly being recorded about your heart rate, sleep habits, and your location throughout the day. Each time you perform a Google search, you’re not just gaining information; you’re giving it, too, because Google learns about you based on your searches. The same goes for purchasing things on a credit card, plus whatever you do with the more and more smart devices released.
Making sense of it all
With such vast amounts of data available to us, big data must be the immediate solution to all of society’s problems, right?
While it is a powerful tool, big data actually presents its own set of challenges and important questions that must be addressed. First off, there’s the issue of storing all this information that’s being measured. According to the International Data Corporation (IDC), we created 1.8 zettabytes (or 1.8 trillion gigbytes) of data back in 2011, enough data to fill almost 60 billion 32GB Apple iPads. With those iPads, you could build a Great iPad Wall of China twice as tall as the original. And the IDC now forecasts that in 2020 we will generate 40 zettabytes a year.
But simply having massive amounts of data available to us doesn’t mean that we’ve learned anything new from it. Gleaning valuable insights from these gargantuan data sets requires special tools to make sense of them first. Data scientists use special algorithms to look for patterns or trends in the data, allowing them to pick out specific bits of helpful information and hone in on what they mean, as opposed to getting lost in a sea of information. Tools like these algorithms, as well as others like artificial intelligence and machine learning, are the lens through which we can see how to apply insights from big data to make informed decisions.
This changes everything
As we continue to refine the tools we use to store, organize, and interpret all this data, we’re also getting a clearer picture of how to use big data for good. For example, as Earth’s population continues to increase, it puts a serious strain on our resources. But big cities can use this data to gain insights about their residents’ tendencies, allowing them to more efficiently distribute resources. Visualize a city bus system, for instance, that can detect riders’ tendencies and automatically direct buses to the appropriate locations at the appropriate times. You don’t need to plan around the bus schedule if it plans around you!
In the health field, where all kinds of measurements are being taken on patients regularly, there is tremendous potential for more effectively storing and interpreting all that data. If we could record the vital signs of millions of patients over a long period of time, then when we sift through the data, we can look for patterns that could allow us to predict when an individual is at risk for a certain condition. Additionally, doctors would have the ability to use big data to more quickly identify the best drug for treating various conditions. And that’s just the tip of the metaphorical iceberg. Big data has the potential to allow us to make predictions about when large-scale natural disasters are likely to occur and even drive the way we respond to these types of situations.
With great data comes great responsibility
When we use the right tools, big data can be an overwhelmingly powerful force for good in our world. But as Uncle Ben told Peter Parker, “with great power comes great responsibility.” The ability to quickly gather useful insights from large swaths of data can save lives, but it also attracts those with business interests in mind. For many parties involved in big data, the main objective is finding out as much information about you as possible. Largely, this is in the hopes of marketing products that, based on information about you, they think you are more likely to be interested in—and thus more likely to buy, and thus that makes more money for the business.
This idea in and of itself isn’t necessarily something that we should be afraid of; after all, you may find it convenient that some sporting goods store places ads for shin guards on your social media feed if you’re an avid soccer player. You’re also not being paranoid if the thought of companies collecting so much information about you makes you uneasy.
But don’t panic just yet: big data is still a relatively new concept, and businesses and governments alike are still just beginning to work out how to handle these newfound powers. The important thing is that we are aware of big data’s power and inevitable role in our world’s future in the hopes that we can ensure it is used for good.