The world’s most competitive businesses will be looking at the data they get from the Internet of Things (IoT) to help boost their big data initiatives. But they will need to handle copious amounts of data and they will need to store these data somewhere. Your company should start thinking now about where and how to store all the data that you will get from the Internet of Things.
You are going have to prepare two types of data storage for two very different types of data. One of these data types is large-format data. The Internet of Things will be capturing videos and photos taken from connected cameras, smartphones and other devices. Then there are the small-sized data from sensors. Sensors may collect very small bits of data, but they typically create thousands or millions or even billions of these small files.
These two types of files have different needs. You would need to keep videos and the like in storage that is able to handle large data files that are accessed sequentially. But you would need a high-performing, random-access storage system for sensor data in order to perform efficient analysis. The storage also needs to be searchable so you can quickly identify the data your analysis will need.
Most datacenters, however, are used to handling only one type of data. For instance, one datacenter will know how to handle large image files and put them on network-attached storage (NAS) systems that have large capacities, while another datacenter might be expert at maintaining billions of small sensor data on high performing NAS systems. Given the mix of data types coming from the IoT, a datacenter will need to learn how to store both types of data.
And even NAS systems are changing. More and more datacenters are looking at object-based storage for large data types and all-flash arrays allow you to get faster analytics.
There are two other issues that complicate data storage when it comes to the Internet of Things: protecting your data from loss and making your data secure.
Most of the data you get from sensors should be kept safe and backed up. You cannot recreate these data if you lose them; traffic data or temperature readings from last month or last year will not be the same today. Unfortunately, todays back up applications would simply go berserk if you ask them to back up billions of small files.
One alternative you might find easier is to use tapes in creating copies of your data. Once data is recorded on the disk cache, you can create copies of these onto different tape devices. That way, you have great data protection and you can retain these for a very long time.
While securing your data from unauthorized access is also important, few people are taking the security of data gathered from the Internet of Things seriously. Most companies today are bent on getting their IoT initiatives up and running; they pay no attention to the security of their data. Yet, according to a case study by security firm Duo Security, an Internet of Things device may have as many as 19 vulnerabilities, including unencrypted consumer data, information leaks, and failure to use authentication. This is leaving IoT data ripe for plunder.
Data coming from the Internet of Things is quickly going from something that is “nice to have” to something that is “essential to the business.” If you think that IoT data is something that could make or break your business, try to think about how to store the data in a way that you could use it best. Meanwhile, protect the data from loss and make sure it is secure.