Is your company looking to get started with IoT and big data or improve on how you’re handling it now? Here are six tips from the pros
Is your company looking to get started with the Internet of things and big data, or are you looking to improve on how you’re handling it now? Here are six tips from the pros that should help anyone:
Use the right people. Data scientists are in short supply and get very sizable salaries. But you don’t need to hire data scientists, says Andrew Brust, Senior Director of Technical Product Marketing and Evangelism at Datameer, a big data Analytics and Visualization company. Instead, look at your existing staff for people with data warehouse and IT experience, and are willing to learn, and train them.
Be smart about data capture. Carefully design exactly how you’ll capture IoT data. GE, for example, uses small data-collection appliances that determine what kinds of data to collect, what protocols to use for collection, and how the data should be stored. And keep all of your data, even if you don’t know how you’ll use it, recommends Mike Maciag, Chief Operating Officer at Altiscale, which offers a cloud-based Hadoop platform. As your company strategies change, you may well find a need for it.
Provide an abstract data layer. IoT comes in many different protocols and data standards that aren’t always compatible with one another. Sometimes the data is highly structured, and other times it isn’t. Your best bet is to provide an abstraction layer that can handle multiple data types, including new ones you haven’t yet encountered.
Choose the right platform. Your company may not want to spend its time and money building a large data analytics platform on its own. Consider using one of the many cloud-based ones currently available.
Start with a small pilot, then build out. Intel’s Sharma says many companies bite off more than they can chew when taking on IoT big data projects. Instead, he says, start small with a pilot. Once you’ve got all the problems ironed out, roll it out to the rest of your enterprise.
This story, “6 tips for working with IoT and big data” was originally published byITworld.
- Published in Internet of Things
Amazon dives into the Internet of things with a two-pronged strategy covering both data and devices
Quibble if you will about the definition or long-term viability of the Internet of things, but Amazon is charging full ahead to fashion itself into an catch-all IoT platform.
At the Re:Invent keynote today, Amazon unveiled the AWS IoT framework to not only gather data from devices, but provide device-specific management and introspection functions as well.
AWS IoT presents devices in two ways: the devices themselves, aka “things,” and virtualized representations, or “thing shadows.” The latter lets the user preemptively set the state of devices without requiring a network connection; once a disconnected thing reconnects, it attempts to sync with its shadow and apply any changes pushed (a function natively supported in the MQTT protocol). Devices can also be tracked through a registry.
Amazon surrounds these features with a few additions that, while not explicitly IoT-related, can fall under the heading. A new function for Amazon’s Kinesis Analyticsallows SQL queries to run against streaming data — for instance, as part of a time-series processing job. The service is set to include many prebuilt functions, such as moving averages or totals.
In terms of construction, the heart of AWS IoT isn’t drastically different from that of other Web service back ends. The fact that it’s Amazon makes the difference, what with so many customers already building on top of Amazon’s application, data-storage, and data-ingestation frameworks. Anyone already on Amazon’s cloud has one fewer reason to bother with other IoT integrators. Contrast that withSalesforce IoT Cloud, which limits its appeal to existing Salesforce customers, whereas nearly everyone is a potential AWS customer.
InfoWorld’s David Linthicum made a case for why IoT and public clouds like Amazon’s complement each other: a measure of built-in security, elasticity, and a geographically distributed architecture that works with the devices themselves. It was inevitable that Amazon would become a center of gravity, but now we’ll see if its device-management-plus-data-collection approach pulls people in.
- Published in Internet of Things
The complexity of the Internet of things will only get worse, and focusing on interoperability is not the solution.
In the next five years, our homes will have lots of devices connected to cloud services: thermostats, security systems, refrigerators, washers, dryers, coffeemakers, cars, TVs, set-top-boxes, doorbells, light switches … you name it.
All the Internet of things devices will communicate with different cloud systems for data storage, compute services, and software updates. Our homes may suffer from a level of cloud complexity to rival that of smaller enterprises.
In turn, enterprises will have evermore complexity. Thousands of industrial devices will need to be monitored and gigabytes of data transmitted on a daily basis. However, each device manufacturer will have its own cloud, so we’ll have enterprise devices connected to hundreds, perhaps thousands, of back-end cloud systems.
Before the situation gets out of hand, let’s think about common cloud services that can be used as a standard for Internet of things devices and providers.
Of course, I’m not the first person to think of this. Standards are already being developed for IoT. But the two biggest vendor organizations, the Open Interconnect Consortium and AllSeen Alliance, are fighting over whose standards will prevail, and you can count on more standard organizations jumping in to the fray. There’s going to be a lot of arguing before the market decides which should be the standard or if there should be one at all.
Even when they finally come together, their focus is on interoperability, not on the consolidation of back-end services. Although interoperability is a noble objective, we’ll still end up with the same number of cloud services to support these devices, so we’ll continue to deal with IoT complexity.
The better effort might be to look at how we can use common data and storage models, where most devices can use the same cloud services.
Obviously, there are issues with security, governance, and privacy, and most IoT device manufactures won’t want to open up their devices and data to a common cloud platform. But once retail and commercial markets see how much of a hindrance complex cloud usage can be, they may demand that the manufacturers play from the same sheet of music. Let’s hope that happens sooner than later.
- Published in Internet of Things