Annual Analytics Assessment


At this time last year I celebrated the Chinese New Year by publishing my predictions about the world of analytics and Big Data. Its always fun to look back and hold yourself accountable, so let’s go!

LittleBlogAuthorGraphic  David Hodgson, January 14, 2016

In 2015 the Chinese year of the Ram started on February 19. For 2016, the year of the Monkey, the new year has moved forward to start February 8. It is determined by a zodiac cycle and so bounces around a bit compared to our Julian calendar, which approximates the solar year.

So how did I do on my five predictions?

To keep me honest you can read my blog entry from last year here.

#1 I correctly predicted that people would still be using the term “Big Data”. No-one likes this buzzword, but those who predicted its demise were too optimistic! It’s here to stay because culturally we like reductionist simplicities.

#2 I may have been a bit optimistic in saying that every major enterprise would have a real, funded, Big Data strategy by the end of 2015. I bet it’s close though. Analysts reported continued adoption driven by LoB departments, with Marketing front and central.

#3 I said that “data agility” will be the aspirational driver of big data strategies. We did indeed continue to see people moving data from traditional proprietary data warehouses into more portable forms. However, the erosion slowed as the incumbents found ways to embrace Hadoop and the new became more additive to the old than replacing it. T’was ever thus with IT.

#4 2016 will get us to 10 years of Hadoop being used as the primary tool of Big Data analytics. I got this right, but several forecasters were saying that we would have moved on from Hadoop to other technologies. The elephant still stands largest and squarely in the middle of the room. True we saw Cloudera and many enterprise adopters embrace Apache Spark over MapReduce, but still in the context of the Hadoop stack and ecosystem.

#5 The IoT did not yet become the shaping force that I thought it might. Of course it waits in the wings, growing slowly (who got Nest thermostats last year?), just needing enterprises to really embrace the new technologies involved.

So what trends did emerge to continue as shapers for 2016? I’ll just call out two areas this year.

#1 Analytics and the IoT

It is still early days for enterprises finding business advantage with analytics on IoT generated data, but I think we will see significant progress in the year ahead. Most speculators would bet on the manufacturing industry for results here. But given the proliferation of wearable technology in the last 18 months, my bet is that we will see serious headway in healthcare related analytics.  To come true this prediction is probably dependent on the next area.

#2 Security and Compliance

Security and compliance issues remain a barrier to larger scale production implementations, particularly where PII information may be involved. I predict that we will start to see better defined process and procedures around the handling and merging of structured and unstructured data.

Otto Berkes of CA Technologies has suggested that Bitcoin’s blockchain protocol could be re-used as a secure and validated way for IoT devices to communicate and exchange data. Otto is a lot smarter than me, so I will just say that in 2016 we will see stronger solutions emerge to make the IoT secure and less vulnerable to corruption by hackers.

The Monkey Wrench

Ok, so that’s three predictions really not two – I’ll review them next January. What will the Year of the Monkey really bring? With the economy picking up steam, analytics will be central to IT investment and hiring. We will see a lot of companies copying each other (as monkeys do) but let’s look out for the “alpha ape” trend setters; those who will take us into new territory. Who are you watching? Let me know by commenting at the top of this blog.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s