Annual Analytics Assessment


At this time last year I celebrated the Chinese New Year by publishing my predictions about the world of analytics and Big Data. Its always fun to look back and hold yourself accountable, so let’s go!

LittleBlogAuthorGraphic  David Hodgson, January 14, 2016

In 2015 the Chinese year of the Ram started on February 19. For 2016, the year of the Monkey, the new year has moved forward to start February 8. It is determined by a zodiac cycle and so bounces around a bit compared to our Julian calendar, which approximates the solar year.

So how did I do on my five predictions?

To keep me honest you can read my blog entry from last year here.

#1 I correctly predicted that people would still be using the term “Big Data”. No-one likes this buzzword, but those who predicted its demise were too optimistic! It’s here to stay because culturally we like reductionist simplicities.

#2 I may have been a bit optimistic in saying that every major enterprise would have a real, funded, Big Data strategy by the end of 2015. I bet it’s close though. Analysts reported continued adoption driven by LoB departments, with Marketing front and central.

#3 I said that “data agility” will be the aspirational driver of big data strategies. We did indeed continue to see people moving data from traditional proprietary data warehouses into more portable forms. However, the erosion slowed as the incumbents found ways to embrace Hadoop and the new became more additive to the old than replacing it. T’was ever thus with IT.

#4 2016 will get us to 10 years of Hadoop being used as the primary tool of Big Data analytics. I got this right, but several forecasters were saying that we would have moved on from Hadoop to other technologies. The elephant still stands largest and squarely in the middle of the room. True we saw Cloudera and many enterprise adopters embrace Apache Spark over MapReduce, but still in the context of the Hadoop stack and ecosystem.

#5 The IoT did not yet become the shaping force that I thought it might. Of course it waits in the wings, growing slowly (who got Nest thermostats last year?), just needing enterprises to really embrace the new technologies involved.

So what trends did emerge to continue as shapers for 2016? I’ll just call out two areas this year.

#1 Analytics and the IoT

It is still early days for enterprises finding business advantage with analytics on IoT generated data, but I think we will see significant progress in the year ahead. Most speculators would bet on the manufacturing industry for results here. But given the proliferation of wearable technology in the last 18 months, my bet is that we will see serious headway in healthcare related analytics.  To come true this prediction is probably dependent on the next area.

#2 Security and Compliance

Security and compliance issues remain a barrier to larger scale production implementations, particularly where PII information may be involved. I predict that we will start to see better defined process and procedures around the handling and merging of structured and unstructured data.

Otto Berkes of CA Technologies has suggested that Bitcoin’s blockchain protocol could be re-used as a secure and validated way for IoT devices to communicate and exchange data. Otto is a lot smarter than me, so I will just say that in 2016 we will see stronger solutions emerge to make the IoT secure and less vulnerable to corruption by hackers.

The Monkey Wrench

Ok, so that’s three predictions really not two – I’ll review them next January. What will the Year of the Monkey really bring? With the economy picking up steam, analytics will be central to IT investment and hiring. We will see a lot of companies copying each other (as monkeys do) but let’s look out for the “alpha ape” trend setters; those who will take us into new territory. Who are you watching? Let me know by commenting at the top of this blog.


Protecting sensitive mainframe data with CA Data Content Discovery



How a new software solution from CA announced recently at CA World can help your business conquer the mainframe data mountain and protect your organization’s most sensitive data.

 LittleBlogAuthorGraphic David Hodgson, December 8, 2015

Your mainframe has been collecting data for years — probably decades — and you rely on it to run your business and the apps that serve your customers. But over the years, its collected mountains of records and files, many of which contain sensitive data that require special controls stipulated by government regulations.

With so much information residing in your mainframe, it’s hard to locate regulated or sensitive data when you need it (and takes up far too much time). You may not be limiting internal access appropriately, and copies may end up somewhere else, without proper access control. At last count, 400 mainframes worldwide are connected directly to the Internet and accessible to anyone via a login screen.  So yes, while mainframe remains the most securable platform – it isn’t 100 percent immune to data breaches.

Recently at CA World in Las Vegas, CA Technologies announced two new mainframe solutions to help organizations become more agile. One of these new solutions, CA Unified Infrastructure Management for z Systems, supports our DevOps portfolio and helps customers accelerates problem resolution with a unified view across mainframe and distributed systems.

In this blog I’d like to focus on the new solution, CA Data Content Discovery that supports our Security portfolio. Bottom line, if you don’t know where your sensitive data is, you can’t protect it. CA Data Content Discovery scans your mainframe data to identify the location of data that matches regulations such as PCI, PII or HIPA, so you can make business decisions around securing, encrypting, archiving or deleting those records.

This isn’t just good business sense; it will help you address potential audit findings and risks.

If it ain’t broke, why fix the mainframe?

But why now? After all, the adage “if it ain’t broke, why fix it” often applies to mainframes. But these days, mainframes are not only tied to mission-critical applications, but those applications now face your customers through the web and mobile apps.

In the application economy, the mainframe plays a key role in how apps perform — and how happy your customers are.

Unknown unknowns: trust isn’t a strategy 

The stakes for mainframe security have changed.  In a recent blog post, Jeff Cherrington offers a colourful history lesson and metaphor comparing mainframe security to the evolution of fortifications of medieval castles.

The plain fact is today’s application economy puts different demands on the mainframe data – everyone wants in!  The Chief Digital Officer wants access to systems of record for his pet big data project or some backup project didn’t follow all the necessarily controls – the fact is mainframe data is moving off the platform when it shouldn’t be and if it needs to – let’s at least know about the location of that sensitive data and apply the right controls.  Companies that make security a priority understand that blind trust and “nothing will happen” isn’t the solution.

With the right tools and processes, you can be confident to leverage the mainframe as part of your digital transformation while safeguarding sensitive and regulated data. CA Data Content Discovery has three distinct advantages:

  • Find: You can locate regulated and sensitive data using data-pattern scanning, helping to gain insight into the magnitude of potential data exposure on z Systems.
  • Classify: Once you’ve found the data, you can prove to auditors that you’re compliant with regulations (controls are checked by data type and content).
  • Protect: Critical data never leaves the z/OS platform. Integration with CA ACF2, IBM RACF and CA Top Secret for z/OS means you can quickly visualize who has access to regulated or sensitive data.


For more details, check out the Data Content Discovery page.

It’s not enough these days for organizations to embrace software — they need to use it strategically. And that includes the mainframe.

In the era of digital transformation, organizations need to be more agile — and this is possible, even with legacy systems. With the right tools, people and processes, it’s possible to bring your mainframe along on the digital transformation journey.


Connectedness for the mainframe in the application economy: blessing or curse?


With the mainframe online and connected to apps that people use every day, the proximity of users on the mainframe has never been closer. So how should businesses think about security in the application economy?

LittleBlogAuthorGraphic  David Hodgson, August 20, 2015

I recently discovered an app that dials you into conference calls without the hassle of having to dig up phone numbers and participant codes. All you have to do is put your network ID and password into the app once and you’re off.

This begs the question: What is being done with this data and is it secure? What if at the other end is someone sitting in a café halfway around the world, calmly sipping a latte while receiving data from the app and using it to access your company’s network or maybe even the mainframe.

In a white paper I recently authored: “Mainframe Reframed for the Application Economy,” I explore the implications for the mainframe in today’s application economy and the need to reframe the mainframe for this new era’s demands. I point out that as this digital transformation is occurring, where we’re interacting with companies more through the palm of our hands than face to face, there’s pressure mounting on all areas of IT, including the mainframe platform.

The curse of connectedness

The application economy is driving the need for transparency, availability and reliability to meet customer demand in an always on, 24-7 world.

That very connectedness which makes apps like the one referenced above possible in the application economy is also shrinking the proximity of people to enterprise systems, including the mainframe, exposing businesses to unprecedented security risks.

With many high profile security breaches of late, the old “fortress” security mentality of keeping the bad guys out is no longer the prevailing approach to guard against the bigger risk – the people inside your organization, who often times do not realize they are creating vulnerabilities, or sometimes are themselves the threat.

Security is now more about detection and compliance than it is locking people out. More specifically, that is:

  • Knowing who is in your network
  • What they are doing
  • Whether they should be doing that.

A savvy auditor is certainly going to ask you how you would know if someone accessed data they were not supposed to.

In these days of rampant identity theft, people are not always who they appear to be. Once hackers have phished someone’s credentials the only way you detect a breach is when they start doing something anomalous. Social engineering is now much more sophisticated and your ‘trusted’ mainframe expert in the data center is as susceptible to social engineering as anyone else.

Remember also that as mainframe experts retire the very lack of skills and know-how left in their wake could inadvertently open new doors if proper transition plans aren’t put in place. That way, businesses can take action early when they know something is amiss.

Security breaches haven’t seen anything yet

While to-date, relatively few security hacks involve the mainframe, think of the impact of something such as the recent breach of U.S. computer systems for visas and passports and how much worse that would be if it was the mainframe of a major bank or airline, for example.

A recent MIT Technology Review article discusses how the mainframe, which has been around since the 1960s and houses some of our most precious data from banks, airlines and governments, has been put online, exposing it to a previously unknown world of cybercrime. The article goes on to quote security researcher Phil Young, who said he has found around 400 mainframes on the Internet prompting a login screen to anyone who connects.

Mainframe modernization or exposing the classic system of record data to new services means that the data is no longer isolated on the mainframe – the world is now “unknown, unknown.” We have lost sight and control of where the data is going the minute we try to harness mainframe data for other purposes than batch or transaction applications.

Think of the potential and, more importantly, the scale of damage. We’re not just talking about one database of customers of a retail chain – this would be something more far reaching than we could ever imagine.

How to catch a thief before they act

So how can organizations build the ability to detect problems before they arise into the mainframe platform? I came across this post by that shows how to use USS shell script to create a C program that can be piped over a network to run on z/OS. Fundamentally, it’s the same as how you’d do it for any other platform, just that you have to generate z object code and call different system devices.

While this is simply one vector into a system, it’s possible to create a product (or put it into an existing product such as CA Auditor for z/OS) that can scan for these vulnerabilities on a system, plug them and report on the number of times these attempts were blocked.

Last but not least, such news about technical exploits helps, but there is a huge cultural and communication barrier for mainframe security professionals in getting the broader organization and the rest of the security community to understand the risk. There is still a culture of denial or, “Wait my mainframe has never been compromised.” This is why we believe the mainframe reframed discussion is a timely and thoughtful conversation we need to have as a community.

If you want to find out more about how the mainframe is being reframed to handle the new security threats in the application economy, join us at CA World ’15. We will be giving a number of talks about security across platforms – including mainframe security in sessions such as “Castle Walls under Digital Siege – Risk Based Security for the z/OS.”

You will learn how identity can be applied to engage, serve and protect customers while they interact with your digital business – and make the connectedness we all enjoy in the application economy a blessing rather than a curse.

How security can be the key to your castle


In the application economy does security help or hinder your business?

LittleBlogAuthorGraphic  David Hodgson, September 23, 2014

Traditionally people view security like a castle moat. That is a great start but while a firewall with strong authentication is good it is not enough. Conceptually it does not go much further beyond: “Stop, who goes there?”

We are constantly seeing examples of people finding ways under, over or around our moats and in the current digital era this problem is getting worse. The fortress mentality just doesn’t work well in our highly connected world.

A wholly, more sophisticated approach is needed. One that keeps our resources secure, but doesn’t involve the ‘clunkiness’ of heavy security that ends up hindering your business.

With this post I complete the four-part series about navigating your journey in the application economy – exploring the topic by drawing allusions with my daily walk to work and back. The four principles I have suggested are:


In this post I’ll cover the topic of security.

Untethered capability

I quickly found out that wearing a tie while walking to work in Manhattan is a killer – I need much more air circulation around the neck area when moving at speed, particularly in the warmer summer months. So now I keep some ties at the office.

Keeping that tie in the office is a security feature for me in case I feel the need to put one on for an important meeting. Now I don’t have to slow down my walk and secure at work.

If your security slows down your employees or your customers it is holding your business back. In today’s connected world a well thought and thorough security strategy is critical.

We cannot deny the benefits of the cloud, but we need policies and tools that enable BYOD and the use of tools like Dropbox, not a police-state mentality that blocks useful activity. Security in the application economy must allow seamless application experiences while being sure that only authorized people are using the services.

The answer for the application economy is content-based data placement and content-based access to data, coupled with a much stronger concept of identity. We need a stronger sense of who is doing what and a stronger definition of how to do it.

However, it must not be restrictive to the user experience, because today’s user of both internal IT and consumer apps has choices, and if one service is slow or hard to use, they will use another that is easier and faster.

Who goes there and why?

At CA Technologies we have solutions for single sign-on and two-factor authentication. We also have the leading solution for credit card authorization that can dynamically detect anomalies and decide if a greater degree of authorization is required.

And new for the mainframe, we are developing content based access control that will allow you to set up policies to control access, changes and movement of data based on what it is rather than what container it is in. This will maximize usage and streamline management while ensuring good control and compliance with regulations.

The possibilities for this are awesome and much needed in today’s Hybrid Cloud infrastructures. Our new Cloud Storage for System z (CS4z) allows applications to seamlessly place tape data onto on-premise, private storage clouds or at public providers like Amazon and Google.

This is an incredible break-through in flexibility for the mainframe. But how do you manage that data placement? How do you stay in compliance with the latest regulations? Do you even know for sure what is on those 10-year-old tapes?

Awareness of data content allows both policy based data-placement and policy-based access control. So old reports that don’t contain personal information can be stored on Amazon Glacier but confidential information must stay in-house.

And if you connect identities to roles, you can limit access to data not only by a file name but also by the metadata that describes the sort of data that the file contains. This is an approach that will be safer, more adaptable and will expand securely to your business needs.

Arriving Home

The best part of my daily commute is of course the walk home. I hope you found value in this series and it helps you along your journey into the application economy.

We are right at the start of our journey and we are all learning from each other. I’d love to hear your stories. What security challenges have you bumped into along the way?

If you have solutions that help Development integrate tightly with Operations, I would love to hear about them too. Just leave a comment below and I’ll be sure to reply.

And I hope our paths will one day cross somewhere along our travels.

Image credit: Ashitaka San