As I walk around various offices or even in social gatherings, I find many conversations about Artificial Intelligence (AI), Robotic Process Automation (RPA), big data. And logically, then, the discussion quite often rolls into how our life will change due to the availability of data, how each of our actions is turning into data, how future consumer behaviour thus can be predicted etc. Thus people quite often discuss predictive analysis (PA), and we hear stories about its use in elections to predict voters' behaviour, customer behaviour, payment risks, etc.

It reminds me of the famous Octopus, Paul, who almost correctly predicted the various match results during the 2014 World Cup Soccer. Was that PA?

When I probe a bit and ask – “so, how do you use PA in your day-to-day business?” …. most of the time, the answer is – “ well….we are still evaluating, we just did a Pilot which was quite successful but has yet to roll out across the organisation.”

Concerning - is not it? When we discuss so much that PA is the next big opportunity, it can change the way we do business, it can save so much time/effort and thus cost ….I wonder why most organisations are not quickly moving beyond a mere Proof of concept?

What is Predictive Analysis?

Before going further, let me quickly define how people understand predictive analysis in simple terms. It is very close to using statistical modules and mathematical logic based on historical data and taking into account the current trends/ market condition/ competitive activity/ economic condition to predict future outcomes.

Let us take one simple example to illustrate the point and let us look at sales forecasting. For sales forecasting, what we actually do is we look at past trends, market share, economic forecast, competitive activities, pricing trends, raw material pricing, and new product launches and arrive at a forecast. So, logically what we do is we use available information and make highly informed guesses by observing the past trend, looking at the current conditions to arrive at a future forecast.

And hence, if we have similar information, why can't we capture relevant parameters that we use for regular sales forecasting and build in logic and thus use PA to arrive at the same forecast? Why not use available technology and replace the sales forecasting process where thousands of man-hours are spent when that can be done by a few data scientists with the help of data?

In fact, to take the argument further, why can’t we abolish the traditional budgeting process and replace that with PA?

Why do many organisations still use traditional processes?

But in reality, even today, after so much talk about PA, Machine Learning, and AI, most organisations still use traditional budgeting/forecasting processes involving almost the whole organisation for months, spending millions of dollars and yet embracing technology.

Surprisingly but true.

In my view, there are a few distinct reasons for such behaviour…

1. Lack of understanding by senior leaders

It is hard to acknowledge, but many senior executives, though aware of the concepts of big data, Machine Learning or AI, lack in-depth understanding, which leads to their lack of appreciation of the possibilities. Till we can impart the knowledge and get them on board, opportunities like PA will be on paper and won’t be realised. 

2. Lack of data- & process-driven culture 

It is easier said than done. A culture needs to be top-down and should be driven across the company. It takes years to build such a data-driven and process culture. Strong sponsorship from the top is essential for such a change in culture, which can only happen when the senior leaders appreciate the missed opportunity. 

Thus, many of us still do not believe that concepts like Machine Learning, Artificial Intelligence cannot replace human effort. We may not talk about that in open, as we do not want to give others the impression that we are far behind but in our minds.

3. Lack of trust

Most of the executives are not fully satisfied with the analytics they receive for running the business. And hence many of the decisions are not data-driven but are based on their experience and knowledge. And thus, they are not quite sure of the analytical capability of an organisation and hence consider adopting PA is too farfetched. We need to educate the organisation that things have changed over time and everyday technology is progressing faster than yesterday. What we should realise is that trusting too much in our past experience is not helping us to give the new opportunity a fair try. 

4. Silo approach  

Many organisations are still run in departmental silos and yet to have a culture that’s organisation-wide. Thus cross-functional holistic insight is very often missing or, if done, may be insufficient for an effective business decision. Thus analytics are silo-based and not integrated. PA needs an integrated approach, not a silo culture. It will take time to build trust in moving from human-driven analytics to system-driven analytics. We must recognise that in order to take a step towards embracing concepts like PA, we as leaders need to learn, unlearn and relearn. Or else we will remain in silos doing more proof of concept and not the real PA. 

Are we not missing some big opportunities?