TagPerformance Improvement

Analytics and AI: Humans and Machines are Good at Different Aspects of Prediction

Mostly driven by growth in the IoT, and the widespread use of internet, social media and mobile devices to perform search, send text, email and capture images and videos, the amount of data that we are producing on a daily basis is startling.

Consequently, companies are turning to data analytics and AI technologies to help them make sense of all the data at their disposal, predict the future, and make informed decisions that drive enterprise performance.

Although adoption of analytics and AI systems is increasingly extending in more mission-critical business processes, the implications of these emerging technologies on busines strategy, management, talent and decisions is still poorly understood.

For example, the single most common question in the AI debate is: “Will adoption of AI by businesses lead to massive human job cuts?”

Borrowing lessons from historical technological advances, yes, certain jobs will be lost, and new ones also created. However, machines are not taking over the world, nor are they eliminating the need for humans in the workplace.

Jobs will still be there albeit different from the traditional roles many are accustomed to. The majority of these new roles will require a new range of education, training, and experience.

For instance, nonroutine cognitive tasks demanding high levels of flexibility, creativity, critical thinking, problem-solving, leadership, and emotional intelligence do not yet lend themselves to wholesale automation.

Analytics and AI rely on data to make predictions

As more and better data is continually fed to the machine learning algorithms, the more they learn, and improve at making predictions.

Given these applications search for patterns in data, any inaccuracies or biases in the training data will be reflected in subsequent analyses.

But how much data do you need? The variety, quality and quantity of input, training and feedback data required depends on how accurate the prediction or business outcome must be to be useful.

Training data is used to train the predictive algorithms to predict the target variable, while the feedback data is used to assess and improve the algorithm’s prediction performance.

Undoubtedly, advanced analytics and AI systems are only as good as the data they are trained on. The data used to train these learning algorithms must be free of any noise or hidden biases.

You therefore need to understand how predictive technologies learn from data to perform sophisticated tasks such as customer lifetime value modeling and profitability forecasting.

This helps guide important decisions around the scale, scope and frequency of data acquisition. It’s about striking a balance between the benefits of more data and the cost of acquiring it.

Humans and machines both have shortcomings

In the context of prediction, humans and machines both have recognizable strengths and weaknesses.

Unless we identify and differentiate which tasks humans and machines are best suited for, all analytics and AI investments will come to naught.

For instance, faced with complex information with intricate interactions between different indicators, humans perform worse than machines. Heuristics and biases often get in the way of making accurate predictions.

Instead of accounting for statistical properties and data-driven predictions, more emphasis is often placed on salient information unavailable to prediction systems.

And, most of the times, the information is deceiving, hence the poor performance.

Although machines are better than humans at analyzing huge data sets with complex interactions amidst disparate variables, it’s very crucial to be cognizant of situations where machines are substandard at predicting the future.

The key to unlocking valuable insights from predictive analytics investments involves first and foremost understanding the definite business question that the data needs to answer.

This dictates your analysis plan and the data collection approaches that you will choose. Get the business question wrong, conclusively expect the insights and recommendations from the analysis to also be wrong.

Recall, with plentiful data, machine predictions can work well.

But, in situations where there is limited data to inform future decision making, machine predictions are relatively poor.

To quote Donald Rumsfeld, former US Secretary of Defense:

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.

Donald Rumsfeld, former US Secretary of Defense

Thus, for known knowns, abundant data is readily available. Accordingly, humans trust machines to do a better than them. Even so, the level of trust changes the moment we start talking about known unknowns and unknown unknowns.

With these situations, machine predictions are relatively poor because we do not have a lot of data to ingest into the prediction model.

Think of infrequent events (known unknowns) that occur once in a while, or something that has never happened before (unknown unknowns).

At least for infrequent events or happenings, humans are occasionally better at predicting with little data.

Generally so because we are good at comparison and applying prudent judgement, examining new situations and identifying other settings that are comparable to be useful in a new setting.

We are naturally wired to remember key pieces of information from the little data available or the limited associations we have had in the past.

Rather than be precise, our prediction comes with a confidence range highlighting its lack of accuracy.

Faced with unknown unknowns, both humans and machines are relatively bad at predicting their arrival.

The simple truth is that we cannot predict truly new events from past data. Look no further than the current Brexit conundrum.

Nobody precisely knew the unintended consequences of the UK leaving the EU. Leavers and Remainers both speculated as to what the benefits and disadvantages of leaving the EU maybe.

Of course, nobody knows what will happen in the future but that doesn’t mean we can’t be prepared, even for the unknown unknowns.

In their book Prediction Machines: The Simple Economics of Artificial Intelligence, Ajay Agrawal, Joshua Gans, and Avi Goldfarb present an additional category of scenarios under which machines also fail to predict precisely – Unknown Knowns.

Per the trio:

Unknown knowns is when an association that appears to be strong in the past is the result of some unknown or unobserved factor that changes over time and makes predictions we thought we could make unreliable.

PREDICTION MACHINES: THE SIMPLE ECONOMICS OF ARTIFICIAL INTELLIGENCE

With unknown knowns, predictive tools appear to provide a very accurate answer, but that answer can be very incorrect, especially if the algorithms have little grasp of the decision process that created the data.

To support their point of view, the authors make reference to pricing and revenue analysis in the hotel industry, although the same viewpoint is applicable elsewhere.

In many industries, higher prices are analogous to higher sales, and vice versa.

For example, in the airline industry, airfares are low outside the peak season, and high during peak seasons (summer and festive) when travel demand is highest.

Presented with this data, and without an understanding that price movements are often a function of demand and supply factors, a simple prediction model might advocate raising different route airfares to sell more empty seats and increase revenues. Evidence of causal inference problems.

But, a human being with a solid understanding of economics concepts will immediately call attention to the fact that increasing airfares is unlikely to increase flight ticket sales.

To the machine, this is an unknown known. But to a human with knowledge of pricing and profitability analysis, this is a known unknown or maybe even a known known provided the human is able to properly model the pricing decision.

Thus, to address such shortcomings, humans should work with machines to identify the right data and appropriate data analysis models that take into consideration seasonality and other demand and supply factors to better predict revenues at different prices.

As data analytics and AI systems become more advanced and spread across industries, and up and down the value chain, companies that will progress further are those that are continually thinking of creative ways for machines to integrate and amplify human capabilities.

In contrast, those companies that are using technology simply to cut costs and displace humans will eventually stop making progress, and cease to exist.

How to Transform Your Business in Times of Continuous Change

In times of continuous change, there are both winners and losers. Some company’s grow to become high performing, innovative and competitive enterprises while others develop into fighters, fighting for survival on a daily basis.

Today’s business environment is constantly evolving, with many factors both internal and external to the organization affecting the achievement of its stated objectives including the level of its competitiveness compared to competitors.

Some of the contributing factors include prolonged geopolitical and economic uncertainty, unresolved trade issues, rapid advancement in technological innovation, increased competition from new market participants, and fickle customers with constantly evolving needs.

As a result, the business has to be adaptive if it is to grow and succeed in such a disruptive environment.

When everything is going well, it’s easy to focus more attention on the good stories and less on what could go wrong. The blue overshadows the red, and this is a major problem in some companies.

These companies allow their past success stories such as successful product launches, increased market share, core technologies, and other organizational capabilities to blind their ability to view the future with a different pair of eyes.

Culturally, they are locked into the old way of working, bound by legacy systems and processes. Little time is spent on reviewing and evaluating the existing business model to establish whether it is still viable or not in these disruptive times.

On the contrary, transformational companies are not satisfied with the status quo. They are appreciative of the fact that past success is not a guarantee of future success.

Just because you are doing well today doesn’t not mean you’re going to enjoy everlasting success.

Business history pages are littered with doom and gloom stories about companies that have collapsed due to lack of innovation and unwillingness to evolve with the market.

Examples of such companies include the technology company Xerox, the retailer JC Penney, the social networking company MySpace, the department store Sears, the high tech company Polaroid, the bookstore Borders, and Circuit City the consumer electronics company.

What do all these companies have in common? At some point in time, they were all mighty industry titans, too big to fail and led by great, smart people.

However, in the midst of their successes they failed to adapt to changing customer needs, new technologies, competition and business models.

Even though these companies had built their businesses from the ground to the top of their respective industries, their death knell was the self belief that no other company was capable of doing better than what they were already doing and unseat them at the top.

Unfortunately, because of this fallacious way of thinking and ignorance they all paid a hefty price.

To avoid having your company join this list of colossal business failures:

  • Don’t get comfortable doing the right thing for too long. Continuously look for opportunities ahead and remember that today’s success can obscure tomorrow’s possible failures.
  • Regularly ask yourselves if what you’re doing and how you’re doing it is enough. It’s about making productive use of the resources available to you to improve your company’s performance and competitiveness.
  • Don’t dwell too much on the past. It’s important to know what has happened, but more importantly you need to understand why it has happened and how your company would perform in the future.
  • Commit sufficient time to analyzing new technologies, industry trends and competitors. Reviewing financials provides a rearview mirror of business performance, and you need forward looking indicators to understand your customers, competitors and the competitive status of your business (in terms of products, core technologies, market share, talent, culture)
  • Stay open minded. As highlighted above, when a company has been successful for too long, very little time is spent on thinking through alternative downside scenarios. It’s so easy to focus on the good news, spurn bad news and avoid discussing negatives. Questions such as “Why haven’t we done it before, What if this doesn’t work? What would we do then? What might make this not work?” are reluctantly answered. As a result, what begins as minor issues eventually develop into major issues. Don’t be a victim of own success to such an extent that you become ignorant of change.

Transforming a business into a high performing, innovative and competitive enterprise is a journey characterized by ups and downs. Consider every challenge, every problem and every piece of bad news as an opportunity to learn and improve.

Finance as the Custodian of Enterprise Performance Management

The days of having CFOs responsible for only preparing the statutory financial reports of the business and play the role of the bookkeeper are long gone. Today, finance leaders are expected to play the role of the strategic advisor to senior management and the board,and help drive operational and strategic performance across the enterprise.

That is, become custodians of enterprise performance management (EPM) by taking the lead on performance management and delivering informed business insights. EPM takes a completely different approach towards measuring, monitoring and improving enterprise performance.

Instead of assessing business performance in a siloed approach, EPM ensures the business evaluates and monitors its performance holistically. Although a majority of business decisions have either positive or negative financial implications on the health of the organization, evaluating, monitoring and improving business performance extends beyond a sole focus on financial metrics.

In order to perform better in their new business performance custodian role, it is imperative that finance leaders develop knowledge and a deeper understanding what constitutes and doesn’t constitute EPM. 

A simple google search of the words “Performance Management” brings up results that associate performance management to the process of conducting employee performance appraisals and supervising employees and departments to ensure that goals and objectives are met efficiently.

As a result, many people think that performance management is a human resources process that is only people-focused and has nothing to do with finance – Human Performance Management. To a lesser extent, their thinking is correct in the sense that people are part of the process. However, to a larger extent, they are wrong.

Performance management is not entirely focused on carrying out the outdated employee annual performance appraisals or reviews based on isolated individual key performance indicators (KPIs).

Rather, performance management is the integration of multiple managerial methods and or frameworks such as strategy maps, balanced scorecards, activity based costing/management, driver-based rolling forecasts, process improvement, risk management and advanced analytics to support strategic decisions and drive performance.

This is not achievable individually, hence the key word “enterprise.”

As custodians of business performance, finance should play a leading role in implementing a robust EPM framework across the organization. The framework should enable the organization to communicate and translate its strategy into financial and non-financial metrics and targets, monitor its performance, create accountability, and focus its efforts and resources on the key business drivers.

Additionally, the EPM framework should ensure there is alignment between individual KPIs and reward and recognition systems, and corporate objectives, as opposed to mere job descriptions, in order to encourage behaviours which positively contribute to the overall strategy of the business.

When implementing the EPM framework, it is important to ask the following questions:

  1. What do we want to achieve and excel at? This helps define your goals and key value drivers.
  2. How do we know if we are actually excelling at this? This involves defining financial and non-financial KPIs, which are measures that help you understand whether you are achieving your goals
  3. What is the desired level of performance that we would like to see? Defining KPIs is not enough, you also need to define your targets that represent the level of success or failure at achieving your KPIs.
  4. What initiatives should we pursue or engage in to meet our performance targets? These are actions or projects or strategies or processes needed to achieve a target, or improve performance level.
  5. What resources and or investments are needed to achieve our target? Utilizing driver-based budgets and dynamic rolling forecasts will ensure resources are allocated strategically and efficiently.

Asking and answering the above questions helps design and implement management processes and systems that align business strategy to drive accountabilities, decision support and performance improvement.

For example, when KPIs are aligned with business strategy, decision makers will focus on the critical success factors of the organization. On the contrary, if there is lack of KPI alignment, senior management end up getting overwhelmed with an increasing number of performance reports that lack key insights necessary to move the business forward.

As custodians of enterprise performance, it’s important for finance to have the ability and capabilities to consolidate, analyze and interpret business performance in real-time. Rather than simply report on the past, finance teams must be able to explain the reasons behind the numbers (the whys and what-ifs).

Leveraging consolidation solutions enable teams to quickly model and assess the impact of alternative business scenarios and formulate appropriate solutions.  

Given the complexity of today’s business environment, finance leaders need to rise to the occasion and perform the strategic advisory role now expected of them by senior management and the board.

 

Risk-Based Decision Making

Risk is an inherent element of the business. Given that every business activity or decision has a risk consequence, a business should not expect to operate and progress by making risk an afterthought.

Technological advances, evolving customer expectations, volatile markets, global political instabilities, shifting demographics and natural disasters are impacting business models and forcing organizations in every sector to rethink the way they operate.

Left unaddressed, these forces of change have a huge potential of derailing the strategic plan of the business and accelerate the organization towards failure. In order to survive in this VUCA world, the business should make a paradigm shift from reactive to proactive mode. This helps prepare and plan for the future rather than respond to it  after it has arrived.

The challenge today in many companies is that risk-based decision making is an afterthought. Only after going through a turmoil do people start asking where is risk management and why did the risk experts fail to anticipate the events in advance. In financial services companies where there are dedicated risk management teams, it is easier to point the finger of blame.

However, not all companies have a dedicated risk management function.

Risk is everyone’s responsibility

What exactly is the meaning of this statement?

Is making risk everyone’s responsibility part and parcel of risk culture?

What is the best approach of making everyone embrace risk-based decision making?

If some employees within the organization have never received training or guidance towards risk decision making, are they still held responsible?

By making risk everyone’s responsibility are we increasing confusion and blurring the lines between accountability and responsibility?

In the event that a business fails as a result of activities or decisions that could have been avoided, who is held accountable and responsible?

The CFO as the champion of risk-based decision making

In companies lacking a dedicated risk management function with a CRO at the helm, overseeing of risk management is normally under the purview of the CFO. The CFO is better positioned to champion meaningful risk conversations across the organization and drive better decision making processes.

Many people connect risk management with the negatives, hence the desire to avoid risk at all costs. Risk-based decision making is not about managing or avoiding risk. Effective risk management involves looking at the upside of risk and making informed risk decisions that help the organization achieve its stated objectives.

Driving a risk-based decision making culture therefore goes beyond lip service. It is not about merely saying everyone is responsible for risk. It is about raising risk management awareness and developing risk competencies across all staff levels through training, discussion and sharing of risk information.

Risk doesn’t start to happen once the strategy has been set. With the world always changing, risk is a constant present both before and after strategy setting. That is why it is important to understand the risks of your strategy including risks to the execution of the strategy.

Once every employee has a better understanding of risk, how it applies to their individual area of responsibility and align with the overall strategy of the business, risk-based decision making ultimately becomes part of the culture.

Given that finance has a unique end-to-end view of an organization the CFO plays a critical role in helping business partners understand the strategic plan of the business, identify, quantify, and mitigate any risk that affects or is inherent in the company’s business strategy, strategic objectives, and strategy execution.

The CFO is capable of leading the risk conversation and ensuring that the focus is more on taking advantage of opportunities and achieving strategic objectives and less on the downside, in turn ensuring that more value is created than is preserved.

Although the CFO has the bird’s eye view of the organization and an understanding of where the risks are coming from including the mitigation strategies, s/he cannot do it alone. Risk management requires an holistic approach across the company, and different risks are the problem of the function that they most impact.

It is therefore imperative that the CFO co-ordinates efforts and works alongside other C-suite executives to identify and assess emerging risks and best understand how to mitigate them.

Having the ability to partner with the business and speak their language is key to leading and engaging C-suite executives in meaningful risk conversations that help mitigate risks to the execution of the strategy.

Relationship between risk and performance

Risk conversations have to keep pace with the complexity of the business. Elevate the conversation to include a discussion around sources of potential disruption, their impact on the day-to-day execution of your strategy and the creation of value, and what your organization should do to increase the possibility of success.

Risk and performance are two sides of the same coin. A business cannot manage risk in isolation of performance. At the same time, the business cannot manage performance without consideration of risk. It is therefore imperative to integrate risk into your strategy and performance management decision making processes.

One way of embedding risk in the strategic planning process involves connecting your risk reporting and your strategy execution. Unfortunately, companies spend a significant amount of time compiling risk registers that do not inform strategic decision making. I have come across risk registers that list hundreds of risk events with very few of the events connected to the achievement of strategic objectives.

Risk assessment exercises should not be performed in isolation to strategic decision making. It is therefore important for the team responsible for performing risk assessments and compiling risk reports to understand what the strategy of the organization is, including what the strategy colleagues are doing on a day-to-day basis.

Not only will this help understand the business environment but also key assumptions. Instead of churning out the same report with the same list of risks on a monthly or quarterly basis, your report should be a reflection of key risk management changes overtime and help influence business decisions.

Conclusion

Risk-based decision making should be integrated into the overall management system of the organization. Given the constantly changing business environment, the business should always be ready for the unthinkable.

Business leaders should therefore focus on continuously improving the organization’s risk management framework and employee risk competencies to ensure both are capable of withstanding the test of times.

Current State of Enterprise Risk Oversight

A recent publication, Global Risk Oversight, by North Carolina ERM Initiative, in partnership with the Chartered Global Management Accountant ( CGMA ) provides insights on the current state of enterprise – wide risk oversight, including identified similarities and differences in different parts of the world.

Here are some key findings, with emphasis added:

  • Organizations all around the world perceive an increasingly complex risk environment.
  • Risk management practices appear to be relatively immature cross the globe. Around 30% or less of organizations indicate they have ‘complete’ enterprise risk management ( ERM ) processes in place. Only about 25% of the survey respondents describe their organization’s risk maturity as “mature” or “robust”.
  • Most organizations struggle to integrate their risk management processes with strategic panning. Despite the fact that most strategies maybe impacted by a number of risks, only about 50% of organizations around the world “mostly” or “extensively” consider risk exposures when evaluating new strategic initiatives. 
  • There is a lack of detailed risk oversight infrastructure in most organizations. Only a few organizations have formal risk management policy statements and frequently update risk reports.
  • Around 80% of organizations have not conducted any formal training risk management training for their executives.
  • There is increased pressure on management to strengthen risk oversight. Depending on the geographical location of the organization,  this pressure is coming from either the board of directors, the CEO or the audit committee.
  • Lack of sufficient resources to invest in ERM and the perception that there are more pressing competing priorities have been identified as the biggest barriers impeding the progress of maturing the organization’s risk management processes.

In light of these findings, the authors of the report recommend that:

  • Senior executives and boards of directors honestly and regularly assess their organization’s current approach to risk oversight in the today’s changing risk environment.
  • Management genuinely consider whether the process used to understand and evaluate risks associated with the organization’s strategies actually delivers any unique capabilities to manage and execute their strategies.
  • Organizations appoint a risk champion such as a Chief Risk Officer (CRO) or create a management-level risk committee in order to help strengthen the risk management function and ensure all risk management processes are appropriately designed and implemented.
  • Organizations spend time analyzing the vast amounts of data they have to generate insights about emerging risks that may impact their organizations’ strategic success.

Overall, the report is a good read and a great starting point for improving enterprise-wide risk oversight.

It helps senior executives ask important questions when evaluating their organizations’ overall approach to risk oversight. However:

  • Although the authors mention regular updating of the risk register. I would add risk management is not about list compilation,  otherwise organizations might find themselves building risk lists that lack any insight for effective decision making. It is about identifying and evaluating those key risks with the potential of derailing the organization’s strategic success and finding effective ways of mitigating any losses. Furthermore, intelligent risk decision-making does not look only at the downside of risks but also at the opportunities found in taking calculated risks.
  • There is no mention in the report about offering risk management training to middle-level and lower-level employees, only to senior executives.  The tone at the top and culture will determine if the organization succeeds at maturing risk management processes. Identifying and managing enterprise risks should be everybody’s responsibility within the organization. Thus, I believe there should be a common risk language throughout the organization.
  • Appointing a risk champion to strengthen risk oversight is critical. However, the individual appointed must have a deeper understanding of the business, its critical performance drivers and the ability to partner with the rest of the business. He or she must also be able to deliver the necessary risk training required.
  • Clear communication channels should be established to enable free flow of risk communication from top-down and bottom-up. People should not be scared to raise red flags or emerging risk issues to senior executives. Although the board of directors ultimately holds the risk oversight responsibilities to shareholders and other stakeholders of the business, if they receive inappropriate risk reporting from the bottom, the information they will feed to these interested parties will also be inadequate.
  • Risk management should be ingrained in the DNA of the business. Risk conversations should be about supporting strategic objectives achievement and enhancing business performance, as opposed to being a box-ticking exercise all the time.

Do the survey findings reflect the situation at your organization? If so, what are you doing to improve this situation?

I welcome your comments and views.

 

© 2019 ERPM Insights

Theme by Anders NorénUp ↑