Contact

Machine Predictions, Uncertainty, and Decision Making

Predictive technologies are transforming how we live and work, undoubtedly upending old ways of decision making. Every day, algorithms are significantly influencing our decisions concerning the products to buy, content to consume, movies to watch, investments to make, and whom we date and marry.

In business, they have increasingly become an integral part of supply chain and logistics, marketing, pricing, hiring and recruitment, and risk decision making, ultimately creating opportunities for new business models, product offerings, and strategies to compete.

Understandably, predicting the future is at the core of making decisions under uncertainty. But, in a continuously changing, uncertain, and complex world, to what extent should we trust algorithms and their predictions? When and how do we gain the necessary comfort and allow algorithms to move from a decision support role to critical autonomous decision-makers?

In 2020, Zillow, the online real-estate marketplace company, permanently shut down its iBuying business because its machine learning algorithm couldn’t accurately predict home prices and caused the company to suffer huge financial losses.

Wired magazine published an article that highlighted Why Zillow Couldn’t Make Algorithmic House Pricing Work.

The premise of Zillow’s iBuying business was being able to forecast the price of homes accurately three to six months in advance using Zestimate, the company’s pricing algorithm. The period reflected the amount of time Zillow expected to fix and sell homes it had bought.

Before being put to work, Zestimate was trained on millions of home valuations across the US and analyzed dozens of variables, including the home’s age, size, condition, location, and ZIP code. For a while, the algorithm appeared to get it right until COVID-19 came.

Due to government-mandated lockdowns and restrictions, housing market activity came to a screeching halt in early 2020. Then in late 2021, after governments eased restrictions, activity rebounded.

As part of its goal to buy 5000 homes a month by 2024 and increase market share, Zillow went on a buying spree, this time even looking beyond its initial business model focused on cookie-cutter homes.

The company also added lower quality and more complex homes into the mix, which some experts believe was beyond the bounds of its algorithm’s ability on the premise that Zillow ended up buying other homes at a higher price than the market average.

Not only did the COVID-19 pandemic cripple global economies, but it also led to a shortage of contractors in some parts of the markets the company had bought homes. As a result, this made it hard for Zillow to flip its homes as quickly as it hoped and left the company with an oversupply of overpriced homes.

Ultimately, the company sold homes at lower prices than the algorithm had predicted.

There are plenty of lessons to be learned from Zillow’s iBuying pricing algorithm:

  • Even if the right technology, data, people, and processes are in place, the future is not easily predictable. I stand to believe Zillow had all these capabilities in place, but the gyrating markets in 2021 plus other things made it difficult to accurately predict home prices.
  • A prediction is not a decision. Rather, effective decision-making involves applying judgment to a prediction and then taking the appropriate course of action. Zillow failed to assess the big picture and consider other factors that affect home valuations that even very advanced algorithms cannot comprehend.
  • While an algorithm predicts what is likely to happen, humans are still integral to deciding the required course of actions to take based on their understanding of the objectives to be achieved and the key determinants of business success.
  • It’s critical to develop a better, deeper more nuanced understanding of how well an algorithm works in both typical and outlier cases, the types of assumptions and inferences it is making, and the situations in which these assumptions might fail. Zestimate performed well for cookie-cutter homes but failed when complexity was thrown into the mix.
  • The same algorithm can have significantly different results or outcomes depending on the context in which it is applied. It’s therefore imperative to evaluate how the algorithm’s predictions or recommendations change with minor modifications to the input, training, or feedback data.
  • The same input data can have different results depending on the algorithm that is acting on the data. Would the same input data but a different algorithm to Zestimate have calculated similar valuations and predicted similar selling prices?
  • Similar to humans, no algorithms are the same. Accounting for these differences is critical to understanding what drives trust in algorithms and their predictions. In addition to expected results, simple design choices can also cause unintended and complex outcomes.

Data-driven predictions can succeed, and they can fail. So, in an uncertain world, algorithmic thinking alone is not sufficient to make good decisions.

Not everything is known or predictable. Good rules of thumb and intuition are also needed.

Thanks for sharing:

Related posts:

  1. Plan Continuation Bias and Decision Making
  2. Are You Using Scenario Planning To Improve Decision Making?
  3. Managing Volatility and Uncertainty
  4. Risk Decision Making in an Interconnected World
back to blogs

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to get notified of new posts by email

Recent Posts

Categories

The Art of Risk Management

This is the title of the article by BCG published a few years ago. The article discusses the principles that should govern the approach to risk management by companies of all shapes and sizes.

The authors make several points with which I agree. Here are some excerpts:

  • Risk management is essential in today’s volatile economy. In a continuously changing economic environment, companies cannot assume a stable risk landscape.
  • Stop thinking of risk management as primarily a regulatory issue. Embed risk management in the mindset of the broader organization.
  • Risk management is a value-creating activity that is an essential part of the strategic conversation inside the company. The goal of that discussion should not be to eliminate or minimize risk but to use it to create a competitive advantage.
  • Risk management starts at the top. The organization needs to demonstrate that it has made risk management a high priority and an integral part of the decision-making process by appointing a dedicated risk leader who reports back frequently to the CEO and the board to discuss the latest trends and any changes in the company’s risk scenarios.
  • Risk cannot be managed from an ivory tower. Risk Management should not exist in isolation from the rest of the organization, with an insufficiently granular understanding of the actual business-specific risks the company faces. To avoid this outcome, integrate risk management into the company’s entire routine management processes, including planning, capital allocation, controlling, and reporting.
    • Understand the scope of the risks the company faces.
    • Plan for how the company will manage those risks.
    • Act to mitigate the risks or take advantage of strategic opportunities.
  • Avoid relying on black boxes. Although sometimes appropriate, over-reliance on complex metrics or models can muddy the risk management process, turning it from a transparent management activity into a frustrating black box. The appropriate level of complexity is company-specific and depends on the industry, business model, availability of data, level of experience, and mandatory legal requirements.
  • Align risk management with a company’s overall business strategy. Companies need to identify all relevant risks – not just those that can be easily quantified. Some of the relevant risks for a company may be those that are qualitative and especially difficult to quantify.
  • Risk management is more than a policy; it is a culture. The objective of a company’s risk-management system should be not only to enforce new policies but also to create a risk-aware culture that addresses risks proactively, not reactively, and manages them to create new sources of competitive advantage.
  • Effective risk management depends on the free flow of information throughout the organization. Unless employees at all levels of the organization are actively involved in the risk management process, it will be difficult to maintain the unrestricted flow of information. This can result in the most important data getting buried in one part of the organization unavailable to other parts of the business.
  • Risk management deals with uncertain futures. As a result, the goal should not be to develop precise metrics or future outcomes but to strive for a general understanding of the probabilities and potential impact of various trends or scenarios on business performance and enable decision-makers to confront the uncertain nature of risk and act accordingly.
  • Risk management is never about finding “the answer.” Rather, it is about continually refining the organization’s assumptions about the future and its understanding of the implications of those assumptions for the company’s business. Assumptions about risk often change quickly, so the relevant parameters, probabilities, impacts, and correlations should be revisited frequently.
  • It is possible to prepare for unknown risks by building an organization that so excels at crisis management that it is resilient even in situations in which it is blindsided by unprecedented challenges. For example, through developing the ability to detect, capture, and exploit information patterns as well as to think outside existing frameworks and risk landscapes.
  • Avoid the downside, but don’t forget the upside. Companies should use risk management also to identify new opportunities and to exploit them systematically. For example, scenario planning should be used to define not only worst-case scenarios but also best-case scenarios. Think in advance about how a company can make the best use of the latest market developments and trends and ultimately make the right decisions.

I enjoyed reading the article and highly recommend it.

Thanks for sharing:

Related posts:

  1. Innovation Risk Management: Evaluating Your Freedom To Operate
  2. Integrating Risk Management and Strategic Planning
  3. Integrating Risk and Performance Management
  4. Rethinking and Elevating the Status of Risk Management
READ MORE

Leading in Uncertain Times

One of the biggest challenges facing business leaders today is making the right decisions that will ensure their organizations succeed, survive, and remain competitive in an increasingly uncertain and complex environment.

A recent post, The best way to lead in uncertain times may be to throw out the playbook, by Strategy+Business has several good points.

The article is about the COVID-19 pandemic, how global companies navigated through the crisis, and how best to prepare for future disruptions. Here are some key points and my comments.

  • Rather than follow a rigid blueprint, executives must help organizations focus on sensing and responding to unpredictable market conditions.
    • Comment: Senior leaders play a vital role in providing clarity about the organization’s strategic direction, creating alignment on key priorities to ensure the achievement of enterprise objectives, and ensuring the business model is continuously evolving to create and capture value in the face of uncertainty. They must not rest on their laurels and stick to the beliefs and paradigms that got them to where they are today and hope they will carry them through tomorrow. Regulatory changes, new products, competition, markets, technologies, and shifts in customer behavior are upending many outdated assumptions about business success. Thus, the businesses you have today are different from the ones you will need in the future hence the importance of continuously sensing changes in the global economy. Employees and teams often feed off the energy of their leaders and tend to focus their attention where the leader focuses attention. If the leader is comfortable with current business practices and rarely embraces the future or challenges the status quo, then the team is highly likely to follow suit.
  • When it became clear that supply chains and other operations would fracture, organizations began scenario planning to shift production sources, relocate employees, and secure key supplies.
    • Comment: Instead of using scenario planning to anticipate the future and prepare for different outcomes, it seems most of the surveyed organizations used scenario planning as a reactionary tool. Don’t wait for a crisis or a shift in the market to start thinking about the future. The world is always changing. As I wrote in The Resilient Organization, acknowledge that the future is a range of possible outcomes, learn and develop capabilities to map out multiple future scenarios, develop an optimal strategy for each of those scenarios, then continually test the effectiveness of these strategies. This does not necessarily mean that every change in the market will impact your business. Identify early warnings of what might be important and pay closer attention to those signals. In other words, learn to separate the signals from the noise.
  • The pandemic forced the organization’s senior management team to re-examine how all decisions were made.
    • Comment: Bureaucracy has for a very long time stood in the way of innovation and agility. To remain innovative and adapt quickly in a fast-changing world, the organization must have nimble leadership and an empowered workforce where employees at all levels can dream up new ideas and bring them to life. Identifying and acting on emerging threats and potential opportunities is not the job of the leader alone but every team member. To quote Rita McGrath, in her book Seeing Around Corners, she writes, “Being able to detect weak signals that things are changing requires more eyes and ears throughout the organization. The critical information that informs decision-making is often locked in individual brains.” In addition to the internal environment, the leader must also connect with the external environment (customers, competitors, regulators, and other stakeholders), looking for what is changing and how.
  • It’s worthwhile for leaders of any team to absorb the lessons of sense-respond-adapt, even if there is no emergency at hand.
  • Sensing: Treat the far-flung parts of your enterprise as listening stations. The question leaders must ask is, “What are we learning from our interactions beyond the usual information about costs and sales?” Train your people to listen for potentially significant anomalies and ensure that important information is not trapped in organizational silos.
    • Comment: Cost and sales data are lagging indicators that reveal the consequences or outcomes of past activities and decisions. Although this information can help leaders spot trends by looking at patterns over time, it doesn’t help understand the future and inform what needs to be done for the numbers to tell a different story. In addition to lagging indicators, pay attention to current and leading indicators and understand the relationship between these indicators and outcomes.
  • Responding: Improve communication across intra- and inter-organizational boundaries. Leaders should view business continuity as an essential function that acts as connective tissue for the enterprise.
    • Comment: In addition to creating mechanisms that allow the free flow of information both inside and outside the organization, decision-makers should also be comfortable receiving information that challenges their personal view of the world, even if it’s not what they want to hear. Create a culture of psychological safety where people are not afraid to share bad news for fear of getting punished, but rather are acknowledged and rewarded for speaking up. Leveraging the diversity of thought enables leaders to anticipate the future as an organization, decide what to do about it collectively, and then mobilize the organization to do what’s necessary.
  • Adapting: Challenge assumptions, and question orthodoxies. There’s always the temptation to mitigate threats simply by applying existing practices harder and faster. One way to get at those deeper issues and encourage double-loop learning is to ask, “What needs to be true for this to be the right approach?”
    • Comment: In an increasingly uncertain environment, it’s difficult to survive and thrive with an old business model or outdated technologies. Many businesses fail because they continue doing the same thing for too long, and they don’t respond quickly enough and effectively when conditions change. As a leader, stay curious and connected to the external environment, look for market shifts, understand what needs to be regularly refreshed and reimagined, adopt new technologies and capabilities, and adapt in ordinary times but also during times of transition. Unfortunately for many leaders, it’s just more convenient for them to continually downplay the fact that conditions are changing than take the appropriate course of action that drives business success.

How are you preparing your organization for potential future disruptions?

Thanks for sharing:

Related posts:

  1. Reimagining Forecasting in Uncertain Times
  2. Boards Must Play A Leading Role in Effective Risk Oversight
  3. Transforming Your Business in Times of Continuous Change
  4. Building a Culture that is Aligned
READ MORE

The Collaborative Organization

These days the term collaboration has become synonymous with organizational culture, creativity, innovation, increased productivity, and success.

Let’s look at the COVID-19 pandemic as an example. At the peak of the crisis, several companies instructed their workers to adopt remote working as a health and safety precautionary measure.

Two years into the pandemic, they are now asking their employees back to the office full time or are planning to adopt a hybrid model.

The need to preserve our collaborative culture and accelerate innovation are two of the top benefits being cited by organizational and team leaders for bringing workers back.

Collaboration is indeed essential for the achievement of team goals, functional objectives, and the overall success of the organization.

Today’s breakthrough innovations are emerging from many interacting teams and collaborative relationships.

When teams, functions, and organizations collaborate, the whole is greater than the sum of its parts; group genius emerges, and creativity unfolds.

But, what makes a successful collaboration? What are the key enabling conditions?

  • It extends beyond the boundaries of the organization. Business success is a function of internal and external relationships. Instead of viewing your business in vacuo, understand that you are part of an ecosystem. External to your organization, who do you need to partner with to enhance your value creation processes, achieve/exceed your objectives, or successfully execute your strategy?
  • Ensure the objectives are clear and there is shared understanding by everyone. Unclear objectives are one of the topmost barriers to team and organizational performance.
  • Foster a culture that encourages opinions and ideas that challenge the consensus. People should feel free to share their ideas and not hold back for fear of others penalizing them or thinking less of them. Collaboration is hindered when one or two people dominate the discussion, are arrogant, or don’t think they can learn anything from others.
  • Groups perform more effective under certain circumstances, and less effective under others. There is a tendency to fixate on certain topics of discussion amongst groups which often leaves members distracted from their ideas. To reduce the negative effects of topic fixation, members of the group should be given periods to work alone and switch constantly between individual activity and group interaction.
  • Effective collaboration can happen if the people involved come from diverse backgrounds and possess complementary skills to prevent conformity. The best collective decisions or creative ideas are often a product of different bodies of knowledge, multiple opinions, disagreement, and divergent thought processes, not consensus or compromise.
  • New technologies are making collaboration easier than ever, enabling us to increase our reach and broaden our network. Although new technology helps, it will not make your organization collaborative without the right culture and values in place. First, define what you want to achieve through collaboration then use these tools to promote creative collaboration.

How else are you championing collaboration within your organization to create value and succeed?

Thanks for sharing:

Related posts:

  1. Mapping the Organization’s Operational Processes
  2. Identifying Your Organization’s Critical Success Factors
  3. Improving Process Effectiveness In A Complex Organization
  4. The Resilient Organization
READ MORE

ERPM Insights is dedicated to helping leaders,
teams, and organizations implement their
strategies more effectively, make informed risks
decisions and improve business performance.

Send an email

Connect

Linkedin-in Twitter

Copyright © 2023 – ERPM Insights. All Rights Reserved.

See our Privacy Policy