Predictive Algorithms Are No Better At Telling The Future Than A Crystal Ball

By Uri Gal, University of Sydney

An increasing number of businesses invest in advanced technologies that can help them forecast the future of their workforce and gain a competitive advantage.

Many analysts and professional practitioners believe that, with enough data, algorithms embedded in People Analytics (PA) applications can predict all aspects of employee behavior: from productivity, to engagement, to interactions and emotional states.

Read more: Digital public: looking at what algorithms actually do

Predictive analytics powered by algorithms are designed to help managers make decisions that favourably impact the bottom line. The global market for this technology is expected to grow from US$3.9 billion in 2016 to US$14.9 billion by 2023.

Despite the promise, predictive algorithms are as mythical as the crystal ball of ancient times.

Predictive models are based on flawed reasoning

One of the fundamental flaws of predictive algorithms is their reliance on “inductive reasoning”. This is when we draw conclusions based on our knowledge of a small sample, and assume that those conclusions apply across the board.

For example, a manager might observe that all of her employees with an MBA are highly motivated. She therefore concludes that all workers with an MBA are highly motivated.

This conclusion is flawed because it assumes that past patterns will remain consistent. This assumption itself can only be true because of our experience to date, which confirms this consistency. In other words, inductive reasoning can only be inductively justified: it works because it has worked before. Therefore, there is no logical reason to assume that the next person our company hires who has an MBA degree will be highly motivated.

Read more: How marketers use algorithms to (try to) read your mind

Assumptions like these can be coded into hiring algorithms, which, in this case, would assign a weighting to all job applicants with an MBA degree. But when inductive reasoning is baked into the code of hiring applications, it can lead to unfounded decisions, adversely impact on the bottom-line, and even discriminate against certain groups of people.

For example, a tool used in some parts of the United States to assess whether a person arrested for a crime would re-offend was found to unfairly discriminate against African Americans.

They lead to self-fulfilling prophecies

Another flaw in the predictions thrown up by algorithmic analysis is their propensity to create self-fulfilling prophecies. Acting on algorithmic predictions, managers can create the conditions that ultimately realise those very predictions.