Edward DeBono argued three decades ago that creativity is the most important human resource of all, and it’s truer now than ever. Without a strong culture of innovation that harnesses employee creativity effectively, organizations are much more likely to fail in the modern economy.
But capturing employee creativity towards organizational innovation is challenging. Our current management practices evolved from Fordist (and Taylor’s scientific management even earlier) approaches to measuring productivity that served well for linear, predictable systems. For example, if a production line produces X widgets/week, you could be sure that by adding another production line you can get 2X widgets/week. Unfortunately, applying these approaches to innovation doesn’t work for two reasons:
- Creativity is fundamentally non-linear. During brainstorming two people with individually weak ideas might discover key insights that lead to a billion dollar one. Or, an individual could be wrestling with a challenging problem for several days with very little to show for it and then suddenly find an interesting solution in a day.
- Complex problems often require a group to collaborate in order to find efficient solutions. But simply bringing people is not enough. Depending on the group and how individuals interact, you could get collective intelligence or collective stupidity. Most incentives are aligned with individual performance and backfire when work is highly dependent on group performance.
So instead of focusing on linear metrics like bugs fixed per week or number of features shipped, we need better mechanisms that allow for collective intelligence to flourish.
Swarm Intelligence
Some of the most fascinating examples of complex problem solving in groups come from nature. Groups of ants, bees, birds and other creatures can solve surprisingly complex problems easily even though each individual in the swarm is only following simple rules. Intelligent behavior emerges from these simple interactions. For example, army ants can travel for long distances without traffic jams in dense three lane highways by following simple rules like outgoing ants turn aside when they encounter incoming ants.
Collective intelligence works surprisingly well in many kinds of problems like estimation or prediction. One of the most well known examples comes from Francis Galton, a pioneering statistician and half-cousin of Charles Darwin, who asked people to guess the weight of an ox at a fair in 1906. While individual responses varied from 1,074 to 1,293 lbs, the mean came to 1,197 lbs closest to the true value of 1,198 lbs!
Group intelligence also outperforms individual intelligence in much more complex situations like hiring the right candidate for a role. Google at one point had a fairly complex hiring process where candidates could face up to 25 interviews. After analyzing data on interview performance and subsequent job performance, they were able to simplify the process down to 4 interviews which gave them an 86% confidence level. The interesting thing that came out of their analysis was that none of their managers – regardless of the years of experience under their belt – were individually good at predicting who would make a good employee. The wisdom of the group of four always outperformed individual prediction!
Research from several domains shows that for groups to work intelligently, few key conditions need to be met:
- Individual members must be well-informed about their area to have a better than 50-50 chance of getting the right answer.
- Cognitive diversity of the group is important.
- Individual members have to think independently and not influence each other’s opinions.
- Individual members should be unbiased.
Most organizations spend incredible amounts of time trying to find the most qualified candidate with the right set of domain skills during hiring, so the first criteria is often not an issue. The remaining three criteria often get underestimated (or ignored) which limits the amount of innovation companies produce.
Cognitive Diversity
The kind of diversity important in complex problem solving is cognitive diversity which includes knowledge (people skilled in different domains), perspectives (different ways of viewing a problem) and problem solving approaches (different heuristics or ways of generating solutions to the problem).
Cognitive diversity is correlated with diversity based on common factors like gender, race or ethnicity. Companies that prioritize increasing diversity have a better chance of being innovative and real-world results back that up. More diverse and inclusive companies are 1.7 times more likely to be innovation leaders in their industry. Scott Page, complexity scientist and author of The Difference, captured this perfectly when he said, “being different is as important as being good.”
Independent Thinking
Perhaps the most important criteria for harnessing group intelligence is for individuals in a group to think independently and not be swayed by others opinions. Unfortunately, this is also one criteria that most groups don’t pay attention to. When groups get together without adequate preparation, a few people might dominate discussions or people might be overly influenced by their leaders’ opinions. In such situations, decisions quickly fall prey to groupthink.
To mitigate this, individual members need to think about the problem at hand and commit to their ideas and reasoning before they share with the group. This makes a huge difference in the quality of decision making. Despite the popularity of traditional brainstorming, groups that engage in nominal brainstorming (where people brainstorm individually before sharing with the group) come up with twice as many ideas overall and more original ones.
Unbiased Thinking
Groups also suffer from biases which include both stereotypical (like gender) or cognitive biases. Hiring a more diverse workforce can mitigate the effects of stereotypical biases but cognitive biases can still remain. Two common cognitive biases are myside and one-sided thinking.
Myside bias results from people’s inclination to favor arguments that support their opinion while ignoring or minimizing contradictory viewpoints. Organizational cultures where “defending your idea” is deemed an important leadership trait accentuate this bias.
One-sided thinking is our preference for arguments that are one-sided rather than those that offer multiple perspectives. A one-sided solution appears simpler and cleaner, and because it causes less cognitive strain, it’s more persuasive. As a result, leaders are easily swayed by a person who presents a simple argument compared to another who presents a more nuanced view.
Useful Strategies
Groups can be smart or dumb. It’s easier now to see why groups end up with harmful outcomes in the age of social media where most, if not all, of the conditions get violated. Organizations can avoid some of these pitfalls and create a more innovative culture by implementing the right norms and incentives. Here are a few strategies to consider:
- Hiring a diverse workforce is necessary but it’s not sufficient. Without implementing good systems where diverse viewpoints are included in decision making, one can’t reap the benefits of diversity. Despite the focus on DEI, inclusion from the perspective of innovation is still an issue among many companies.
- Creating the right norms for decision making are important when it comes to ensuring independent and unbiased thinking. The Build, Teardown and Rebuild (BTR) method is one such example that removes biases from decision making in groups and can be implemented relatively easily.
- Ensure that your incentives and metrics don’t create the wrong kind motivations when it comes to contributing to groups. Complex problem solving is an emergent property when groups follow certain conditions. It requires individuals to behave in prosocial ways that are often at odds with incentives which encourage individual performance at the expense of group benefit.
Complex and creative problem solving is a highly social problem. What gets created in any company – code or product or service – is heavily dependent on the people that come together to solve the problem. In such an environment, social engineering is as important (if not more) because how intelligent a group behaves depends on how the group interacts and problem solves. Creating the right processes and incentives can improve a company’s innovation throughput and set them on a competitive path.