Posted on

When Incentives Go Wrong: The Hidden Costs of Motivating Creative People

In the early 1900s, commercial fishing interests along the New England coast had a problem: the abundant seal population was reducing the fish supply. Convinced that fewer seals would be the solution, Maine and neighboring states began paying bounties for every seal killed. As proof, one simply needed to bring a seal’s nose to a town clerk and collect the reward. 

But reality turned out to be far more intricate than what the policy’s creators envisioned.

For generations, the Passamaquoddy people of eastern Maine had hunted seals for food and clothing. By 1900, logging, fencing, dammed rivers, and mounting regulations had restricted the Passamaquoddy people’s access to their ancestral lands. Ocean hunting remained one of the few avenues left to maintain traditional subsistence practices, providing both food and clothing. As a result, when commercial interests pressured the state to eliminate local seal populations, they directly endangered an essential Indigenous livelihood.

In response, several Passamaquoddy hunters devised a clever workaround. Guided by traditional values that discouraged killing seals en masse, skilled artisans crafted convincing fake noses using small fragments of seal hide. A single pelt could thus produce dozens of bounty payouts. Suspicions rose in January 1904 after two Passamaquoddy men attempted to redeem 86 noses at once in Portland. The sheer scale of the operation was striking: claims spiked from 208 bounties in 1903 to 2,632 in 1904, prompting authorities to quickly end the program.

The scheme was illegal but reducing it to fraud over-simplifies the story. Commercial fisheries wanted to eliminate competition for fish, leading officials to establish a quantifiable metric (seal noses). Faced with a policy that threatened both their environment and their culture, the Passamaquoddy capitalized on the fact that the state was not truly paying for population reduction. It was paying for physical tokens. 

It is a wonderfully absurd example of a serious management problem. Incentives require us to translate what we really want into something observable enough to reward. But the moment we define that, the measure itself begins to compete with the mission.

This problem is particularly acute in creative work. Leaders want innovation, so they reward patents, product launches, ideas submitted, revenue generated, individual performance, or promotion-worthy accomplishments. 

But creativity works differently. So the right question is not, “Do incentives motivate?” because, of course, they do. The more useful question is: “What exactly do they motivate people to do?”

Research suggests that incentives shape creative work through at least three broad mechanisms: the effort type,  the orientation of thought, and the quality of social interaction.

1. Type of Effort 

In the 1990s, Safelite Glass Corporation changed how its windshield installers were paid. Instead of relying primarily on hourly wages, it introduced piece-rate compensation tied to output. Economist Edward Lazear studied what happened next.

Productivity per worker increased by roughly 44%. Part of the effect came from attracting and retaining more productive employees, but existing workers also increased their output. 

This is an important place to begin because incentives clearly work.

But notice the character of the work. Both Safelite and the workers know what a successful windshield installation looks like. You could consider this kind of work a “routine” or a “predictable” task, where both outcome and the process to achieve that outcome are very well defined. Employees, therefore, have considerable control over accomplishing the goal and incentives encourage people to achieve the goal faster.

Creative work is different because the process (and sometimes even the goal) often do not yet exist.

Imagine asking one engineer to install ten more windshields and another to invent a radically better way of replacing automotive glass. Greater intensity is likely to help the first. The second may need to slow down, explore a strange possibility, discard several promising ideas, or spend a week understanding whether the problem has been framed correctly.

A meta-study analyzed 183 studies involving more than 200,000 participants. They found that both intrinsic motivation and extrinsic incentives predicted performance, but their relationships differed according to the type of performance. Intrinsic motivation was particularly important for performance quality, while incentives were comparatively more important for performance quantity. 

Organizations often use the effort in the sense of intensity like working faster or longer hours. But creative work requires a different kind of effort that’s not as easily measured:  searching broadly for alternative solutions, experimenting, tolerating ambiguity, or persisting through roadblocks. 

A researcher under pressure to produce publications can work extremely hard while avoiding risky research questions. A product team can sprint heroically toward a launch date while failing to ask whether customers really need the product. An executive can relentlessly optimize quarterly results while starving experiments whose payoff lies years away.

In these cases, the incentives simply push effort in the wrong direction. 

If the path is well-known, extrinsic incentives can accelerate progress. However, if discovering the path is the work, pushing harder may simply get people to the wrong destination faster.

2. Cognitive orientation

In an elegant series of experiments, Teresa Amabile and her colleagues, had children and adults perform creative activities under different reward conditions. The key manipulation was whether participants understood the activity itself as something they were doing in order to obtain a reward. Across the studies, explicitly contracting to perform the creative activity for a reward reduced the creativity of the resulting work relative to relevant comparison conditions. 

The problem is not necessarily the reward itself. It is what the reward can do to attention. Without a salient external incentive, someone absorbed in a creative problem might ask: “What would happen if we tried the opposite?” or “What assumptions are we making?”

But when you make the reward salient, a different question enters the mental workspace:”What do I have to do to earn it?” That question can be highly productive when the task is unambiguous and predictable. But innovative work is often the opposite.

A meta-analysis of 60 studies found that rewards explicitly contingent on creative performance tended to improve creativity, particularly when accompanied by constructive, task-focused feedback and meaningful choice. Ordinary performance- or completion-contingent rewards, however, showed a small negative relationship with creative performance. 

In other words, incentives become attention-directing devices. Tell a team you need twenty ideas and twenty becomes important. Tell engineers that they will be evaluated on lines of code and code volume becomes important. 

Again, the employees aren’t behaving irrationally. They are learning what the organization has made salient and they are simply altering their effort accordingly.

3. Social interaction

The most consequential effects of incentives may not happen inside an individual’s head at all. They happen between people.

In one study, researchers looked at forced-distribution performance systems i.e. the practice of evaluating employees relative to one another. When participants performed a task individually, forced ranking increased their speed. But when work became collaborative, the results reversed. Not only did forced ranking slow down task completion, it also significantly reduced knowledge sharing. 

Consider a scenario where two people are collaborating on a new product. If one person discovers an insight that could significantly enhance the other’s part of the  project, passing that information along makes sense when the mutual outcomes are aligned.

However, if both people are competing for a single promotion, things get tricky. One study investigated this dynamic and found that strong promotion incentives were associated with greater individual effort but lower helping effort toward coworkers. 

A conventional performance-management system may record the first effect but completely miss the second one. For creative work, that blind spot is more dangerous. Innovation is rarely the product of isolated brilliance. Someone supplies an analogy, challenges an assumption, shares a contact, offers technical knowledge, or spends an hour helping a colleague escape a dead end.

Competitive incentives can put a price on that generosity. And under stronger competitive conditions, the consequences can move beyond withholding help.

In one study researchers created experimental tournaments in which participants could improve their chances of winning through productive effort or through actions that reduced a competitor’s performance. As the difference between the winner’s and loser’s rewards increased, participants exerted more productive effort and more sabotage. 

Most workplace sabotage isn’t very dramatic. It often looks more like a delayed reply,  a useful insight kept private, or an idea presented as “mine” rather than “ours.” Individually, these decisions can be rational but collectively, they can destroy the culture needed to support creative work.

That is why incentive design is ultimately also relationship design. It tells employees whether a talented colleague is primarily a source of knowledge, a partner in discovery or a threat to their own reward.

The real question behind every incentive

The Maine officials who paid for seal noses made an understandable mistake. They could not directly purchase their real objective, so they created a measurable proxy. 

More than a century later, organizations do the same thing with sales targets, KPIs, patent counts, performance ratings, promotion tournaments, publication metrics, and innovation bonuses.

The lesson is not that incentives are inherently corrosive. Incentives can increase effort, but they can also redirect effort, narrow attention and change colleagues into competitors. 

So before attaching a reward to creative work, leaders should think carefully about these questions: What behavior will this incentive intensify? What will it cause people to pay attention to? And how will it affect collaboration? These questions shift incentive design from a compensation problem to a problem of organizational architecture.

Posted on

What If Becoming a “Human in the Loop” Is a Bigger Risk Than Replacement?

By the late 1960s, the automobile assembly line stood as one of modern management’s great achievements. It had transformed carmaking into a system of extraordinary scale, speed, and consistency. Yet the same system was experienced very differently by many of the people working within it. Describing his job in a General Motors paint shop, one worker quipped:

“There’s a lot of variety in the paint shop. . . . You clip on the color hose, bleed out the old color, and squirt. Clip, bleed, squirt, think; clip, bleed, squirt, yawn; clip, bleed, squirt, scratch your nose. Only now the Gee-Mads [the General Motors Assembly Division industrial engineers] have taken away the time to scratch your nose”.

The worker remained essential to production but had little authority over the work as a whole. He was, quite literally, a human in the loop.

Several thousand miles away, another worker also repeated a small set of movements every day. Behind his ten-seat sushi counter in Tokyo, Jiro Ono sliced fish, shaped rice, pressed, and served with gestures refined over decades. From a distance, his practice could appear almost as standardized as the automotive work. Yet its apparent sameness concealed continual variation. Rice changed with temperature and humidity; different fish required different preparation; pressure appropriate for one piece could be excessive for another. What looked to a novice like execution of a recipe was, for the master, a sequence of situated judgments developed through attentive practice. Ono described his process simply:

“I do the same thing over and over, improving bit by bit. There is always a yearning to achieve more. I’ll continue to climb, trying to reach the top, but no one knows where the top is. Even at my age, after decades of work, I do not think I have achieved perfection. But I feel ecstatic all day.”

Ono’s repetition made his perception increasingly discriminating. Each iteration provided feedback, improving both the outcome in the present and his ability to judge more finely in the future.

The automobile worker and the sushi master therefore present a paradox. Both repeat, operate within constraints, and participate in larger production systems. Yet repetition degrades the work of one while making it meaningful to the other. The difference is not repetition itself, but how work distributes agency, cognitive challenge, and opportunities for learning. 

The contrast raises an important question for the age of generative AI:

Will AI make more of us like Jiro Ono or more like the assembly-line worker?

The answer may have less to do with the sophistication of the technology than with how we choose to design work around it.

A new kind of Fordism

Much of the conversation about AI and work focuses on jobs: Which jobs will disappear? Which professions are safe? What percentage of a role can be automated?

Those are reasonable questions, but long before AI eliminates an occupation, it can redistribute the thinking used in that occupation.

Imagine a teacher who once designed lessons himself but now increasingly asks AI to create them and then reviews the result. Their productivity might have increased but something important changed in the process. As AI performs more of the judgment-rich activities through which the person previously learned, experimented and developed expertise, human work becomes narrower, limited to prompting, checking, approving and correcting.

We call this emerging pattern cognitive Fordism: a division of cognitive labor in which AI increasingly performs the thinking through which people develop judgment, while humans are left supervising AI’s output.

This raises a fundamental question: Which parts of our thinking are we happy to outsource, and which parts do we need to keep exercising if we want people to remain capable?

Not every use of AI should look the same

One reason this question is difficult is that the same task can mean very different things to different people.Suppose three people ask an AI system to help write a report. 

The first person understands the subject thoroughly so writing the report might simply be a routine task for them. Having AI produce a draft may be an excellent use of automation for such a person.

The second person is new to the field. Writing the report is partly how they will learn to structure an argument and understand the material. Generating the answer for them may save time today while removing the very struggle that would have developed their capability tomorrow.

The third person is trying to invent a completely new approach. They may not want AI to give them an answer at all. They may want it to challenge assumptions, offer unusual alternatives and provoke new directions.

So while on the surface, the activity might look the same it needs three completely different interaction models with AI. 

This is why the conceptual framework in the figure above begins not with the task itself, but with the person’s intention. It describes three broad orientations for AI-mediated work: Execute, Learn and Create.

In Execute, the goal is reliable completion. We already know roughly what good looks like, so allowing AI to carry more of the workload makes sense.

In Learn, the goal changes. The immediate answer matters less than what the person will understand or be able to do afterwards. Here, the best AI may behave more like a tutor by offering explanations, questions or graduated hints rather than simply completing the task.

And in Create, the objective is neither efficient execution nor mastery of an existing solution. It is expanding the possibility space. AI becomes a thought partner: generating alternatives, challenging assumptions or helping connect ideas that might otherwise remain separate.

These orientations are not a ladder in which Create is somehow superior to Learn and Learn superior to Execute. Sometimes execution is exactly what we want. There is little benefit in forcing a senior scientist to manually perform routine writing if their attention would be better spent solving a difficult scientific problem.

The mistake is using Execute as the default relationship for everything simply because AI can produce an answer quickly.

Agency plays a strong role

Most moderately complex work does not stay neatly inside Execute, Learn, or Create. People continually move between them.

A scientist may first need to learn enough to understand an unexpected result, then create several possible explanations, then execute an experiment to test them. The outcome may trigger another round of learning, reframing, and experimentation. The same is true in many professional and educational tasks.

That is why agency matters so much in AI-mediated work.

If the appropriate relationship with AI changes as the work unfolds, people need the ability to change that relationship too. We call this directional autonomy: the ability to influence whether the interaction is oriented toward Execute, Learn, or Create. This is different from substantive autonomy, which concerns how much authority a person has over the work itself—its goals, framing, methods, and standards.

Both matter, but they are not equally available in every situation.

An employee may have little control over the objective they have been assigned. A student may not get to choose the learning outcome, assessment criteria, or even the method they are expected to use. In those situations, substantive autonomy is partly constrained by the surrounding organization, teacher, or institution.

But even when people cannot choose what they are ultimately trying to accomplish, they can still benefit from having some control over how cognition is divided between themselves and AI.

This is why an AI system should not lock someone into a single mode simply because it inferred that mode at the beginning of a task. The system might reasonably infer that the user is trying to Execute, Learn, or Create, but that choice should remain visible and easy to override.

Without that flexibility, AI risks pushing people toward a default role (often execution and supervision) even when the work actually calls for learning or creative exploration. 

From AI adoption to work design

Organizations can easily fall into a productivity trap when using AI. It’s easy to see how much coding was completed or how quickly an email was written. But human development is harder to see. Did the student deepen their understanding of the topic? Or, did a manager develop better judgment? These capabilities accumulate slowly, often through exactly the parts of work that initially feel inefficient: wrestling with uncertainty, considering alternatives, making mistakes and revising our thinking. 

Prioritizing short-term efficiency over deep engagement can erode the very skills needed for future work, putting sustained success at risk.

Viewed from a long-term perspective, incorporating AI becomes a work-design problem, not just a technology-deployment problem. At an organizational level, that might mean conducting a cognitive audit: examining where AI is taking over analytical, creative and practical effort; whether employees can reclaim or redirect that work; and whether jobs are becoming richer or slowly collapsing into narrow monitoring and approval activities.

The future of human–AI collaboration should be judged by more than the speed or quality of the output. A good system should help us produce better work without reducing our capacity to shape the work that comes next.

Posted on

Creativity Needs a Map, Not Just a Bigger Imagination

In 1941, Swiss engineer George de Mestral returned from a walk with burrs clinging to his dog’s fur that were surprisingly sticky. Most of us would have brushed them off but Mestral looked closer. Under a microscope, he saw tiny hooks gripping the loops in fabric and fur. That observation led to an idea that eventually became Velcro. 

The story captures something essential about creativity. New ideas rarely appear from nowhere. More often, they emerge when the mind travels from something familiar to something related, but not obvious. The breakthrough is not simply “thinking harder” but finding a different path.

Our latest paper, Graph Enhanced Creative Cognition for Alternate Uses Task, explores whether large language models can be helped to make those associative journeys. The headline finding is: when AI models were guided toward moderately distant concepts before generating an idea, they produced a much broader range of responses. The number of distinct idea categories increased by 147% for Gemma-4B, 70% for Mistral-7B and 33% for OLMo-7B. When a learning system identified the most promising conceptual paths, those paths generated 16–20% more idea categories than randomly selected ones. 

The study suggests that creativity may depend less on explicitly asking for originality and more on designing better routes to discover it.

Why “be more creative” usually fails

Anyone who has led a brainstorming session knows the pattern. Ask a group for unusual uses for a paperclip and the first answers arrive quickly: a hook, a lock pick, a cable holder. Then the room slows down and people begin repeating variations of the same themes.

Language models face a similar problem. They generate likely continuations from patterns in their training data. This makes them fluent, but it also creates a pull toward familiar answers. Asking a model to “be highly creative” does not necessarily change the territory it explores. Turning up randomness may produce stranger wording, but strangeness is not the same as a genuinely different idea. 

Creativity researchers have long described creative thought as an associative process. Sarnoff Mednick’s classic theory proposed that original ideas arise when people connect elements that are relatively remote from one another. The challenge is to travel far enough from the obvious to find novelty, but not so far that the result becomes meaningless. 

That is the “Goldilocks zone” of creativity: not too close, not too distant.

Giving AI a conceptual stepping-stone

Our study tested this idea using the Alternate Uses Task, a widely used divergent-thinking exercise in which participants propose unconventional uses for ordinary objects. We used six objects (book, table, fork, pants, bottle and coin) and tested three small, open-weight language models, collecting 3,552 valid generations. 

In the baseline condition, the model received a straightforward request: suggest an unusual use for the object.

The experimental condition received one extra instruction to incorporate a concept reached by taking two steps through ConceptNet, a large commonsense knowledge graph. ConceptNet represents everyday concepts as a network of labeled relationships. A “book,” for example, might connect to a “bed” through location, and “bed” might connect to a “plant” through another relationship. The result might be a vertical planter made from stacked books, an idea that is not an obvious association with “book,” yet remains understandable.

The system provides a conceptual stepping-stone: something sufficiently removed to disrupt the default answer, but still connected through a traceable path.

Diversity is not the same as originality

In the paper, we also make a distinction between an individually creative answer and a genuinely diverse body of ideas.

Suppose an AI proposes using a book as a shield. Judged in isolation, the answer might seem original. But if the model returns to “shield” repeatedly, or other models also propose the same, its creative range is narrower than the individual score suggests. We found that standard originality scoring sometimes gave different ratings to semantically similar answers and compressed many responses toward the high end. To address this, we grouped similar responses into semantic clusters. Ideas involving protection might form one cluster; furniture another; gardening a third. The more clusters a model reached, the more widely it had explored the idea space.

Using this approach we saw that graph-guided prompting increased the number of unique clusters by 147% for Gemma-4B, 70% for Mistral-7B and 33% for OLMo-7B. The prompts helped the models explore new categories of ideas. 

For Gemma and Mistral, graph guidance also produced more responses in the rarest (and therefore most original) clusters. OLMo became more diverse overall but showed a small decline in the highest-originality group showing that diversity and originality often reinforce each other, but they are not identical. 

Not every detour leads somewhere useful

Simply wandering through a network does not guarantee inspiration. Some paths loop back to where they started. “Book → cover → book” creates no real distance. Other paths end at concepts already closely associated with the starting point.

We therefore asked whether a system could learn which associative paths were more creatively productive.

We trained a graph neural network to rank pathways through the “book” portion of ConceptNet. Its technical performance was modest, but it learned enough to identify better routes. Compared with 100 randomly chosen paths, its top-ranked 100 paths produced 16–20% more unique idea clusters across the three models and more ideas in the highest-originality category. 

The finding shows that creative support systems may eventually do more than provide information or generate answers. They could help people navigate conceptual landscapes by suggesting which analogy, adjacent field or surprising connection is most likely to open productive territory.

Creativity as guided exploration

While the study is an early proof of concept, it provides some useful insights. One, it is computationally easier to measure novelty and diversity but usefulness requires human judgement. Generating truly creative solutions requires (at least as of now) a healthy collaboration between humans and AI. Two, designing the right process, where AI can suggest different directions to think about, can yield higher levels of creativity. Completely open prompts often leave people and machines circling the most accessible ideas in practice. AI can be a useful partner by suggesting new associations, analogies or metaphors as stimuli. Finally, traceability is a promising theme. A graph-guided idea comes with a kind of “cognitive lineage” that allows us to inspect the route that helped produce it. In scientific discovery, strategy and learning, the path may be almost as valuable as the answer because others can evaluate, adapt and extend the thinking. 

George de Mestral did not invent a fastener by staring harder at fabric. He followed a path from burrs, to hooks, to loops, to textiles that led to his world changing idea. Perhaps the future of creative AI, and of innovative organizations, will not come from demanding more creativity but by being more deliberate about the conceptual journeys that make better answers possible.

Posted on

Designing AI to Stretch the Mind

In our innovation programs with students, we often began with a deceptively simple exercise: take two things that do not obviously belong together and force a connection.

At first, the combinations sound absurd and students look unsure. But as they start working together ideas start to make sense. An umbrella and a jump-rope becomes a “Jumbrella” — a water-skiing device where you can sit and relax while being towed by a motorboat.  The point is not to reward randomness for its own sake but to help students escape the first layer of obvious ideas and enter a more interesting space where new meaning has to be constructed.

A less random exercise uses association maps where you try to connect ideas that are 2-hops away from the core object that you are trying to improve. One student, using an association map, started with the idea of a glove. And as he drew the map, he reached “scissors” and a new idea emerged: a glove with a cutting blade attached, making it safer and easier for children who find scissors difficult to hold. In the process of bringing the two concepts together, he recognized a common human problem and found a useful solution. 

This is what creativity looks like. It is not simply “thinking outside the box.” More often, it is a disciplined way of making novel and meaningful connections.

And the same cognitive techniques that help students invent also help them learn.

Creativity As a Learning Engine

We often treat creativity as a detour from learning, but it is actually one of the most effective ways to learn traditional subjects as well. In another of our programs, students created their own numbering systems as part of an imaginary world-building exercise.

On the surface, this looked like imagination: invent a world, design its rules, create its language, build its symbols. But when students had to create a numbering system, they started doing serious mathematical thinking. A worksheet about place value can tell a student how a base system works. But inventing a base-5 or base-11 numbering system forces the student to confront the mechanics of place value at a deep, structural level. 

We usually treat learning and creativity as separate capacities. Learning is associated with acquiring knowledge, mastering facts, and performing correctly. Creativity is associated with novelty, imagination, and original production. But this separation is misleading. At a cognitive level, both require the same fundamental act of viewing the problem from many different perspectives, recognizing gaps in existing mental models and updating them. 

A learner encounters something new and must fit it into what they already know. Sometimes the new idea can be assimilated easily. Sometimes it does not fit, and the learner has to reorganize their understanding. A creative thinker does something similar. She takes existing ideas, experiences, concepts, and constraints and recombines them into a structure that did not exist before. In both cases, the mind is not passively receiving information but actively reconfiguring an internal model of the world.

And classroom evidence points in this direction. For example, in one study students learning statistics were asked to invent ways of comparing data sets before receiving direct instruction on standard measures. Their early solutions were often incomplete, but the act of invention prepared them to understand the formal ideas more deeply. Similarly, in science classrooms, students learn abstract concepts more effectively when they create analogies and examine them. A circuit can be compared to water flow, but the analogy must also be questioned. What is like the battery? What is like resistance? Where does the comparison mislead us? Learning  does not come from the analogy alone but also from the act of mapping, testing, and revising the analogy. A newer study found that goal-directed association can better explain the creativity-learning link. 

Creativity Techniques as Metacognitive Tools

Creative techniques like associative or analogical thinking, are not just ideation tools — they also act as metacognitive tools. Much of our thinking is invisible, even to ourselves. A student might say, “I don’t get it,” without knowing if the roadblock is vocabulary, structure, or a false assumption. However, when that same student maps associations or compares metaphors, their thinking becomes something they can inspect. They can see which connections are obvious, which are missing, which are forced, and which open a new path.

This gives students a toolkit for ambiguity. In school, problems are often presented with clear instructions, known methods, and expected answers. In the real world, the most important problems rarely arrive that way. They are open-ended, poorly structured, and full of incomplete information. The student who has practiced making thinking visible has an advantage. She knows how to begin when the path is unclear: generate associations, map the territory, compare frames, test analogies, revise assumptions.

The AI Challenge

Learning and metacognition are precisely what are at risk with artificial intelligence.

AI can be an extraordinary tool for learning. It can explain concepts, generate examples, translate language, summarize research, and provide feedback at a scale no human could manage alone. But it can also short-circuit the mechanisms through which learning and innovation occur. If students ask AI for the answer before they have formed their own associations, challenged their assumptions and wrestled with their own confusion, they may produce better work but atrophy their thinking skills in the process.

Humans have always used tools to reduce mental effort. We write notes so we do not have to remember everything. We use calculators so we do not have to perform every computation by hand. Offloading is not inherently bad. In fact, civilization depends on it. The question is what and how much we offload.

When we offload storage, we may free the mind for higher-order work. But when we repeatedly offload sense-making, judgment, and creative struggle, we risk weakening the very capacities that make us learn and create.

So, the real question is: How should AI be designed if the goal is not to replace thinking but to stretch it?

An AI tutor could give the answer immediately. Or it could ask the student to first generate three associations, choose the strangest one, and explain how it might connect. A writing assistant could rewrite a paragraph. Or it could offer competing metaphors and ask the student which one best fits the argument and why. 

These are not small design choices. They reflect two very different theories of learning. One treats the learner as a consumer of answers. The other treats the learner as a builder of models.

We often associate technology with a certain kind of dopamine loop: the ping, the scroll, the like, the instant answer. This kind of reward captures attention by hacking into our fears and insecurity. But there is another kind of reward that is underused: the reward of insight.

It’s the “aha” moment when a strange association suddenly makes sense. It is the pride and satisfaction of finding a clever solution to a real problem. That is the “better dopamine” to use. We should design systems that provoke association, analogy, reflection, and metacognition. That can lead to a more effective and beneficial partnership between humans and AI.

Posted on

The Illusion of Rationale

In 1931, Norman Maier designed an elegantly simple experiment about human reasoning. Subjects entered a large room at the University of Chicago where two cords hung from the ceiling: one near the wall, the other from the center of the room. Their task was to tie the ends of the two cords together. 

The catch was that the cords were too far apart. If a subject held one cord, the other was out of reach. The room contained objects that could help: chairs, poles, clamps, pliers, extension cords, tables. Maier was not interested in whether people could find any solution. In fact, several solutions were available. A person could anchor one cord to a chair and bring the other over. They could lengthen one cord with an extension cord. They could pull one cord closer with a pole.

But Maier was especially interested in a fourth, less obvious solution: tie a weight to the cord in the center of the room, set it swinging like a pendulum, grab the other cord, and catch the swinging cord when it returned. This kind of solution requires a shift from viewing a cord not as a cord, but as a pendulum. These kinds of mental shifts are interesting because they often lead to more creative solutions. 

Once a subject found one solution, Maier simply said, “Now do it a different way.” The experiment continued until the person either discovered the pendulum solution or became stuck. If the subject worked for at least ten minutes and insisted there was no other way, Maier introduced what he called “helps.” The first help was subtle. The experimenter walked across the room, brushed the center cord, and set it slightly in motion. The subject was not told that this was a hint. If that failed, the subject was handed a pair of pliers and told there was another way to solve the problem using them.

The results fell into three groups. The first group discovered the pendulum solution without help. The third group failed to find it even after help was given. But the second group was the most revealing one as they solved the problem only after receiving Maier’s hints.

With this group Maier could compare what had objectively influenced the solution with what people consciously reported afterward. The hint had often worked. In fact, the solution appeared on average only 42 seconds after the effective help was given. Yet many subjects did not identify the swinging cord as the cause of their insight.

Instead, they produced explanations that sounded plausible. One said, “It just dawned on me.” Another thought that a course in physics may have suggested it. A psychology professor reported thinking of “monkeys swinging from trees.” These were not necessarily dishonest answers. They were stories constructed from what was available to consciousness.

Maier’s own conclusion was that when a solution finally appears, “the cue which sets it off is not consciously experienced.”

Maier’s study was about the hidden architecture of human judgment. We often know what we decided and we can usually offer a reason. But we may not know what moved the rope.

The Mind’s Coherent Narrative

Nearly half a century later, psychologists Richard Nisbett and Timothy Wilson gave this phenomenon one of its most memorable descriptions: people often “tell more than they can know.” Their argument was that we have access to some mental content: our feelings, beliefs, images, intentions, and fragments of thought. But we often do not have direct access to the cognitive processes that produce our judgments.

So when someone asks, “Why did you choose that?” the mind does something useful but often incorrect. It generates an explanation that sounds reasonable and may even contain part of the truth.

But it is not necessarily a faithful transcript of the decision process. It is more like a press briefing after a complex geopolitical event. A spokesperson stands at the podium and offers a coherent account. The account may be polished but is a simplified version of the more complex reality. 

The same happens inside our minds. We take one or two visible causes and elevate them into “the reason.” We say the strategy was selected because of market opportunity or that the candidate was hired because of their leadership presence. 

Sometimes those explanations are right. Often, they are incomplete. And occasionally, they are beautifully wrong.

Reasons That Sound Right

One of the most important patterns in this research is that people tend to explain decisions using reasons that are socially available. By “socially available,” I mean reasons that are easy to defend and likely to be accepted by the audience.

This matters because many real influences are hard to confess or hard to detect. A person rarely says, “I trusted him because he reminded me of myself.” Or, a team rarely says, “We preferred the familiar option because uncertainty made us anxious.” Instead, we reach for explanations that sound objective, competent, and culturally approved.

We emphasize one or two dimensions and ignore others that may have played a larger role.

This is not necessarily a moral failure but a cognitive one. In a choice blindness experiment, researchers showed participants two options, such as two faces, and asked them to choose which they preferred. Through a sleight of hand, the researchers sometimes gave participants the option they had rejected and asked them to explain why they had chosen it. Many participants did not notice the switch. Instead, they confidently explained why they preferred the face they had not actually selected.

The mind did not say, “Something is wrong here.” It said, “Let me explain.”

This also helps explain why our reasoning so often has a persuasive quality. Hugo Mercier and Dan Sperber have argued that human reasoning may have evolved less as a truth-finding machine and more as a social tool for argumentation. Reasoning helps us justify ourselves, challenge others, and coordinate within groups. In other words, reasoning evolved to improve communication and coordination. But internally, the process of inference is very different from what the external argument we make. 

The AI Mirror

This brings us to artificial intelligence.

When a large language model gives an answer and then explains its reasoning, we are tempted to treat the explanation as a window into how the answer was produced. This temptation grows stronger when the reasoning is step-by-step, articulate, and professionally formatted. It feels like the model is transparent and showing its work.

But recent research on chain-of-thought reasoning suggests that this confidence can be misplaced. Models can produce explanations that are plausible but unfaithful. In one experiment, researchers introduced hidden biases into prompts, such as making a particular answer option more likely. The model’s final answer changed, but its explanation often failed to mention the true influence. Instead, it rationalized the answer after the fact.

This is the AI version of Maier’s rope.

Why does this happen? AI has been trained on vast amounts of human language, and human language is full of post-hoc justification. The model learns what explanations sound like. It learns which reasons tend to accompany which conclusions. It learns the socially available vocabulary of justification like efficiency, fairness, or customer value. 

In that sense, AI mirrors one of our oldest habits. When the real causal path is inaccessible or difficult to articulate, produce a reason that is coherent, acceptable, and close enough to the surface.

Navigating Reasoning Hallucinations

Both humans and AI can produce narratives that are coherent without being causally faithful. Both can overemphasize one or two salient dimensions while ignoring other (and potentially larger) variables. Both can justify a conclusion using reasons that are more available than accurate. 

This creates a new kind of risk. We may begin outsourcing not only analysis to AI, but also justification. A model recommends a course of action, summarizes the rationale, and the rationale sounds reasonable. The explanation appears to increase transparency but if the explanation is unfaithful, it may instead increase misplaced confidence.

The answer is not to reject human intuition or AI reasoning. The answer is to treat explanations differently.

An explanation should not be seen as the end of inquiry. It should be treated as a hypothesis about causality. When a person says, “I chose this because of X,” we should hear, “X is the reason currently available to me.” When an AI says, “The answer is Y because of these steps,” we should hear, “Here is a plausible reconstruction that may or may not reflect the actual basis of the output.”

For humans, that means comparing stated reasons with behavior, context, incentives, and patterns over time. For AI, it means testing explanations against interventions. Does the answer change when irrelevant details change? Does it remain consistent when the same question is reframed? 

The great irony of our time is that in building intelligent machines, we have inadvertently created a perfect explanatory mirror of ourselves. We built systems that generate language, and language is where human reasoning most often performs its magic trick. We tell the story we can live with, and that story serves a vital purpose: it allows for coordination, communication, and forward momentum in communities. But we also need to accept the limits of that story. Progress depends on knowing when the language that comforts us and allows us to coordinate is not the same as the underlying, unarticulated process that actually moved the rope.