Why Progress Goes Unnoticed

Why do some problems seem to defy the human ability to solve them? Are they truly unsolvable? Is the complexity beyond our comprehension? Do we lack the resources, human or otherwise? Or are we solving problems but lack the capacity to recognize it?

Lately, I've become increasingly interested in the disconnect between the pessimism I hear from others and my own experiences. I'm hearing one thing and experiencing something totally different. I see progress all around me, yet I only hear complaints of worsening affairs.

I understand that much of this is due to my circumstances — call it luck, being blessed, even call it privilege — whatever it is, I recognize its existence. But that alone cannot possibly explain the gap. 

To better understand this noise-to-signal mismatch, I've researched various psychological phenomena. To my surprise, I discovered a concept that helps explain the disconnect troubling me: prevalence-induced concept change in human judgment.

In 2018, American psychologist and writer Daniel Gilbert and his team discovered the concept. According to their research, when a problem becomes less prevalent, we redefine it. In other words, as a problem diminishes, we expand its definition. This leads to progress being discounted (or denied altogether). 

Gilbert explained the phenomena:

"When problems become rare, we count more things as problems. Our studies suggest that when the world gets better, we become harsher critics of it, and this can cause us to mistakenly conclude that it hasn't gotten better at all. Progress, it seems, tends to mask itself."

Bingo. 

This cognitive bias, a version of confirmation bias, isn't reserved only for large, seemingly intractable social issues. Gilbert and the team demonstrated this phenomenon in a series of simple experiments.

"We had volunteers look at thousands of dots on a computer screen one at a time and decide if each was or was not blue," Gilbert said. "We lowered the prevalence of blue dots, and what we found was that our participants began to classify as blue dots they had previously classified as purple."

These findings have broad implications. 

First, they are an excellent reminder of the importance of the stimulus domain upon human judgments. 

The Two-Way Street of Neurological Processes: Inputs and Outputs

We often think of our brains as the ultimate arbiters of truth, processing information from our senses and delivering objective judgments. But this view is incomplete. Our inputs also have a significant impact on our outputs. This neurological process is a two-way street, with a heavy traffic flow in both directions.

We are often unaware of the extent to which our thoughts and behaviors are shaped by the information we consume. Even if we think we have a world-class filter, our subconscious constantly absorbs information in ways we don't fully understand.

Prevalence-induced concept change should remind us to be mindful of what we feed our minds. 

Differing Worldviews: The Battle of Problem Perception

The concept can also explain why individuals with similar backgrounds sometimes firmly hold opposing beliefs. Consider someone who is intensely passionate about a cause. Even if the desired change they sought is achieved, it is unlikely that they will move on to a new cause. Over time, their identity becomes intertwined with the issue they have advocated for, and they unconsciously expand their idea of the original problem. In other words, they start counting purple dots as blue ones.

"Expanding one's definition of a problem may be seen by some as evidence of political correctness run amok," Gilbert said. "They will argue that reducing the prevalence of discrimination, for example, will simply cause us to start calling more behaviors discriminatory. Others will see the expansion of concepts as an increase in social sensitivity, as we become aware of problems that we previously failed to recognize."

Gilbert's team wisely didn't take a side, nor should they have. But I'm willing to say that to reduce conflict in modern life, sometimes the location of the goalpost needs to be constant. 

Misallocation of Resources and Policy Decisions

The word "creep," in materials science, describes the deformation of a solid under pressure. However, it is increasingly used to describe the gradual expansion of an original boundary. 

In the military, I used "mission creep" when referring to expanding the objectives beyond what was initially intended. This happens for various reasons, such as changing circumstances or political pressure. Mission creep results in unnecessary casualties, financial costs, and a loss of focus and effectiveness. 

In project management, "scope creep" refers to the growth of a project's scope beyond the original agreement. Scope creep causes delays, cost overruns, and diminished quality. 

Prevalence-induced concept change leads to creep in resource allocation. Policy-makers (sometimes) unintentionally expand the definition of a problem when the actual prevalence of the original problem has decreased and adopt policies misaligned with the current state of affairs. Failing to assess current problems accurately can lead to unnecessary and expensive solutions for issues that are no longer a major concern. Furthermore, it may cause important emerging issues to be overlooked and unaddressed.

Maybe I’m not Crazy

Thanks to Daniel Gilbert, I'm starting to think I'm not losing my mind. Maybe the situation isn't as dire as it seems. It's plausible that some of the issues that consume society's focus are improving, just not at the same pace as our expectations are evolving. Maybe the blue dots are disappearing, and now we're calling purple dots blue ones.

Unfortunately, simply being aware of biases like Prevalence-Induced Concept Change doesn’t necessarily prevent them from happening. Even when Gilbert’s study participants were warned about the effect and offered financial incentives to resist, the bias persisted. However, it’s still important to understand this phenomenon and bring it into discussions. By increasing awareness and discourse around it, we can work towards reducing its impact and fostering a more positive and unified society.