Let’s choose collective intelligence over the madness of mobs

As a Chinese American immigrant growing up in New York City in the 1960s and 1970s, I experienced Asian hate firsthand. But we had a different name for it — we called it “Tuesday,” or sometimes “Saturday,” and so on. It was a fact of life, and I quickly learned that other ethnic groups experienced similar hate. Blacks, Hispanics, Jews, Muslims, and other groups, along with single, divorced women like my mom, were also occasional targets of contempt. So much for the “melting pot,” which sometimes felt more like a cauldron of racism, sexism, and other -isms.

Remarkably, this same milieu also gave us the American dream. Working overtime shifts as a secretary, my mother single-handedly raised three children and saved enough to send us to the best colleges that money, scholarships, and student loans could buy. Her sacrifices made possible my privileged life as a college professor. However, despite all my research experience, I realized that I had never researched how Asian hate could coexist within the same society that had paid for the pre-college and graduate education that has given me the best job in the world.

Thus, I set out to understand the origins of bias and discrimination. Recognizing that I’m an economist, not a race relations expert, I began by asking how group behavior works in financial markets. A central idea of economics — the efficient markets hypothesis — is that the random interactions of many individuals can produce a remarkable degree of collective intelligence. By harnessing this “wisdom of crowds,” financial markets can fuel tremendous economic growth and innovation, leading to advances such as new cancer drugs, self-driving cars, smartphones, the Mars rover, and many more.

On the flip side, failures in collective intelligence give us economic bubbles, crashes, and global financial crises — the “madness of mobs,” or the “animal spirits” that John Maynard Keynes warned about eight decades ago.

Could these same mechanisms also be responsible for bias and discrimination, a form of collective ignorance?

Using a mathematical model of natural selection on behavior, Peking University’s Ruixun Zhang and I revisited the controversial idea of group selection, in which evolutionary forces operate not just on genes but also on groups of individuals. Our model shows that political polarization, bias, and discrimination can emerge in environments where sudden and significant technological changes threaten the dominance of one group while allowing newly emerging groups to replace them.

In other words, the forces of natural selection sometimes cause us to form groups based on hate, sometimes unconsciously, and such alliances often reduce our collective intelligence. These new and unlikely coalitions can emerge autonomously through leaderless organizations, sometimes with disastrous consequences.

Evolution can drive our prejudices. Since Darwin’s publication of “The Origin of Species” in 1859, we have known that groups compete in order to survive. If competition and cooperation can exist together, the results can be remarkable — take, for example, the global collaboration that produced COVID-19 vaccines. Alternatively, competition without cooperation can lead to great harm, such as state-sponsored terrorism.

We also know that implicit assumptions are often baked into existing data sets. Consider bias in its most benevolent form: A talented designer’s bias toward a particular color or fabric will become next year’s popular fashion line. Steve Jobs’ bias for clean design and beautiful typefaces gave us the iPhone.

Yet we know that not all biases are benign. As we become an ever more data-driven society, biases that exist in current data sets must be documented and understood. For decades, pharmaceutical companies tested their drugs only on white males, and automobile crash test dummies were sized only for men, leading to fatal design flaws for those outside the test-group parameters. We must uncover dangerous biases in other data sets before they lead to unintended consequences.

Unfortunately, the current political and economic environment has left our language emotionally charged and our news coverage of these issues counterproductive. The language we use to discuss existing biases and propose positive changes in our system is critical, lest our dialogue become more of a hindrance than a help.

Our evolutionary framework suggests that policy interventions should focus not on simply outlawing undesirable behaviors — which is usually temporary and likely to provoke a backlash — but rather on creating the environment and incentives that will move us toward collective intelligence.

These policies would be geared toward preventing negative feedback loops from emerging, such as by fostering greater interaction among children with diverse backgrounds to reduce inaccurate perceptions, including group stereotypes, and by providing better access to educational and career opportunities for underrepresented groups.

The widespread use of social media coupled with aggressive recommender systems can promote divisive rhetoric. Users are algorithmically shuttled into precision-crafted echo chambers, which only serve to magnify prejudices.

The pandemic has vastly accelerated the adoption of collective, online decision-making across geographic locations. Consequently, it’s now more important than ever to make sure we have the right tools and environment to encourage the natural emergence of the wisdom of crowds and forestall the madness of mobs.

Andrew W. Lo is director of the Laboratory for Financial Engineering at MIT Sloan. He is the co-author, with Ruixun Zhang, of “The Wisdom of Crowds Versus the Madness of Mobs: An Evolutionary Model of Bias, Polarization, and Other Challenges to Collective Intelligence,” published in the new academic journal Collective Intelligence.