CallingBullshit_Chap.10.pdf

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 1/15

CHAPTER 10

Spotting Bullshit

I N SEPTEMBER 2017, A SHOCKING photograph made the rounds on social media. Set in the
locker room of the Seattle Seahawks football team, the picture appeared to show former
Seahawks defensive lineman Michael Bennett bare-chested and brandishing a burning
American flag. Circled around Bennett and cheering jubilantly were his Seahawks teammates
and coaching staff.

This photograph was, of course, a fake. No Seahawk ever burned an American flag during
a team meeting—or any other time. The photo (minus the burning flag) had been taken nearly
two years earlier, when the Seahawks were celebrating a crucial victory against the rival
Arizona Cardinals. But it spoke volumes about an ongoing cultural battle that had swept up
the National Football League. San Francisco quarterback Colin Kaepernick led a growing
number of players across the league in taking a knee during the national anthem as a protest
against police brutality. Donald Trump branded these players as unpatriotic, antimilitary, and
altogether un-American. The image of Michael Bennett with the burning flag, first posted to
the Facebook page of an organization called “Vets for Trump,” provided an extreme
expression of Trump’s narrative. Many viewers overlooked the low-quality image
manipulations and shared the photograph, with angry invectives directed at the Seahawks
players. The anger and disgust that many felt in seeing the image likely overshadowed any
inclination to critically evaluate its authenticity and primed them to fall for the bullshit.

If bullshit is everywhere, how can we avoid being taken in? We think it is crucial to
cultivate appropriate habits of mind. After all, our habits of mind keep us safe on a daily basis.
We don’t think about it, necessarily, but as we drive to work, our eyes are scanning for a
driver about to run a red light. Walking alone at night, we are aware of our surroundings and
alert for signs of danger. Spotting bullshit is the same. It takes continual practice, but with
that practice one becomes adept at spotting misleading arguments and analysis. While
developing a rigorous bullshit detector is a lifelong project, one can go a long way with a few
simple tricks that we will introduce in this chapter.

1. QUESTION THE SOURCE OF INFORMATION

J ournalists are trained to ask the following simple questions about any piece of information
they encounter:

Who is telling me this?

How does he or she know it?

242

243

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 2/15

What is this person trying to sell me?

These questions are second nature to us under some circumstances. When you walk into a
used-car dealership and the salesman starts talking about how the car in the corner of the lot
had only a single owner, a little old lady who drove it once a week to church on Sunday, you
are, of course, thinking this way: Who is this person? A used-car salesman! How does he
know this? Well, maybe he heard it straight from the little old lady herself. Or maybe, he
heard it from the dealer across town who sold him the car. Or, just maybe, there never was a
little old lady to begin with. What’s he trying to sell you? That one’s obvious. The 2002
Pontiac Aztek you made the mistake of glancing toward as you walked on the lot.

When we scan through our social media feeds, or listen to the evening news, or read the
latest magazine page about how to improve our health, we need to ask the same questions.

In the process of writing this chapter, we read online that crystals “retain all the
information they have ever been exposed to. Crystals absorb information—whether a severe
weather pattern, or the experience of an ancient ceremony—and pass it to anyone that comes
into contact with them.” Now this doesn’t even remotely jibe with our understanding of
physics, so it’s worth asking ourselves these three questions about this claim.

Answering the first question is relatively straightforward. Who is telling us this? This text
comes from an interview about healing crystals from the website of the lifestyle brand Goop.
The interviewee is Colleen McCann, a “fashion stylist turned energy practitioner” and
“certified shamanic energy medicine practitioner” who “utilizes a combination of crystals,
color theory, chakra systems, astrology, naturopathy, and Feng Shui principles.”

Answering the second question, “How does she know it?,” can be harder to ascertain. In
this case, however, McCann’s website gives us enough material to make some educated
guesses. In her bio, we learn that McCann “started hearing voices, also known as
Clairaudience, in a Brooklyn bodega” and that the “reputable Manhattan psychic” she
subsequently consulted “gently broke the mystical bomb to her that she too was psychic. The
voices were in fact her Spirit Guides saying hello.” She then “jumped heels first into the
crystal-laden rabbit hole with three years of private mentorship with an Intuitive Healer. Her
journey then led her to train at The Four Winds Society in Peruvian Shamanic Studies.”
Finally, we learn that she “has spent a decade studying with a Buddhist Feng Shui master to
learn Crystal Healing and Space Clearing.” Perhaps it was through these experiences, or
others like them, that McCann picked up the information that she shared in this interview.

And the third question, “What are they trying to sell us?” Here again we have to guess a
little bit, but only a little bit. The Goop company and the interviewee may be selling slightly
different things, of course. McCann may be aiming to sell us on a set of ideas or a philosophy.
In addition, and perhaps not coincidentally, her website reveals that she also sells crystals and
provides services including “intuitive crystal readings,” “crystal gridding,” and “crystal
luxtration.” The Goop company, for their part, might argue that they are promoting a lifestyle.
But they also sell a so-called Goop Medicine Bag, wherein for eighty-five dollars one receives a
set of eight “chakra-healing crystals.” To us these seem nearly indistinguishable from the
polished gems and such that you would get for five dollars a bag at tourist shops—but think
again. The Goop stones have been “cleansed with sage, tuned with sound waves, activated
with mantras, and blessed with Reiki.”

244

245

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 3/15

In short, people may be trying to sell you used cars or life insurance or beauty treatments
—or they be trying to sell you ideas, viewpoints, and perspectives. Some sales jobs get you to
part with your hard-earned money. Other sales jobs convince you to believe something that
you didn’t believe before, or to do something you wouldn’t have done otherwise. Everyone is
trying to sell you something; it is just a matter of figuring out what.

We could also ask these questions about the photograph of Michael Bennett with a
burning flag. Who is telling me this? The Facebook group called Vets for Trump. How do they
know it? Since the photo appeared only on Facebook and was not reported in any traditional
media outlet, the only possible story would be that someone had a camera in a locker room,
but somehow the media was either not present or they all agreed not to report on what had
happened—and none of the Seahawks players or staff spoke up about it afterward. That seems
highly implausible. What are they trying to sell us? They want to convince us that the NFL
players who are protesting racial injustice hold anti-American sentiments and may be a threat
to the country. Even without the giveaway signs of a poor photoshopping job, the answers to
these three questions should be enough to make us question the authenticity of such a
shocking and unexpected picture.

2. BEWARE OF UNFAIR COMPARISONS

“A irport Security Trays Carry More Germs Than Toilets!” Media outlets around the world
ran some version of this headline after a research study was published in September 2018,
confirming the fears of every germophobe who has ever suffered through the airport security
screening process.

But the claim is somewhat disingenuous. The scientists who did this study looked only at
respiratory viruses, the kind transmitted through the air or through droplets on people’s
hands when they cough or sneeze. It isn’t surprising that security trays have more respiratory
viruses than toilet seats. People don’t usually cough or sneeze onto toilet seats, nor do they
tend to touch them extensively with their hands. Toilet seats have plenty of germs, just not of
the kinds that the researchers were tallying.

Airline trays may be important vectors for colds and flus, but when the headlines bring
toilet seats into the picture, they are making an unfair comparison for shock value. Trays
don’t have more germs than toilet seats, they just have more germs of the type likely to land
on trays.

Let’s look at another example. People have always enjoyed ranked lists. In a clickstream
economy, where advertising revenues depend on page views, they’re gold. A single top-ten list
can generate ten page views per reader by putting each list item on a separate page. Farewell
Casey Kasem, hello “12 Reasons Why Sam, the Cat with Eyebrows, Should Be Your New
Favorite Cat.”

One form of list that continuously reappears is some variant of “America’s Most
Dangerous Cities.” Recently we came across such a list, released by financial news outlet 24/7
Wall St. and based on a compilation by the FBI. At the top of the list were

1. St. Louis, MO
2. Detroit, MI
3. Birmingham, AL
4. Memphis, TN
5. Milwaukee, WI

246

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 4/15

Well, that got personal quickly. Carl was born in St. Louis; he spent much of his later teens
exploring the neighborhoods of Detroit. Are these cities really all that bad? he wondered. The
rankings are based on hard numbers from a trusted government agency. But is this really a
fair comparison? Or is there something funny going on that sends St. Louis and Detroit to the
top of the list? The first question we could ask is how they have quantified how dangerous a
city is. Biggest potholes? Most bedbugs? Greatest number of golfers hit by lightning?

In this case, the metric of danger is the number of violent crimes per capita. We might try
to argue that this measure does not do a good job of characterizing how dangerous a city is.
Perhaps violent crime reporting is accurate in St. Louis and Detroit, but such events are
severely underreported in other places. Perhaps St. Louis and Detroit have high assault rates
but few murders. Or perhaps the data used to compute violent crimes per capita
underestimate the populations of St. Louis and Detroit due to recent population growth.

A more likely problem is that there is something arbitrary in how cities are defined. City
limits are political boundaries. Some cities include primarily a central urban region and
exclude the outlying suburbs. Others encompass the majority of the surrounding
metropolitan area. This could make a huge difference when tallying violent crime rates. For
complex reasons, crime rates within many US cities tend to be high in the historical urban
core of a city and lower in the suburbs.

Why does this matter? It matters because the crime rate in a city will depend on how
tightly the city boundary encircles the urban core. And because the locations of city
boundaries depend on a city’s history and politics, we see a great deal of variation in how
tightly the boundaries circumscribe the city center. At this point we have a hypothesis—that
city boundaries have a substantial effect on the violent crime rate—but no hard evidence. We
have some grounds for skepticism about top ten most-dangerous lists,*1 but if we hope to
argue that the way city limits are defined impacts the results, we need to collect additional
data and test this hypothesis directly. Violent crime data are readily available, as are
population data. But how do we control for whether the city boundaries include the suburbs?

The US government compiles a list of metropolitan statistical areas and assembles
statistical and demographic data about each. Each metropolitan area comprises a core city or
multiple core cities, surrounded by outlying suburbs. If differences in violent crime rates are
influenced by differences in the way city boundaries are drawn, we would expect cities that
are small compared to the surrounding metropolitan areas to have higher crime rates, on
average, than cities that are big compared to their surrounding metropolitan areas.

In the scatter plot below, each dot represents one city. On the vertical axis, we show the
violent crime rate (measured as the number of violent crimes reported per 100,000 people
per year). On the horizontal axis we show the fraction of the metro’s population that resides
within its major city.*2 This gives us a measure of how narrow or expansive the city’s
boundaries are.

247

248

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 5/15

As we suspected, the higher the fraction of the metropolitan area that lies within the city
limits, the lower the violent crime rate tends to be. Cities that have narrow boundaries tend to
have higher crime rates, and cities with expansive boundaries tend to have lower crime rates.
If we fit a line representing crime rates through the points, that line slopes downward.
Statistical tests show that this slope is likely to be meaningful rather than merely the result of
chance.*3 Thus there is a correlation between the violent crime rate in a city and the way the
city boundaries are drawn. The overall amount of crime in a metropolitan area influences
whether a city appears dangerous or safe—but so does the way in which the city’s boundaries
are drawn. People are making apples-to-oranges comparisons when comparing cities such as
St. Louis or Detroit, which include only the urban core, with cities such as Anchorage or
Laredo, which include the suburbs as well.

This example of violent crime rates serves to illustrate a more general principle: Ranked
lists are meaningful only if the entities being compared are directly comparable.

3. IF IT SEEMS TOO GOOD OR TOO BAD TO BE TRUE…

E arly in 2017, the Trump administration instituted a set of policies restricting travel and
immigration to the United States. Trump’s policies influenced many aspects of American life,
including higher education. In March 2017, NBC News sent out a message on Twitter about
the consequences of these policy changes: “International student applications are down
nearly 40 percent, survey shows.”

The tweet linked to a news story and was widely shared on Twitter. But the claim it put
forth seems implausible. Sure, Trump’s travel ban and associated changes to US immigration
policy were unlikely to make the US seem more welcoming to international students. But a
catastrophic 40 percent drop in applications struck us as too large to be real. Not only is the
size of the effect massive, its timing is suspect. Applications to many US universities would
have been due in December or January, before Trump took office. We were skeptical.

Our skepticism follows from a general principle for spotting bullshit: If a claim seems too
good—or too bad—to be true, it probably is. We all apply this rule of thumb on a regular basis
in our daily lives. How many people actually think they’ve won a free vacation when they get
the robocall from a telephone solicitor?

249

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 6/15

So how do we figure out whether the 40 percent statistic that NBC provides is actually
bullshit? Simple: Dig to the source. Don’t trust the tweet. In a world dominated by social
media where any information we receive has already been rewritten, rearranged, and
reprocessed, it’s important to cultivate the habit of digging to the source.

The NBC tweet provided a link to its source, an NBC Nightly News story titled “Survey
Finds Foreign Students Aren’t Applying to American Colleges.” This story explains that
foreign applications are down at a number of schools, and attributes this decline to Trump’s
travel ban and anti-immigration rhetoric:

Applications from international students from countries such as China, India and
in particular, the Middle East, are down this year at nearly 40 percent of schools that
answered a recent survey by the American Association of Collegiate Registrars and
Admissions Officers.

Educators, recruiters and school officials report that the perception of America has
changed for international students, and it just doesn’t seem to be as welcoming a place
anymore. Officials point to the Trump administration’s rhetoric surrounding
immigration and the issuing of a travel ban as having an effect.

But hold on! This article is telling a different story than the tweet suggested. The tweet
claimed applications were down by 40 percent. The story suggests that applications are down
at 40 percent of schools. That’s a different matter altogether. Applications could be down just
a small percentage at those schools, for example, resulting in a very small overall drop in
foreign applications. Already we’ve found a discrepancy between the tweet and the news story
it advertises.

But which is right? The tweet or the news story? To figure that out, we have to keep
digging. The news story cites a bulletin from the American Association of Collegiate
Registrars and Admissions Officers; a bit of searching leads us to this report, posted eleven
days before the NBC story, which provides a critical detail.*4 Yes, international applications
decreased at 39 percent of universities—but they increased at 35 percent of universities.
Taken together this isn’t news, it’s statistical noise. Given the information presented in the
article, there is no meaningful indication of a “Trump effect” on international applications.
Most likely, these numbers merely reflect chance fluctuations in the number of applications to
different US schools.

So how did all of this happen? There seem to have been lapses at multiple levels. First, the
NBC article is misleading because it describes only the fraction of schools with declining
applications, and fails to mention that a comparable fraction of schools received an increasing
number of applications. We can imagine how that would have come about. A large-scale
survey reveals no evidence of systematic change in international applications in response to
Trump policies; that’s hardly an exciting news story. Either to liven up the story, or simply
because writers or editors lack quantitative sophistication, they highlight the decline in
applications at 39 percent of schools and ignore or downplay the increase in applications at 35
percent of schools. Here is an example of how a statement can be true and still qualify as
bullshit. It is true that applications declined at 39 percent of schools. But the absence of
context is bound to mislead the reader.

250

251

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 7/15

The new story was then presumably misinterpreted by whoever runs the social media feed
for NBC, and a decline at 40 percent of schools was transposed into a decline by 40 percent.
This improbably large effect is where the “if it sounds too good or bad to be true…” rule of
thumb comes into play. We find that this third rule is particularly good for spotting bullshit
that spreads across social media. In a social media environment, the posts that are spread
most widely are often those that shock, spark a sense of wonder, or inflame feelings of
righteous indignation: namely, those that make the most extreme claims. And the most
extreme claims are often too good or too bad to be true.

4. THINK IN ORDERS OF MAGNITUDE

T hink back to philosopher Harry Frankfurt’s distinction between bullshit and lies. Lies are
designed to lead away from the truth; bullshit is produced with a gross indifference to the
truth. This definition gives us a considerable advantage when trying to spot bullshit. Well-
crafted lies will be plausible, whereas a lot of bullshit will be ridiculous even on the surface.
When people use bullshit numbers to support their arguments, they are often so far off that
we can spot the bullshit by intuition and refute it without much research.

The National Geographic Society sent out a mailer warning that plastic waste is polluting
our oceans. “9 Billion Tons of Plastic Waste End Up in the Ocean Every Year,” the headline
proclaimed. That sounds dreadful, but pause and think for a moment. There are fewer than
eight billion people on the planet. Is it really possible that each person puts an average of one
ton of plastic waste into the ocean each year? That seems unlikely. In fact, the total
production of plastic throughout all history is only about eight billion tons—and not all of
that ends up in the oceans. Clearly the figure of nine billion tons per year is in error. What is
the correct figure? National Geographic themselves recently reported that nine million tons
of plastic waste go into the ocean each year. Plastic pollution of our oceans is surely an
ecological disaster in the making—but inflating the magnitude a thousandfold doesn’t help
anyone. It merely undermines the credibility of pro-environmental sources. Not that there is
any reason to suspect that the mistake was intentional; we suspect that in the production of
the mailer, someone accidentally typed “billions” instead of “millions.”

Because the nine billion tons of plastic waste per year is readily compared with the eight
billion people on Earth, this mistake is pretty easy to catch without doing any mental math.
Often, however, one needs to do a few simple mental calculations to check a numerical claim.
For example, suppose a friend claims there are more than 121,000 men in the UK named
John Smith. Does that sound right? We think that the key to working out these kinds of
questions quickly, without even pencil and paper, is to break down a number into
components that you can estimate. The estimates can be very loose; it is usually good enough
to estimate to the nearest power of ten (sometimes called an “ of magnitude”). Here we
might ask, “How many people are there in the UK? What fraction of those are named John?
What fraction of the UK Johns have the surname Smith?”

So how many people are in the UK? About 1 million? 10 million? 100 million? 1 billion?
Most of us know that 100 million is the best estimate from among these (the true value in
2018 was about two-thirds of that, 67 million).

How many of those people have the first name John? One in ten? Well, since few women
are named John, that would require that about one in five men be named John. Not even
close. (Remarkably, until around 1800 one in five men in England was named John—but that
isn’t true today.) One in a thousand? Clearly the name John is a lot more common than that.
One in a hundred sounds about right.

252

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 8/15

How many people in the UK have the last name Smith? Again, one in ten seems too many
and one in a thousand seems too few, so one in a hundred makes for a good guess.

So how many John Smiths do we expect in the UK? To make the calculation easy we will
assume that people with the surname Smith are just as likely as anyone else to have the given
name John, though in practice this is probably not quite right. But we are only
approximating; it should be fine to make this assumption. Therefore we have roughly 100
million people in the UK, of whom one in a hundred are named John. This gives us 1 million
Johns. Of these, we estimate that one in a hundred is named Smith, giving us an estimate of
ten thousand John Smiths in the UK.

This estimate turns out to be pretty good. In practice, there are about 4,700 people named
John Smith living in the UK today. If we’d used the actual UK population of 67 million, we’d
have had an even closer estimate of 6,700. But either way, we can see that our friend’s claim
of 121,000 John Smiths is off by a factor of ten or more.

This process of making back-of-the-envelope approximations is known as Fermi
estimation, after the physicist Enrico Fermi, who was famous for estimating the size of an
atomic blast using these simple methods.*5 For spotting bullshit on the fly, we suggest that
making these approximations in powers of ten is often good enough. Freeing yourself up to be
within a power of ten in your estimates encourages you to think through a problem quickly,
using information you already know, instead of getting hung up on calculating the number of
seconds in a fortnight (1,209,600 seconds) or using a search engine to find how many gallons
of water the average New Yorker uses in a day (115 gallons). Even if you’re off by 50 percent
here and there, your final estimate is very likely to be within tenfold of the true value, which is
often sufficient to spot bullshit. Of course, if your estimate leads you to believe a figure is
nonsense, and you want to be sure, you can always look up the true numbers or make a more
accurate estimate using pen and paper.

At a May 2018 hearing of the US House Committee on Science, Space and Technology,
Representative Mo Brooks (R-Ala.) speculated that perhaps rising sea levels could be
attributed to rocks falling into the ocean. For example, he asked his constituents to consider
the continuously eroding White Cliffs of Dover. These have to be filling up the ocean over
time, and all the water that they displace must be going somewhere. It is comforting that like
Aesop’s crow,*6 Representative Brooks understands the consequences of putting rocks in
water. But this is an entirely inadequate explanation that belies Representative Brooks’s
ability to grasp how vast the oceans are.

The oceans take up roughly two-thirds of the earth’s surface and run to an average depth
of about two miles. That is an enormous amount of water, spread over an almost
unimaginably large area. Given that, how much of an effect could collapsing cliffs have?

We can work this out in straightforward fashion. Imagine that tomorrow the entire White
Cliffs of Dover, and all the land for a kilometer inland, fell into the sea in a cataclysmic
collapse and displaced an equivalent volume of seawater. Setting aside the gargantuan
tsunami that would savage Calais and the northern coast of France, what would happen to sea
levels globally?

Would the rising waters flood low-lying areas of coastal cities? Hardly. We can see this
with a simple calculation. The White Cliffs are a bit over 10 kilometers in length and roughly
100 meters high. So in our imagined collapse, we have 10 km × 1 km × 100 m = 1 billion cubic

253

254

6/27/2021 Chapter 10: Spotting Bullshit, Calling Bullshit

https://web-b-ebscohost-com.ezproxy.losrios.edu/ehost/ebookviewer/ebook/bmxlYmtfXzIyOTMxMDlfX0FO0?sid=3be3e61a-003b-4171-9b17-ba1a46… 9/15

meters. Wow! It would take the Crayola company about two million years to package that
much chalk.*7

But the Earth’s oceans are all connected, so this billion cubic meters of land would have to
raise the water level of all the world’s oceans. The surface area of these oceans is
approximately 360 million square kilometers, or 360 trillion square meters. (We care about
the surface area of the oceans, not the volume, because it is the surface that is raised.) So the 1
billion cubic meters of water displacement would be spread across 360 trillion square meters.
The ensuing sea level rise would be 1,000,000,000 m3 / 360,000,000,000,000 m2 =
0.000003 m. In other words, we are looking at a 3-micrometer (μm) rise in sea level. By
comparison, a human hair is about 100 μm in diameter. So if the White Cliffs of Dover and
thirty other comparably sized stretches …