39 Treating Google’s “Snippets” with Suspicion

Occasionally when you search for an answer to a question on Google, you will not only find websites, but you may also find a “knowledge panel” that appears to have what search expert Danny Sullivan calls the “One True Answer,” an answer that appears on a knowledge panel on top of the results.

Sometimes Google pulls an answer from a source algorithmically. For example, in response to “How many men landed on the moon?,” this panel answers “12 men,” citing a Quora article.

Sometimes Google does not pull out the answer but makes the answer apparent in the blurb or headline of the card, as in this answer to the query, “last person to walk on the moon”.

This function of Google can be useful, but it malfunctions frequently enough that it should not be trusted without verifying the source and context of the answer. There are two major problems: false simplicity and false (or non-standard) information.

False Simplicity

Consider this question – how many apostles are there in the Christian tradition? Google tells you, via a panel, even pulling out the number, thereby making it look decidedly objective – there are twelve!

If you click through to that Quora question, you will find that it answers a much more specific and simpler question – how many original apostles did Christ have (according to tradition). And for that answer they are correct; including Judas, there are twelve.

But according to tradition, when Judas dies Matthias becomes an apostle, so that’s thirteen. Then, Paul is an apostle, so fourteen. And Barnabas, Timothy, and James. The truth is that this answer is pretty debatable. It is certainly not twelve, and some versions of the Bible refer to 25 different people as “apostles.”

These numbers, which are varied, come from various Christian traditions. Many historians, on the other hand, see the twelve apostles as a creation of the early Church, that had no reality or significance during the lifetime of the historical Jesus and was later “retrojected” into the Gospels.

The whole question of how many apostles there were and who they were is inextricably bound up with complex questions of religion, history, and 1st century power struggles about who counted in the early church and who didn’t.

Anyone who investigates religious or historical sources for an answer to this question will be confronted by a variety of  opinions. Google‘s panels, however, are oblivious to this kind of complexity and present a simple numerical answer where no simple answer actually exists.

Misleading Highlights

Google uses programming to try and highlight relevant answers in the blurb (below), but the highlighting is confusing. When asked how old Lee Harvey Oswald was when he shot Kennedy, Google highlights 18, 24, and 22.

In reality, the answer is 24 years old, though a quick glance at this might have you thinking 18 or 22.

Blatant Misinformation

Sometimes the panel presents blatant misinformation. Often this material is the product of highly politicized areas or of conspiracy-believing communities, which tend to rank highly in Google search results.

Take for instance this search, where we ask Google which presidents were in the Ku Klux Klan. The Google panel provides what seems to be a definitive answer. There were five!

As Case University Western history professor Peter Schulman points out, this isn’t even remotely true. None of these presidents were members of the Ku Klux Klan (as far as we know), and if you click through to the article, you’ll find the source is a Nigerian newspaper of uncertain stature that references a book by David Barton, a nationalist known for self-publishing dubious works of historical revisionism.

There are numerous examples of similar behavior. Adrianne Jefferies at The Outline details some more bad snippets, including this one claiming Obama is planning for martial law (complete fiction).

Google will also tell you that Lee Harvey Oswald didn’t assassinate John F. Kennedy, despite the overwhelming evidence to the contrary:

Confirmation Bias and Bad Snippets

A lot of times Google is just bad. But sometimes bad answers are often the result of asking questions in ways that tap into the language or concerns of pseudoscience, conspiracy theory, or fringe beliefs. For example, some people have problems with monosodium glutamate, a food additive that triggers an allergic reaction in a small portion of the population. If you search on a phrase likely to be found in the medical literature like “msg sensitivity,” you get a fairly reliable result. Healthline is a recognized provider of reliable health information.

This changes if you use the language of fringe groups that believe the medical community is suppressing a link between MSG and a variety of neurological disorders. Note the results you get when you type in ‘msg dangers’:

The blurb contains words such as brain damage, Alzheimer’s disease, and learning disabilities. If you look up the site, mercola.com, you will find that it is run by a physician who has been repeatedly warned by the FDA  to stop making false claims.

Our Advice

In general, simply treat the Google panel (“one true answer”) as you would any other top search result. Despite Google‘s claims to the contrary, it is not significantly more or less reliable than an average source. Click through, trace the claims on the page to a source, and investigate the source. Never trust the result without validating the source of the claim.