39 Treating Google’s “Snippets” with Suspicion

Occasionally when you search for an answer to a question on Google, you will not only find websites, but you may also find a “knowledge panel” that appears to have what search expert Danny Sullivan calls the “One True Answer,” an answer that appears on a knowledge panel on top of the results.

Sometimes Google pulls an answer from a source algorithmically. For example, in response to “How many men landed on the moon?,” this panel answers “12 men,” citing a Quora article.

Google search result for “how many men landed on the moon” in which a knowledge panel answers the query via Quora with “12 men.

Sometimes Google does not pull out the answer but makes the answer apparent in the blurb or headline of the card, as in this answer to the query, “last person to walk on the moon”.

Google search result for “last man to land on the moon” in which a knowledge panel pulls text from a Wikipedia article and puts the name “Cernan” in bold as the answer to the question.

This function of Google can be useful, but it malfunctions frequently enough that it should not be trusted without verifying the source and context of the answer. There are two major problems: false simplicity and false (or non-standard) information.

False Simplicity

Consider this question – how many apostles are there in the Christian tradition? Google tells you, via a panel, even pulling out the number, thereby making it look decidedly objective – there are twelve!

Google search result for “how many apostles were there” in which a knowledge panel replies “12 apostles” via Quora.

If you click through to that Quora question, you will find that it answers a much more specific and simpler question – how many original apostles did Christ have (according to tradition). And for that answer they are correct; including Judas, there are twelve.

But according to tradition, when Judas dies Matthias becomes an apostle, so that’s thirteen. Then, Paul is an apostle, so fourteen. And Barnabas, Timothy, and James. The truth is that this answer is pretty debatable. It is certainly not twelve, and some versions of the Bible refer to 25 different people as “apostles.”

These numbers, which are varied, come from various Christian traditions. Many historians, on the other hand, see the twelve apostles as a creation of the early Church, that had no reality or significance during the lifetime of the historical Jesus and was later “retrojected” into the Gospels.

The whole question of how many apostles there were and who they were is inextricably bound up with complex questions of religion, history, and 1st century power struggles about who counted in the early church and who didn’t.

Anyone who investigates religious or historical sources for an answer to this question will be confronted by a variety of  opinions. Google‘s panels, however, are oblivious to this kind of complexity and present a simple numerical answer where no simple answer actually exists.

Misleading Highlights

Google uses programming to try and highlight relevant answers in the blurb (below), but the highlighting is confusing. When asked how old Lee Harvey Oswald was when he shot Kennedy, Google highlights 18, 24, and 22.

Google search result for “how old was lee harvey oswold at the time of the assassination” in which a knowledge panel puts in bold 18, 22, and 24, which are numbers from Oswold’s date of birth, date of death, and the date of the assassination via a Wikipedia article. None are an answer to the Googled question.

In reality, the answer is 24 years old, though a quick glance at this might have you thinking 18 or 22.

Blatant Misinformation

Sometimes the panel presents blatant misinformation. Often this material is the product of highly politicized areas or of conspiracy-believing communities, which tend to rank highly in Google search results.

Take for instance this search, where we ask Google which presidents were in the Ku Klux Klan. The Google panel provides what seems to be a definitive answer. There were five!

Google search result for “Presidents in the kkk” in which a knowledge panel pulls the names of several presidents from The Trent Online.

As Case University Western history professor Peter Schulman points out, this isn’t even remotely true. None of these presidents were members of the Ku Klux Klan (as far as we know), and if you click through to the article, you’ll find the source is a Nigerian newspaper of uncertain stature that references a book by David Barton, a nationalist known for self-publishing dubious works of historical revisionism.

There are numerous examples of similar behavior. Adrianne Jefferies at The Outline details some more bad snippets, including this one claiming Obama is planning for martial law (complete fiction).

Google search result for “is obama planning martial law” in which a knowledge panel pulls a quote from newstarget.com claiming that Obama is in fact planning martial law.

Google will also tell you that Lee Harvey Oswald didn’t assassinate John F. Kennedy, despite the overwhelming evidence to the contrary:

Google search result for “why did lee harvey oswold assassinate president kennedy” in which a knowledge panel pulls text from a site claiming that Oswold did not assassinate President Kennedy.

Confirmation Bias and Bad Snippets

A lot of times Google is just bad. But sometimes bad answers are often the result of asking questions in ways that tap into the language or concerns of pseudoscience, conspiracy theory, or fringe beliefs. For example, some people have problems with monosodium glutamate, a food additive that triggers an allergic reaction in a small portion of the population. If you search on a phrase likely to be found in the medical literature like “msg sensitivity,” you get a fairly reliable result. Healthline is a recognized provider of reliable health information.

Google search result for “msg sensitvity” in which a knowledge panel pulls a list of symptoms from Healthline.

This changes if you use the language of fringe groups that believe the medical community is suppressing a link between MSG and a variety of neurological disorders. Note the results you get when you type in ‘msg dangers’:

Google search result for “msg dangers” in which a knowledge panel brings up Mercola, which claims that msg causes brain damage, such as Alzheimer’s disease and learning disabilities.

The blurb contains words such as brain damage, Alzheimer’s disease, and learning disabilities. If you look up the site, mercola.com, you will find that it is run by a physician who has been repeatedly warned by the FDA  to stop making false claims.

Our Advice

In general, simply treat the Google panel (“one true answer”) as you would any other top search result. Despite Google‘s claims to the contrary, it is not significantly more or less reliable than an average source. Click through, trace the claims on the page to a source, and investigate the source. Never trust the result without validating the source of the claim.

 

 

 

License

Icon for the Creative Commons Attribution 4.0 International License

Web Literacy for College Students 2nd Ed Copyright © 2020 by NSCC and Michael A. Caulfield is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

Share This Book