As mentioned earlier, the process of evaluating web content is one of elimination. In a world where information is plentiful, we can be a bit demanding about what counts as evidence. When it comes to research, one would expect that published academic research cited for a claim comes from respected peer-reviewed journals.
Consider this journal:
We ask ourselves, “Is it a journal that gives authority to this article?” Or “Is it just another web-based paper mill?”
Our first check is to find the journal’s “impact factor” (IF). The impact factor is a measure of the frequency with which the average article in a journal has been cited in a particular year. It is used to measure the importance or rank of a journal by calculating the times its articles are cited.
How Impact Factor is Calculated?
The calculation is based on a two-year period, and involves dividing the number of times articles were cited by the number of articles that are citable.
While a flawed metric for assessing the relative importance of journals, impact factor is a useful tool for quickly identifying journals that are not part of a known circle of academic discourse, or that are not peer-reviewed.
When we search Google for PLOS Medicine impact factor, we retrieve a knowledge panel with an impact factor of 13.585 (see below).
Impact factors can go into the 30s, but we’re using this as a quick elimination test, not a ranking, so we’re happy with anything over 1. We still have work to do on this article, but it’s worth keeping in the mix.
What about this one?
When we look for the impact factor for the Journal of Obesity and Weight-loss Medication, we get a result with a link to this journal at the top of the result list, but no panel, as there is no registered impact factor for this journal.
The article may be excellent but we do not know for sure. Likewise, there are occasionally articles published in the most prestigious journals that are pure junk. Be careful in your use of impact factor; a journal with an impact factor of 10 is not necessarily better than a journal with an impact factor of 3, especially if you are dealing with a niche subject.
But after a quick analysis, the PLOS Medicine article seems more trustworthy than the Journal of Obesity and Weight-loss Medication article. In fact, if you were deciding whether to re-share a story in your feed and the evidence for the story came from the Obesity journal, we would recommend skipping this source.