Skip to Main Content

Grey Literature Guide: Evaluating Sources

Evaluating Grey Literature

When using grey literature in your research, evaluating its quality and reliability is crucial. Unlike commercially published sources, grey literature does not typically undergo the rigorous peer-review process of academic journal articles. As a result, it requires careful scrutiny to ensure the information is credible, accurate, and relevant to your research.

In this section, we’ll explore how to apply the AACODS checklist and other strategies for evaluating grey literature effectively. These steps will help you ensure that the sources you use are robust and contribute meaningfully to your research.

Bias in Grey Literature

Although grey literature is essential to help avoid publication bias, it also presents a challenge as many sources are produced by organisations or individuals with specific objectives or agendas. Understanding and addressing bias is crucial for ensuring the reliability and integrity of your research.

Bias in grey literature can take many forms, including:

  • Author or Organisational Perspective: Reports produced by advocacy groups, corporations, or political entities may reflect the values and priorities of their creators.
  • Selective Reporting: Some sources may omit conflicting data or emphasise findings that support a particular agenda.
  • Funding Influences: Publications funded by interested parties (e.g., industry reports) may present a unbalanced perspective.

While it’s not always possible to eliminate bias, you can reduce its impact by:

  • Cross-Checking Information: Compare the source with peer-reviewed articles, other grey literature, or primary data to validate its claims.
  • Using Multiple Sources: Gather evidence from diverse organisations or authors to ensure a balanced understanding of the topic.

How to evaluate content?

Unlike commercially published sources, grey literature does not go through the same level of scrutiny as an article in a peer-reviewed journal would so it's particularly important to critically evaluate grey literature sources.

The AACODS checklist, developed by Jessica Tyndall from the Flinders University Medical Library in her 2008 article, was specifically designed as an evaluation tool for all different types of grey literature. 

It uses five criteria to guide you in questioning the source to determine it's quality and reliability:

Authority - Who is the author(s)? What are their credentials? Are they an expert in their field? (use Google Scholar as quick check to see if they’ve been cited by others) If there isn’t an author listed, then look at the organisation's “mission statement” or “about us” page to assess their impartiality.

Accuracy - What information is being given? Is it opinion, commentary, or factual? Has the source been reviewed by experts in some way? Does the item have a detailed reference list or bibliography to back up its claims?

Coverage - Does the source explain what it does and does not cover/consider? Are any limits clearly stated?

Objectivity - Does the work seem to be balanced in presentation? Is the author’s standpoint clear? Is the source biased in any way? Does it acknowledge it's own bias?

Date - When was the information written or published – is it up-to-date? When was the website last updated? (Sources with no date information should be treated with caution). Check the bibliography for relevant contemporary material.

Significance - Why does this source matter and what does it add to the research? What new information or angle does it offer you in terms of your research question? How is it relevant and important?

Asessing the Quality of Grey Literature

Image credit to the University of Exeter Library for the AACODS checklist.

The AACODS checklist, developed by Jessica Tyndall at Flinders University, provides a structured approach to evaluating grey literature across different formats, such as reports, policy documents, and conference papers.