Library PG Tips

Introduction

The previous PG Tips tutorials demonstrated how to access and search for information for your research.

This session covers questions to ask about the information resources you discover, in order to ensure you are building on high quality scholarly practice.

For any piece of research, it is vital to evaluate the sources of information we find.

What this session will cover:

• Why we need to evaluate our sources of information and research
• How to assess the quality and credibility of sources
• How to avoid bias, bubbles and predatory publishers
• Where to find further support and guidance

What this session will not session will not cover:

• How to develop a search strategy
• How to write your literature review, or
• How to keep up to date with sources of information and research

For more information and guidance on these topics do see our other PG Tips video tutorials as well as the Academic Skills module page on Canvas for further resources.

So why do we need to evaluate our sources?

For your academic work to meet a high standard you need to be able to substantiate your argument with trustworthy, high-quality sources of information and evidence.

Evaluating your sources enables you to:

• Assess the currency and credibility of available evidence
• Filter out irrelevant information
• Hone a discerning eye for quality, detail and bias
• Develop your critical thinking skills and
• Contribute to academic debate and understanding

With so much information at our fingertips online, it is easy to become overwhelmed by quantity and neglect the quality of what we find.

When you start your own research it’s important to evaluate the information sources you find for credibility, reliability, and impartiality.

If the sources you use to support your work are unreliable or unsuitable then the work you produce will inevitably suffer. Poorly chosen evidence will weaken your arguments and could even discredit any conclusions you reach.

The best quality research builds on other high quality research.

You can avoid poor quality sources of information and research using a simple way to evaluate the quality of the evidence you have found to determine whether it passes the “CRAAP test”!

CRAAP is an acronym for each step of the process of evaluating a source according to its currency, relevance, authority, accuracy and purpose.

C is for currency

Currency relates to the timeliness of the resources, or the date when a source was created.

So here you are looking for the date when the information source was created, published or posted. The date of publication may or may not matter, but it’s important to know if you need something current.

Ask yourself: has it been revised or updated; whether it is important to have current information, or would older sources work as well? And if it is an online resource, are all the links within it working or have they expired?

R is for Relevance

Relevance relates to the importance of the information to you.

Relevance is probably the criteria you’ll think to look for first in your information search, as you check whether or how the information relates to or answers your research question.

So when looking for relevance, ask yourself:

• Is the information at an appropriate level, i.e., not too simple or too advanced for your needs?
• Have you looked at a range of other sources? This will help you to decide whether the source you are using is relevant.
• Will your project be stronger if you include this information?
• Are you comfortable citing this as a reputable source for your research paper?

A is for Authority.

Authority relates to the author, publisher or source of the information

Most articles, reports and books should include the credentials of their author – this could include their professional position or expertise, the institution or department where they work, and in some cases their contact details. So when checking the authority of a source, ask yourself:

• Is the source trustworthy or qualified to write on the subject?
• Is there contact information and is it easy find out more about the author/organisation?
• If it is a website, what does the URL (.com .ac .gov .org .net) say about the source? 

A is also for Accuracy.

Accuracy relates to the reliability of the resource.

Here you are checking where the information comes from. You want to make sure that the work you are pulling information from is legitimate and that any claims that works make are supported by valid evidence. When evaluating the accuracy of a source, ask yourself:

• Can you verify any of the information in another reliable source?
• Does the language or tone seem balanced, unbiased and free from errors?
• Can you verify if the information or evidence is correct? Try cross-checking with another source.

And last but not least, P is for Purpose.

Purpose relates to the reason why the information exists and what its intended audience is.

When evaluating the purpose of the information, try to establish whether it is intended, for example, to inform, to argue, teach, sell, entertain or persuade.

Also ask critical questions of the information such as:

• Is the purpose clear and the point of view impartial or are there biases?
• Is the information fact, opinion, or propaganda?
• Might there be a hidden agenda?

This takes us on to the next section of the tutorial, which looks at how to ensure you can avoid biased or misleading sources.

So alongside checking your sources for their currency, relevance, authority, accuracy and purpose, evaluating sources of information and research should also take into account issues of bias, filter bubbles and a growing landscape of predatory publishing. This next section of the tutorial will offer some tips to guard against these factors which may skew your ability to be objective or reliably informed.

When searching for information for your research, it can be easy to get stuck in a "filter bubble" where you only find online results that echo your viewpoint.

The term “filter bubble”, coined by internet activist Eli Pariser, refers to the results of the algorithms that dictate or direct what we encounter online when we browse or seek information, which can thereby ‘filter’ or narrow what we see or find according to our own information behaviour and user data.

Pariser argues that: 

“The filter bubble tends to dramatically amplify confirmation bias—in a way, it’s designed to. Consuming information that conforms to our ideas of the world is easy and pleasurable; consuming information that challenges us to think in new ways or question our assumptions is frustrating and difficult.”

By applying the critical evaluation skills covered in this session, you can avoid the confirmation bias enhanced by the filter bubble effect. To get the most variety of resources, you want to ensure that you break out of these bubbles and find different opinions and viewpoints.

Check out our Essential Search Skills PG Tips video tutorial to learn ways to broaden and improve your search strategies.

Peer review is a rigorous editorial process that ensures articles submitted to an academic journal are fact-checked and reviewed for credibility by several authoritative experts in that subject. The reviewers decide if the article should be published, and may make suggestions of change before publishing. Peer-review is a rigorous vetting process.

For example, a Professor at University of Sussex had his 8,000 word paper peer-reviewed upon submission to a scholarly journal, which resulted in 3,000 words worth of comments from four reviewers, one sub-editor, and one editor. Consequently, peer-reviewed articles are often held in higher regard than those which aren’t.

Open Access sources are publications that have been made freely available to all so that anyone can benefit from reading and using research.

The rise of open access publishing has changed the ways scholars share and use journal articles. You need to be extra vigilant when evaluating what appear initially to be scholarly journals.

Many academic journals you find through Library Search have gone through a peer-review process.

You can apply a peer-review filter to your search in Library Search, to make it easier to access these texts.

You can do the same for Open Access content. 

Journals available through Library Search and your Subject Guides have gone through a rigorous vetting process, and should be peer-reviewed legitimate scholarly journals. However, if you are searching Google Scholar or another search engine, and find an open access article you are unsure about you can quickly and easily check their name in the directory of open access journals via doaj.org

The rise of open access publishing has resulted in an increase of “predatory” or “vanity” publishing. Opportunistic publishing houses publish content in exchange of publication fees, paid for by the authors. These predatory publishers don’t provide any of the editorial and publishing services associated with legitimate academic journals, such as peer-review vetting process. As a result, there is a much higher number of unreliable, unvetted publications published and circulated online.

As a recent Times Higher Education article warns:

“In this new phase of predatory publishing, new markets will likely be created to absorb millions of rejected papers that could go on to be accepted by some other publishing venue; barriers to entry may be lowered; strict rules loosened; or no academic screening (in the form of peer review) might occur. Ultimately, the motivations won’t change: the unscrupulous objective of predatory publishing is to extract intellect and fees while authors might engage in questionable research practices in an effort to gain professional clout.”

You can avoid the tricks and traps of predatory publishing if you take a critically evaluative approach to examining your sources of information and research, by following the steps covered in this tutorial.

We are nearing the end of this PG Tips tutorial and we will finish with information about further library support services that are available to you.

The first of which is Skills Hub - an online academic skills portal supporting your studies, which you can get to via this tile on the Library homepage

Skills Hub is an online academic skills portal supporting your studies. You’ll find support and guidance on writing, researching, referencing, skills workshops, and how to book 1-2-1 support, which you can do via the Workshops and Tutorials page

You can book a 1-2-1 session with the Library, for help and training with your research skills. These can be requested by clicking on Workshops and Tutorials, and the Research Support 1-2-1 link, which you will find on the left hand side menu. To request, simply submit the request form. While the Library building remains closed as part of the national measures to prevent the spread of Covid-19, we are offering 1-2-1 support in online format, either via a Zoom call, or via email.

If you have any other queries, you can contact the Library either by email, or chat live with a member of staff during the weekdays, with our live chat option – look out for the yellow button on our webpages.

Thank you for watching this tutorial. Your feedback is very much appreciated – please take a moment to fill in our short feedback form – the link is on the next slide

Bit.ly/pg-tips-feedback

Thank you

For this activity we will work through the CRAAP test on the book or article that you are currently reading. These questions are not always straightforward to answer and will require you to critically engage with and reflect upon the piece that you have chosen to evaluate.

1. Currency

Find the publication date of this resource. To your knowledge, have there been any major developments in this field since it was published or has a more recent edition been written?

2. Relevance

Spend a few minutes reflecting on how appropriate this item is to include in a postgraduate assignment. Is it an academic journal article or a Wikipedia entry? Is it an introductory textbook or a detailed monograph? Would you expect to see an item like this on your reading list?

3. Authority

Find a piece of information about the author from another source (not the book description or website bio). Can you find the organisation at which the author works? Using Library Search, can you also find more research from this author on similar or related topics?

4. Accuracy

Identify the first citation in the piece of work that you are reading (if there are no citations this may be a warning sign). From the corresponding reference, find the piece of research that the author has cited. Ask yourself, does it come from a legitimate source (using the CRAAP test) and has it been reflected accurately in what you are reading i.e. check that the author is not ignoring important context.

5. Purpose

We have looked at the author but now we will consider the publisher to better understand why this may have been written or published in this particular place. Identify the ‘publisher’ of the item you are reading; is it an academic press, a website, a predatory journal? See if you can find more information about this publisher; do they have a website or can you find other materials they have published? Do they appear to have any biases or agendas?

Further resources

​Evaluating sources - library guide 

List of online resources and readings for dissertation students

 

 

Full details coming soon!