The previous PG Tips tutorials demonstrated how to access and search for information for your research.
This session covers questions to ask about the information resources you discover, in order to ensure you are building on high quality scholarly practice.
For any piece of research, it is vital to evaluate the sources of information we find.
For more information and guidance on these topics do see our other PG Tips video tutorials as well as the Academic Skills module page on Canvas for further resources.
For your academic work to meet a high standard you need to be able to substantiate your argument with trustworthy, high-quality sources of information and evidence.
With so much information at our fingertips online, it is easy to become overwhelmed by quantity and neglect the quality of what we find.
When you start your own research it’s important to evaluate the information sources you find for credibility, reliability, and impartiality.
If the sources you use to support your work are unreliable or unsuitable then the work you produce will inevitably suffer. Poorly chosen evidence will weaken your arguments and could even discredit any conclusions you reach.
The best quality research builds on other high quality research.
You can avoid poor quality sources of information and research using a simple way to evaluate the quality of the evidence you have found to determine whether it passes the “CRAAP test”!
CRAAP is an acronym for each step of the process of evaluating a source according to its currency, relevance, authority, accuracy and purpose.
Currency relates to the timeliness of the resources, or the date when a source was created.
So here you are looking for the date when the information source was created, published or posted. The date of publication may or may not matter, but it’s important to know if you need something current.
Ask yourself: has it been revised or updated; whether it is important to have current information, or would older sources work as well? And if it is an online resource, are all the links within it working or have they expired?
Relevance relates to the importance of the information to you.
Relevance is probably the criteria you’ll think to look for first in your information search, as you check whether or how the information relates to or answers your research question.
So when looking for relevance, ask yourself:
Authority relates to the author, publisher or source of the information
Most articles, reports and books should include the credentials of their author – this could include their professional position or expertise, the institution or department where they work, and in some cases their contact details. So when checking the authority of a source, ask yourself:
Accuracy relates to the reliability of the resource.
Here you are checking where the information comes from. You want to make sure that the work you are pulling information from is legitimate and that any claims that works make are supported by valid evidence. When evaluating the accuracy of a source, ask yourself:
Purpose relates to the reason why the information exists and what its intended audience is.
When evaluating the purpose of the information, try to establish whether it is intended, for example, to inform, to argue, teach, sell, entertain or persuade.
Also ask critical questions of the information such as:
This takes us on to the next section of the tutorial, which looks at how to ensure you can avoid biased or misleading sources.
So alongside checking your sources for their currency, relevance, authority, accuracy and purpose, evaluating sources of information and research should also take into account issues of bias, filter bubbles and a growing landscape of predatory publishing. This next section of the tutorial will offer some tips to guard against these factors which may skew your ability to be objective or reliably informed.
When searching for information for your research, it can be easy to get stuck in a "filter bubble" where you only find online results that echo your viewpoint.
The term “filter bubble”, coined by internet activist Eli Pariser, refers to the results of the algorithms that dictate or direct what we encounter online when we browse or seek information, which can thereby ‘filter’ or narrow what we see or find according to our own information behaviour and user data.
Pariser argues that:
“The filter bubble tends to dramatically amplify confirmation bias—in a way, it’s designed to. Consuming information that conforms to our ideas of the world is easy and pleasurable; consuming information that challenges us to think in new ways or question our assumptions is frustrating and difficult.”
By applying the critical evaluation skills covered in this session, you can avoid the confirmation bias enhanced by the filter bubble effect. To get the most variety of resources, you want to ensure that you break out of these bubbles and find different opinions and viewpoints.
Check out our Essential Search Skills PG Tips video tutorial to learn ways to broaden and improve your search strategies.
Peer review is a rigorous editorial process that ensures articles submitted to an academic journal are fact-checked and reviewed for credibility by several authoritative experts in that subject. The reviewers decide if the article should be published, and may make suggestions of change before publishing. Peer-review is a rigorous vetting process.
For example, a Professor at University of Sussex had his 8,000 word paper peer-reviewed upon submission to a scholarly journal, which resulted in 3,000 words worth of comments from four reviewers, one sub-editor, and one editor. Consequently, peer-reviewed articles are often held in higher regard than those which aren’t.
Open Access sources are publications that have been made freely available to all so that anyone can benefit from reading and using research.
The rise of open access publishing has changed the ways scholars share and use journal articles. You need to be extra vigilant when evaluating what appear initially to be scholarly journals.
Many academic journals you find through Library Search have gone through a peer-review process.
You can apply a peer-review filter to your search in Library Search, to make it easier to access these texts.
You can do the same for Open Access content.
Journals available through Library Search and your Subject Guides have gone through a rigorous vetting process, and should be peer-reviewed legitimate scholarly journals. However, if you are searching Google Scholar or another search engine, and find an open access article you are unsure about you can quickly and easily check their name in the directory of open access journals via doaj.org
The rise of open access publishing has resulted in an increase of “predatory” or “vanity” publishing. Opportunistic publishing houses publish content in exchange of publication fees, paid for by the authors. These predatory publishers don’t provide any of the editorial and publishing services associated with legitimate academic journals, such as peer-review vetting process. As a result, there is a much higher number of unreliable, unvetted publications published and circulated online.
As a recent Times Higher Education article warns:
“In this new phase of predatory publishing, new markets will likely be created to absorb millions of rejected papers that could go on to be accepted by some other publishing venue; barriers to entry may be lowered; strict rules loosened; or no academic screening (in the form of peer review) might occur. Ultimately, the motivations won’t change: the unscrupulous objective of predatory publishing is to extract intellect and fees while authors might engage in questionable research practices in an effort to gain professional clout.”
You can avoid the tricks and traps of predatory publishing if you take a critically evaluative approach to examining your sources of information and research, by following the steps covered in this tutorial.
We are nearing the end of this PG Tips tutorial and we will finish with information about further library support services that are available to you.
The first of which is Skills Hub - an online academic skills portal supporting your studies, which you can get to via this tile on the Library homepage
Skills Hub is an online academic skills portal supporting your studies. You’ll find support and guidance on writing, researching, referencing, skills workshops, and how to book 1-2-1 support, which you can do via the Workshops and Tutorials page
You can book a 1-2-1 session with the Library, for help and training with your research skills. These can be requested by clicking on Workshops and Tutorials, and the Research Support 1-2-1 link, which you will find on the left hand side menu. To request, simply submit the request form. While the Library building remains closed as part of the national measures to prevent the spread of Covid-19, we are offering 1-2-1 support in online format, either via a Zoom call, or via email.
If you have any other queries, you can contact the Library either by email, or chat live with a member of staff during the weekdays, with our live chat option – look out for the yellow button on our webpages.
Thank you for watching this tutorial. Your feedback is very much appreciated – please take a moment to fill in our short feedback form – the link is on the next slide
For this activity we will work through the CRAAP test on the book or article that you are currently reading. These questions are not always straightforward to answer and will require you to critically engage with and reflect upon the piece that you have chosen to evaluate.
Find the publication date of this resource. To your knowledge, have there been any major developments in this field since it was published or has a more recent edition been written?
Tip: you can use the "cited by" filters inScopus or Web of Science to find subsequent research that has referenced this item. If it is a book, you can also use Library Search to look for more recent editions.
Spend a few minutes reflecting on how appropriate this item is to include in a postgraduate assignment. Is it an academic journal article or a Wikipedia entry? Is it an introductory textbook or a detailed monograph? Would you expect to see an item like this on your reading list?
Find a piece of information about the author from another source (not the book description or website bio). Can you find the organisation at which the author works? Using Library Search, can you also find more research from this author on similar or related topics?
Tip: you can use the Google site search to look at university webpages in the UK where the name of the author appears. Simply type their name into Google followed by site:.ac.uk
Identify the first citation in the piece of work that you are reading (if there are no citations this may be a warning sign). From the corresponding reference, find the piece of research that the author has cited. Ask yourself, does it come from a legitimate source (using the CRAAP test) and has it been reflected accurately in what you are reading i.e. check that the author is not ignoring important context.
We have looked at the author but now we will consider the publisher to better understand why this may have been written or published in this particular place. Identify the ‘publisher’ of the item you are reading; is it an academic press, a website, a predatory journal? See if you can find more information about this publisher; do they have a website or can you find other materials they have published? Do they appear to have any biases or agendas?
Full details coming soon!