Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Psychology

Systematic Reviews: Introduction

What is a systematic review?

A systematic review is a review of research literature (often limited to randomised controlled trials) that is searched for in a systematic or exact manner, and only includes studies that have met strict inclusion criteria. The methodology of the review often follows a internationally recognized systematic review standard or guideline. 

PRISMA definition

"A systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review." (PRISMA, 2015)

 

How does it differ from a systematic literature review?

A systematic literature search is a literature review which demonstrates that you have compiled a list of appropriate search terms and includes the structure of your search history, which provides the evidence on which your assignment is based. You may be told that you need to conduct a systematic review when in fact you just need to perform a literature search in a systematic manner. 

This is a less rigorous process than a systematic review. A systematic review usually covers a wider scope; you would be expected to look at all the available research in the area in question, systematically searching multiple academic databases.

If you are unsure about the differences between a systematic review and a literature review take a look at this guide: What’s in a Name? The difference between a Systematic Review and a Literature Review and Why it Matters.

 

Systematic reviews: your research question

Your research question

It is essential that you have a focused, carefully considered research question before you start to build your search strategy. 

Where possible, select a research topic that you have some basic knowledge in, or are familiar with in. If this is not possible, seek guidance from your supervisor on the most relevant terminology for your chosen topic. This will make it easier to devise your research question, ensuring you are using the most appropriate and relevant phrasing.This is especially important when you come to build your search strategy, as the wording of your research question research question will frame your initial search. 

The search terms, phrases, words and subject headings you use in your systematic search play a vital role in your systematic review, because they determine the number and range of results you will get from each database. If your question is too wide-reaching so will your search, and you will be overwhelmed with  an unmanageable number of search results. You also have to think very carefully about the limitations of your search - such as whether or not you will have a specific date range or limit to a particular study design. Remember, the clue is in the name - be systematic in your approach. 

Inclusion and exclusion criteria 

Before you start to build your search strategy you will need to decide on your review's inclusion and exclusion criteria. This will have a bearing on your search strategy, and potentially determine which studies your systematic search captures. Systematic reviews are measured against a set of specific criteria outlined at the start. Your inclusion and exclusion criteria dictates which studies will be included in your systematic review (they meet all aspects of your inclusion criteria) and those that will be excluded (does not match your inclusion criteria fully, or meets the exclusion criteria). You need to think very carefully about the limitations of your research - will you limit to a specific date range or to a particular study design for example. Remember, the clue is in the name - be systematic in your approach. 

 

Example

Research question:

"For UK children under 5 years of age with a suspected diagnosis of ADHD, what are the recommended clinical assessments used for diagnosis?" 

Inclusion criteria:

  • children 0-5 years old 
  • suspected ADHD
  • limited to clinical assessments specifically, for diagnosis
  • UK population studies only

Exclusion criteria:

  • any member of the population over the age of 5 
  • animal studies
  • pre-determined diagnosis of ADHD 
  • assessments not defined as "clinical" 
  • studies of populations outside of the UK

PICO Framework

In order to structure your search strategy it can be helpful to use a structured framework.They are designed to help you formulate your research question, search strategy, and inclusion and exclusion criteria. The PICO framework is a popular framework for physiological and health sciences systematic reviews.

 

PICO stands for:

Population / Patient

Intervention 

Comparison

Outcome

 

You do not have to structure your research to fit into all of the above - it is not prescriptive, but merely a framework to guide you in building your search strategy. Your research question may not have a defined comparative intervention, for example, so you may only use PIO for your framework. 

 

PICO example

Research question: What impact does Cognitive Behavioural Therapy (CBT) versus pharmaceutical therapy have as treatments for diagnosed anxiety in children in reducing their anxiety?

Framework:

Population

Children who have been diagnosed with anxiety

Intervention

Child receives CBT

Comparison

Child receives medication/pharmaceutical therapies

Outcome

Improved/no change/worsening of anxiety


Systematic Reviews: the search

Your search strategy - where to start

student taking notes on laptop

The search terms, phrases, words and subject headings you use in your search play a vital role in your systematic review, because they determine the number and range of results you will get from each database. 

Start with your research question to pick out the keywords you will use as the basis of your search. A common practice in systematic review searches is to refer back to the framework you used to structure your research question, such as the PICO framework. This framework gives a clear picture of not only the terms you want to use in your search, but also how you will combine or connect them together in the database.

 

Using PICO framework for your search 

Research question: Is Cognitive Behavioural Therapy (CBT) more effective than pharmaceutical therapy as a treatment for diagnosed anxiety in teenagers aged 13-18 in managing symptoms of anxiety?

Framework 

Population

Teenagers aged 13-18 who have been diagnosed with anxiety

Intervention

Teen receives CBT

Comparison

Teen receives medication/pharmaceutical therapies

Outcome

Improved management of symptoms


Search terms identified: 

  • teen/teenager
  • 13-18 year olds
  • diagnosed
  • anxiety 
  • CBT/cognitive behavioural therapy
  • pharmaceutical/medication
  • Improved management of symptoms

Where to search - Databases

Your search needs to be exhaustive in order to capture all relevant literature on your research topic, which means searching more than one database or resource for your sources. Search engines like Google or Google Scholar are not optimal for systematic searches, as you cannot easily conduct an exhaustive or structured search due to the way content is indexed and the limitations of the search functionality. Instead, you will need to identify the academic databases that contain collections most relevant to your topic and search those in a systematic way. Most academic database collections are comprehensively indexed (the process of classifying an article by tagging it with a term that represents the focus/topic of the article), making the collections more discoverable when searching free text terms in defined fields and Subject Headings. 

We have grouped the key databases and online resources for psychology in the Key Resources tab. Your dissertation supervisor may also suggest or advise you on the various databases suitable to your search. Some of the most commonly searched databases include: PsycINFO, Web of Science, Cochrane, Scopus, and PubMed. The links for these can be found in the Key Resources tab. 

 

Where to search - Grey Literature

Depending on your research, you may also need to search grey literature sources in addition to academic database collections. Have a look at the grey literature tab on the left hand side more information about what defines grey literature and the sources you can search. 

Search techniques 

Consider your keywords carefully 

notice board with keywords pinned It's important to search a variety of search words as not every author will use the same words that you have used to search - nor will databases search for synonyms as standard. Use the keywords identified from your research question as a starting point and build on them, taking into account any possible synonyms, acronyms, spelling variations (eg behaviour/behavior), etc.

It's often helpful to look at the abstracts and reference lists of studies or papers you have already engaged with on your topic for a sense of what terminology and specific words/phrases are most common. 

Once you have made a comprehensive list of all the keywords you want to include in your search you need to combine them using what are known as Boolean operators or search connectors. 

 

Boolean search operators 

In order to combine multiple facets of your search together in a database you will need to use the search operators: AND, OR, NOT. Using operators is common in systematic reviews, and improves the efficiency of your search resulting in more relevant results. Watch the Boolean searching videos to learn more about how to use operators.

 

Proximity operators 

Using proximity operators in your search allows you to search for two or more words that occur within a certain number of words from each other. Proximity operators can be helpful when you don't want to limit yourself to searching an exact phrase, for example "diagnosed anxiety", but where you still want to refine your search to a degree. Each database will have its own method of proximity searching, but generally your search looks something like: diagnosed NEAR/3 anxiety - NEAR signifying near to, and 3 signifying the number of words that can appear between "diagnosed" and "anxiety". To ensure proximity searching is available in the database you are searching, have a look for their "search tips" or "search help" page. 

PRESS (Peer Review of Electronic Search Strategies)

Checklist on paperAs we've established, a comprehensive search strategy is fundamental to the success of your systematic review literature search. "The aim is to achieve comprehensiveness of coverage while maintaining a moderate degree of precision of the records retrieved." (PRESS guideline statement). As a result, it can be beneficial to quality assure your search strategy before beginning to search, mitigating any potential errors or omissions and reducing the likelihood of revision at a later date.

The PRESS checklist is a comprehensive evidence-based checklist that does just that. It covers six key aspects of search strategies: 

  • the translation of the research question into a search strategy
  • the use of any Boolean or proximity operators
  • the keywords used 
  • spelling, syntax, and commands
  • database filters / limits intended for use
  • database Subject headings intended for use not essential, and may not apply to your strategy)

Each aspect above has a set of questions to check against your search strategy. The full checklist can be downloaded here

Research question

Is Cognitive Behavioural Therapy (CBT) more effective than pharmaceutical therapy as a treatment for diagnosed anxiety in teenagers aged 13-18 in managing symptoms of anxiety?

 

Step 1: Framework

Population

Teenagers aged 13-18 who have been diagnosed with anxiety

Intervention

Teen receives CBT

Comparison

Teen receives medication/pharmaceutical therapies

Outcome

Improved/reduction /no change/ worsening of anxiety

 

Step 2: Identify search terms

Terms in question

Synonyms / spelling variations / similar terms:

teenager

adolescent, teen, youth, 13-18 year olds

anxiety

N/A

(CBT) Cognitive behavioural therapy

cognitive behavioral therapy/therapies, cognitive psychotherapies

pharmaceutical medication/medicate, prescription drugs, (also specific drug names)
managing symptoms improvement, reduction, reduce 

 

Step 3: Build search strategy with Boolean operators / commands / proximity operators

Break-down

Search strategy 

Concept 1

(teen* OR adolescent* OR "13-18 year olds" OR youth*) 

Boolean operator

AND

Concept 2

(diagnos* OR confirm*) NEAR/2 (anxiety) 

Boolean operator

AND

Concept 3

(CBT OR "cognitive behavioural therapy" OR "cognitive behavioral therapy" OR “cognitive therapy” OR cognitive therapies”) OR ("prescription drugs" OR "pharmaceutical treatment" OR medicat*) 

Boolean operator

AND

Concept 4

(improv* OR reduc* OR increas* OR manag*) NEAR/3 (symptom* OR anxiety) 

Systematic Reviews: guidelines and protocols

Standards and guidelines 

There are a number of systematic review reporting standards you can adhere to when writing your systematic review.

These can serve as guidelines for the structure of your review, from what headings/sections to include, (such as methods, search strategy, etc) to best practice protocols for search strategies, reporting, etc. 

‚ÄčThe School of Psychology recommends its students follow the PRISMA reporting guidance when conducting a systematic review. 

 

PRISMA standards

PRISM(Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is an evidence-based protocol for reporting on systematic reviews and meta-analyses. The School of Psychology recommends its students follow the PRISMA reporting guidance when conducting a systematic review. The PRISMA checklist makes this easy, providing a checklist of criteria to be adhered to for each of the sections of your review, for example: introduction, methods, results, etc.

 

Useful PRISMA links: 

  • PRISMA Checklist (checklist of sections to include in your systematic review, eg: methods, results, etc.) 
  • PRISMA flowchart (charts the number of records identified, included and excluded, and the reasons for exclusions.)
  • Citing PRISMA

Cochrane Handbook 

undefinedThe Cochrane Database of Systematic Reviews (CDSR) is the leading guideline for systematic reviews in health care, and most often described as the gold standard of systematic reviews. The handbook provides reviewers with guidance on how to conduct a systematic review, with particular emphasis on the search process. The handbook is very detailed, and you may find not all sections are applicable to your review. It is worth familiarizing yourself with the the section on formulating your search though, as this 

Cochrane Handbook for Systematic Reviews of Interventions 

 

Choosing a systematic review standard/guideline

The School of Psychology recommends its students follow the PRISMA reporting guidance when conducting a systematic review, but it is also helpful to read through the Cochrane Handbook of Systematic Review, particularly if your research topic spans both medical and human sciences.