After executing your comprehensive search strategy in various electronic databases, the next critical phase in a systematic review is selecting eligible studies for inclusion. This stage, often referred to as study screening, ensures that only studies that meet predefined standards are included in the final analysis. Study selection is vital for maintaining the transparency, integrity, and scientific rigor of a systematic review.
To conduct study screening effectively, it is essential to have clearly defined eligibility criteria. These criteria serve as the guiding framework by which studies are judged as relevant or irrelevant for inclusion.
Understanding Eligibility Criteria
Eligibility criteria are central to the study selection process. They are typically classified into two categories:
Inclusion criteria: Conditions that studies must meet to be considered eligible.
Exclusion criteria: Conditions that, if present, will automatically disqualify a study from inclusion.
These criteria should be predetermined and documented during the planning phase of the review, typically within the systematic review protocol. This level of planning enhances transparency and ensures reviewers are accountable for any post hoc modifications.
Frameworks such as PICO (Population, Intervention, Comparator, Outcome), SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type), and PEO (Population, Exposure, Outcome) are commonly used to structure inclusion criteria.
Common Elements in Eligibility Criteria
Below is a detailed table of commonly used elements in eligibility criteria, suitable for both novice and experienced reviewers:
| Element | Description | Example |
|---|---|---|
| Population | The group or participants of interest | Adults aged 18–65 with Type 2 Diabetes |
| Intervention | The treatment, exposure, or condition being studied | Use of Metformin |
| Comparator | What the intervention is being compared against | Placebo or no treatment |
| Outcomes | The effects or results of interest | Reduction in HbA1c levels |
| Study Design | The type of study methodology | Randomized Controlled Trials (RCTs), cohort studies |
| Language | The language of publication | Only studies published in English and Spanish |
| Publication Year | Time frame during which studies were published | Studies published from 2010 to 2024 |
| Setting | The context in which the study was conducted | Community health clinics, hospitals |
| Geography | The region or country of focus | United States and Canada only |
| Availability | Whether full-text is accessible | Studies with full-text available through institutional access |
| Peer Review Status | Whether the study has been peer-reviewed | Only peer-reviewed articles |
Setting these criteria at the protocol development stage ensures the review is replicable and justifiable. If changes to eligibility criteria occur later, authors must provide clear justifications.
Study Selection and the PRISMA 2020 Statement
The PRISMA 2020 Statement outlines essential reporting items for systematic reviews. The study selection process is detailed in:
Item 8 (Methods section): Requires authors to specify:
Methods used to identify eligible studies
Number of reviewers involved in selection
Whether automation tools were used
Item 16 (Results section): Instructs authors to report how many studies were screened, excluded, and included, often with a PRISMA flow diagram.
1. Methods Used to Identify Eligible Studies
After running searches in databases like PubMed, Scopus, or Embase, search results must be exported in bibliographic formats such as:
BibTeX (
.bib)CSV
EndNote XML
RIS
PubMed
.nbibformat
These exports are then imported into screening tools like Rayyan or Covidence.
Rayyan and Covidence provide structured interfaces for screening.
Always confirm which formats each tool supports:
Covidence supports only EndNote XML, RIS, and PubMed formats.
Rayyan supports a broader range: BibTeX, RIS, CSV, PubMed, and more.
If incompatible, convert formats using online tools (e.g., BibTeX to RIS), being cautious not to lose records during conversion.
A significant feature of these tools is automatic duplicate detection, using similarity indices across metadata fields. However, false positives may occur—manual verification is essential.
2. Number of Reviewers Involved
There are two main approaches:
a. Dual Independent Reviewers (Recommended)
Each reviewer screens studies independently using Rayyan or Covidence.
Both tools offer interfaces to flag conflicts (i.e., disagreements).
Discrepancies are resolved via:
Discussion and consensus
A third independent reviewer, if necessary
These tools also support:
Article labeling
Commenting
Tracking resolution history
b. Single Reviewer (Not Recommended for Systematic Reviews)
Increases risk of missing eligible studies:
Up to 15% by inexperienced reviewers
Up to 5% by experienced reviewers
Reviews conducted this way are typically classified as rapid reviews.
3. Use of Automation Tools
Automation can aid in:
Duplicate detection
Screening prioritization using AI (e.g., in Rayyan)
Semi-automated title/abstract exclusion
While helpful, automation must be used transparently, with methods clearly documented.
Title and Abstract Screening
Once duplicates are removed, screening begins with titles and abstracts. Reviewers:
Assess whether each article appears to meet inclusion criteria
Label studies as included, excluded, or maybe
Can add reasons for exclusion
Full Text Screening
After title/abstract screening, full texts of selected articles are retrieved (usually fewer than initially screened).
Studies with unretrievable full texts should be listed for transparency.
Full texts are then uploaded to Rayyan or Covidence.
Reviewers examine each article in detail to confirm:
All inclusion criteria are met
No exclusion criteria apply
The PRISMA Flow Chart
According to PRISMA 2020, study selection results should be both narratively described and presented as a flow chart. The following items should be reported:
| Stage | Detail |
|---|---|
| Records identified | Total number retrieved from all databases |
| Records imported to screening tool | Number of records uploaded into Rayyan/Covidence |
| Duplicates removed | Number removed by automatic/manual detection |
| Records screened | Total after duplicates removed |
| Records excluded | Number excluded at title/abstract level |
| Full-text articles assessed | Number of potentially eligible articles |
| Full-texts not retrieved | Number and details of unretrieved articles |
| Full-texts excluded | Number and reasons for exclusion (e.g., wrong population, poor design) |
| Studies included in review | Final number of studies that met all inclusion criteria |
Example Summary:
A total of 2,156 articles were identified through database searches. After removing 456 duplicates, 1,700 records remained. Of these, 1,450 were excluded during title and abstract screening. Full texts of 250 articles were retrieved, but 10 could not be accessed. Among the remaining 240, 180 were excluded after full-text screening. A total of 60 studies were included in the final review.
This process is summarized in the PRISMA flow diagram below:
Final Thoughts
Study selection is a vital component of any systematic review, determining the quality and relevance of the evidence base. By formulating clear eligibility criteria, using rigorous screening methods, and adhering to PRISMA guidelines, reviewers can ensure their process is transparent, reproducible, and scientifically robust. Tools like Rayyan and Covidence significantly enhance this process, especially when used collaboratively by multiple reviewers. Whether you're a novice or an experienced reviewer, adhering to these best practices ensures the integrity and reliability of your systematic review.
You may also like: