Developing a Systematic Review Search Strategy

Introduction: Why a Good Search Strategy Matters

A high-quality systematic review is defined by its ability to comprehensively retrieve all studies that answer the same research question. However, many systematic reviews fail to meet this standard because of poorly constructed search strategies. A well-designed search strategy ensures that the review includes all relevant literature, minimizing bias and maximizing reliability.

What Is a Search Strategy?

A search strategy is a structured and reproducible plan that a researcher uses to identify all eligible studies from various databases. It includes a combination of keywords, subject headings, Boolean operators, and filters, tailored to locate the most relevant evidence. Think of a search strategy as a diagnostic test: just as a test must detect true cases of a disease without too many false positives, a search strategy must find true studies without being overwhelmed by irrelevant records.

Sensitivity and Specificity: Diagnostic Concepts in Search Strategy

The concepts of sensitivity and specificity from diagnostic testing are directly applicable to search strategies:

• Sensitivity refers to the ability of the strategy to correctly retrieve all or most eligible studies. A highly sensitive strategy minimizes the risk of missing important evidence.

• Specificity refers to the ability of the search to ignore irrelevant studies. A highly specific strategy reduces the number of unrelated articles retrieved.

Balancing these two concepts is essential. Too much sensitivity with too little specificity results in an unmanageable number of irrelevant studies, while too much specificity risks excluding important literature.

The Screening Burden: Why Balance Matters

The consequences of an imbalanced search strategy are practical and significant:

• A sensitive but non-specific strategy might retrieve 5,000 records, of which only 30 are eligible. Reviewers must then sift through thousands of irrelevant studies, which is time-consuming and resource-intensive.

• A specific but insensitive strategy might retrieve just 100 records, but only a handful are truly eligible—and many more have been missed.

Thus, striking the right balance minimizes the screening burden while preserving comprehensiveness.

Can We Measure Sensitivity and Specificity?

In theory, you could measure sensitivity and specificity by comparing retrieved studies to a gold standard list. In practice, this is rarely feasible. Systematic reviewers instead adopt an iterative process, testing and refining search terms until the results are satisfactory.

From experience, a sensitive enough strategy will yield a few eligible studies among the first 20–50 records sorted by relevance. A specific enough strategy will generally not exceed 1,000 results per database.

Most systematic reviews include between 5 and 60 eligible studies. Therefore:

• >1,000 results per database often means poor specificity.

• 400–600 results per database is typical when balance is achieved.

• 1,000–2,000 combined results from multiple databases is ideal.

Using Frameworks to Develop Search Strategies

Systematic search strategies should be structured using evidence-based frameworks. The most common include:

• PICO (Population, Intervention, Comparison, Outcome)

• SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type)

• SPICE (Setting, Perspective, Intervention, Comparison, Evaluation)

• ECLIPSE (Expectation, Client group, Location, Impact, Professionals, Service)

Example: Using PICO to Identify Search Terms

Note: For intervention reviews, it’s best not to include outcomes in the search strategy. Outcomes are often variably reported and inconsistently described across studies. Including them can reduce sensitivity by inadvertently excluding eligible articles.

Boolean Operators and Proximity Searches

Boolean operators are the backbone of combining search terms:

• AND narrows the search: both terms must be present.

• OR broadens the search: either term can be present.

• NOT excludes terms (use cautiously).

Proximity Searching

Proximity operators allow retrieval of terms within a certain distance of each other, useful for capturing phrases that vary in wording.

Examples:

• PubMed: "hip pain"[Title/Abstract:~2] retrieves "hip and lower back pain", "pain in the hip", etc. Proximity search is currently available in PubMed Labs.

• Embase: therapy NEAR/5 sleep finds results where "therapy" and "sleep" appear within 5 words, regardless of order.

• CINAHL: mobile W3 phone* retrieves terms like "mobile smartphone", "mobile and wireless phone", etc.

Search Strategies and Google Scholar

Google Scholar is not a traditional academic database and handles queries differently. Boolean operators are interpreted in the following ways:

• AND is represented by a space: mental health adolescent

• OR is represented by a vertical bar: therapy | counselling

• NOT does not function reliably and should be avoided.

Also, Google Scholar limits advanced control over proximity and truncation. Despite its limitations, it can be useful for citation chasing and identifying grey literature.

Using Subject Headings for Better Sensitivity

Most structured databases include controlled vocabularies:

• PubMed uses MeSH (Medical Subject Headings)

• Embase uses Emtree

• CINAHL uses CINAHL Headings

Using subject headings improves retrieval by capturing variations in terminology. For instance, “heart attack”, “myocardial infarction”, and “MI” all map to the MeSH term “Myocardial Infarction”.

Tips:

• Combine subject headings with keywords using OR.

• Use explosion features to include narrower terms.

• Always review the scope notes for correct definitions.

Example Search Strategies

Note: Modify and tailor search terms based on your topic. Use database-specific syntax (e.g., field tags, proximity operators) for accuracy.

Watch our YouTube video on searching PubMed to deepen your practical skills and enhance your search strategy expertise: Search Strategy for a Systematic Review: PubMed Search Tutorial

Final Thoughts

Crafting an effective systematic review search strategy is a combination of art and science. While there is no universal formula, best practices include:

• Understanding the research question thoroughly.

• Applying appropriate frameworks to structure your terms.

• Leveraging Boolean logic, proximity operators, and subject headings wisely.

• Testing and refining the search iteratively.

• Balancing sensitivity and specificity to maintain quality and manageability.

A carefully constructed search strategy not only saves time but ensures that your systematic review is as complete, transparent, and unbiased as possible.

You may also like:

• Introduction to Systematic Reviews
• Planning a Systematic Review
• Selecting Databases for a Systematic Review