Cochrane Qualitative and Implementation Methods Group (CQIMG) Criteria (for CERQual)

The CQIMG developed and formalized a set of criteria for assessing confidence in findings from qualitative evidence syntheses (QES). The CERQual approach embodies these criteria. According to the CQIMG and CERQual framework, every synthesized qualitative “finding” (e.g., a theme, concept, or interpretation) should be assessed with respect to four core components [1, 2, 3]:

1. Methodological Limitations
2. Coherence
3. Adequacy of Data
4. Relevance

Optionally, a fifth component, dissemination (publication) bias, is recognized as potentially relevant, though not yet routinely applied. Based on judgments across these components, each review finding is assigned an overall confidence rating: high, moderate, low, or very low [1].  

In short: the CQIMG criteria = the four (or five) CERQual components used to systematically assess how much confidence we should place in qualitative review findings.

Components

1. Methodological Limitations

• The methodological limitations capture the extent to which the primary qualitative studies contributing to the review findings have weaknesses or flaws that could potentially undermine the credibility and trustworthiness of their data or interpretations.
• This can be compared to the concept of “bias / internal validity” in quantitative reviews, which is also applicable to qualitative synthesis.

What this includes:

• The credibility of findings may be affected by limitations in study design or conduct (sampling, data collection, data analysis, reflexivity, transparency, ethical issues, etc.) [5].
• Poor reporting, insufficient detail, or lack of clarity about methods because sometimes methodological rigor may exist but is not sufficiently documented [3].

How to assess:

• Choose an appraisal tool appropriate to the qualitative design of the primary studies (e.g., checklists, signaling questions). The CQIMG guidance emphasizes careful selection of a suitable tool [3].
• For each primary contributing study, carry out a critical appraisal and summarize strengths and limitations.
• When assessing a review finding (which may draw on multiple studies), consider collectively the methodological limitations of all contributing studies. Are there widespread or serious problems, or are they minor/mixed?
• Document judgments transparently; these judgments feed into the overall CERQual confidence rating [2].

Significance: Because qualitative findings derive from interpretive processes, weaknesses in methods or poor transparency can seriously undermine the validity of those interpretations. Evaluating methodological limitations ensures that synthesized findings are not built on shaky foundations, which is critical if those findings will inform practice or policy.

2. Coherence

• Coherence refers to the extent to which data from the contributing primary studies fit together sensibly to support the review finding. It assesses whether the pattern of findings across studies makes sense and whether there are unexplained contradictions, gaps, or outlier data that weaken confidence [1, 2].
• In metaphorical terms: does the “story” told by the data hang together reliably?

What this includes:

• The consistency of themes or interpretations across multiple studies is a crucial factor to consider.
• The clarity of the link between data (quotes, observations) and the synthesized finding. For instance, the assessment should focus on whether the findings accurately reflect the underlying data, avoiding over-interpretation or speculation.
• It is crucial to ensure that divergent findings, such as subthemes and context differences, are appropriately addressed rather than ignored.

How to assess:

• During synthesis (e.g., thematic synthesis, meta-aggregation), track which studies support which aspects of the finding; note contradictions, minority views, and context-specific variations.
• For each synthesized finding, the map of supporting data (with citations) shows which studies support which subcomponents and identifies inconsistencies or variations.
• Evaluate whether the finding as formulated adequately describes the data without over-generalizing.

Significance: Qualitative research often deals with complexity, nuance, and context-dependence. Without coherence, a synthesized finding may oversimplify, misrepresent, or gloss over important variation, which undermines its usefulness, especially for policy or intervention design.

3. Adequacy of Data

• Adequacy refers to whether there is sufficient quantity and richness of data to support the synthesized finding. That is, do we have enough “thick,” detailed, contextual data (e.g., rich participant quotations, detailed observations) from a sufficient number of studies to justify the finding [6]?
• Helps rule out findings that are based on sparse or thin data (e.g., a single quote, a single small study), which may not robustly reflect the phenomenon of interest.

What this includes:

• The number of studies contributing data to that finding should ideally be multiple and varied (different settings, populations) unless the phenomenon is necessarily narrow.
• Depth and richness of data: detailed quotes, nuanced contexts, and consistent analytical depth across studies.
• Variation in contexts/populations (if relevant), to ensure findings are not over-contextualized (unless the review aims to be context-specific).

How to assess:

• For each review finding, list contributing studies and examine the nature of the data: how many studies, how many participants, how detailed are data excerpts, and how diverse are contexts?
• Reflect on whether the data together provide adequate support for the finding; if not, downgrade confidence.

Significance: If data are too thin or come from a very limited context, the synthesized finding may not be robust; it may lack explanatory power or generalizability, making it unreliable for application in other settings or for policy design.

4. Relevance

• Relevance concerns how applicable the body of data underlying a review finding is to the context, population, phenomenon, and setting specified in the review question. Essentially: do the contributing primary studies match the review’s focus closely enough [1, 6]?
• It ensures that conclusions drawn from synthesis are meaningful for the intended target context.

What this includes:

• The synthesis ensures that participant characteristics (demographics, socio-cultural background, and clinical status) are similar (or appropriate) to the population of interest.
• The study settings (geographical, health system, cultural, and institutional) should be similar to the target context.
• Relevance of the phenomenon studied to the phenomenon under review (e.g., same type of intervention, same socio-cultural conditions, similar health/social process).

How to assess:

• For each contributing study, compare key contextual characteristics (population, setting, phenomenon) with the review’s inclusion criteria or context-of-interest.
• For each review finding, explicitly note any contextual differences that may limit applicability.
• In the CERQual Summary of Qualitative Findings tables, report relevance judgments (e.g., high, moderate, or low relevance) along with explanations.

Significance: Qualitative findings are often context-sensitive. A finding derived from studies in one cultural or institutional context may not generalize to another. Without assessing relevance, a synthesis risks overextending conclusions, leading to misguided policy or practice recommendations.

Applying the CQIMG/CERQual criteria

The CQIMG has published a series of methodological guidance papers (2018) describing in detail how to apply CERQual. The major steps and recommendations are

• Define review findings Each analytic output (theme, assertion, interpretation) should be clearly phrased as a “review finding” [2].
• For each review finding, assess all four components (methodological limitations, coherence, adequacy, and relevance) [5].
• Use transparent documentation Produce a “CERQual Evidence Profile” and a “Summary of Qualitative Findings (SoQF)” table. These tables list for each finding the contributing studies, judgments on each component, overall confidence rating, and explanation/justification.

Sample: Summary of Qualitative Findings (SoQF) Table in the style recommended by Cochrane & GRADE-CERQual guidance

Review Finding	Studies Contributing to the Finding	CERQual Confidence Rating	Explanation of Judgment
1. Patients valued personalized communication from healthcare providers.	8 qualitative studies across 5 countries	High confidence	There are minor methodological limitations, strong coherence, rich and detailed data, and high relevance to the review context.
2. Workload pressures reduced nurses’ ability to provide emotional support.	6 studies from hospital-based settings only	Moderate confidence	There were some methodological limitations; the data adequacy was moderate, and the relevance to non-hospital settings was limited.
3. Digital health tools were perceived as difficult to use among older adults.	4 studies, limited contextual diversity	Low confidence	Thin data; coherence concerns; contributing studies from similar populations, reducing generalizability.
4. Community involvement increased acceptability of public health interventions.	2 studies in rural LMIC settings	Very low confidence	There are serious methodological limitations, insufficient data, and limited relevance to urban or high-income settings.

Sample: CERQual Evidence Profile Table in the style recommended by Cochrane & GRADE-CERQual guidance.

Review Finding	Methodological Limitations	Coherence	Adequacy of Data	Relevance	Overall CERQual Assessment	Summary of Explanation
1. Patients valued personalized communication from healthcare providers.	Minor concerns	Consistent across studies	Rich, detailed data from multiple contexts	Highly relevant	High confidence	Strong coherence and rich data; minor methodological issues.
2. Workload pressures reduced nurses’ ability to provide emotional support.	Moderate concerns (reporting gaps)	Mostly coherent	Moderately rich data	Relevant but context-limited	Moderate confidence	Some methodological issues and limited settings reduce confidence.
3. Older adults found digital tools difficult to use.	Serious concerns (sampling and reflexivity issues)	Some inconsistencies	Thin data from few studies	Relevant but low diversity	Low confidence	Weak data adequacy and methodological issues reduce confidence.
4. Community involvement increased intervention acceptability.	Serious concerns	Inconsistent across studies	Very limited data	Highly context-specific	Very low confidence	Insufficient data and serious limitations make the finding uncertain.

• Avoid mechanistic scoring / aggregation: CERQual is not a scoring tool; it requires judgment. The guidance emphasizes that context matters and that different components may carry different weight depending on the finding [2].
• Use suitable critical appraisal tools like JBI & CASP for methodological limitations. CQIMG does not mandate one single tool but requires reviewers to pick an appropriate one for the design(s) included and justify their choice [7].

Implications

In conducting a qualitative evidence synthesis (or including qualitative evidence in a broader mixed-method review), using the CQIMG/CERQual criteria offers the following methodological safeguards and advantages:

• Provides a robust, transparent, and internationally accepted standard for evaluating qualitative evidence, increasing the credibility of your findings.
• Ensures that contextual relevance and data richness are considered especially important in cross-cultural, global health, or implementation research.
• Helps produce usable, policy-relevant summaries (via Summary of Qualitative Findings tables) that are easier for stakeholders and decision-makers to interpret.
• Allows for nuanced, reasoned confidence judgments rather than simplistic “good/bad” or “include/exclude” determinations, preserving complexity while enabling synthesis.

Bibliography

1. Chapter 21: Qualitative evidence | Cochrane. (n.d.). https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-21
2. Lewin, S., Booth, A., Glenton, C., Munthe-Kaas, H., Rashidian, A., Wainwright, M., Bohren, M. A., Tunçalp, Ö., Colvin, C. J., Garside, R., Carlsen, B., Langlois, E. V., & Noyes, J. (2018). Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implementation Science, 13(S1), 2. https://doi.org/10.1186/s13012-017-0688-3
3. Noyes, J., Booth, A., Flemming, K., Garside, R., Harden, A., Lewin, S., Pantoja, T., Hannes, K., Cargo, M., & Thomas, J. (2017). Cochrane Qualitative and Implementation Methods Group guidance series—paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. Journal of Clinical Epidemiology, 97, 49–58. https://doi.org/10.1016/j.jclinepi.2017.06.020
4. https://www.cochrane.org/sites/default/files/uploads/PDFs/Selecting%20Studies%20and%20Assessing%20Methodological%20Limitations.pdf
5. Noyes, J., Booth, A., Cargo, M., Flemming, K., Garside, R., Hannes, K., Harden, A., Harris, J., Lewin, S., Pantoja, T., & Thomas, J. (2017). Cochrane Qualitative and Implementation Methods Group guidance series—paper 1: introduction. Journal of Clinical Epidemiology, 97, 35–38. https://doi.org/10.1016/j.jclinepi.2017.09.025
6. Glenton, C., Carlsen, B., Lewin, S., Munthe-Kaas, H., Colvin, C. J., Tunçalp, Ö., Bohren, M. A., Noyes, J., Booth, A., Garside, R., Rashidian, A., Flottorp, S., & Wainwright, M. (2018). Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 5: how to assess adequacy of data. Implementation Science, 13(S1), 14. https://doi.org/10.1186/s13012-017-0692-7
7. Smith, V., Smith, A., & Carroll, L. (2025). Quality appraisal tools used in qualitative evidence syntheses of maternity care research: A scoping review. Midwifery, 148, 104479. https://doi.org/10.1016/j.midw.2025.104479