STATISTICAL METHODS
Evidence Synthesis & Meta-Research
Statistical Consulting for Combining Evidence
Our support with evidence synthesis methods starts by judging what your included studies can legitimately say together, then matches the synthesis to the question, from systematic review and meta-analysis to network, diagnostic and narrative approaches.
Evidence Synthesis & Meta-Research is the family of quantitative evidence synthesis methods that identify, appraise and combine results across studies, estimating a summary effect and its heterogeneity only where the studies are similar enough to support one, and judging the certainty of the evidence either way. Within Statistical Methods, these methods are separated by the question asked, by the data available, and by whether pooling is justified at all. OnlineSPSS is a specialist statistical consultancy for academic, clinical and professional research, and the analysis is carried out in-house by PhD-level statisticians.
METHODS IN THIS CATEGORY
Evidence Synthesis & Meta-Research Methods
Eight methods for evidence synthesis statistics: review methods that identify and appraise studies, pooling methods for effects, accuracy and prevalence, and approaches for moderators, networks and evidence that cannot be pooled.
01
Meta-Analysis
Pools effect estimates across studies with common-effect or random-effects models, with heterogeneity, a prediction interval and a forest plot.
Pooled effect
Heterogeneity
02
Systematic Review
Identifies, screens and appraises every eligible study under a protocol set in advance, whether or not a pooled estimate follows.
Protocol
Risk-of-bias appraisal
03
Meta-Regression
Relates study-level characteristics to effect sizes to explore heterogeneity, within the limits that a small number of studies sets.
Moderators
Study-level covariates
04
Network Meta-Analysis (NMA)
Compares several treatments in one model by combining direct and indirect evidence, under transitivity, and checks their consistency.
Indirect comparison
Consistency
05
Diagnostic Test Accuracy Meta-Analysis
Pools sensitivity and specificity across studies of a test with a bivariate or hierarchical model that keeps the two measures linked.
Test accuracy
Bivariate model
06
Individual Participant Data (IPD) Meta-Analysis
Combines participant-level records from several studies, which allows consistent models and tests of participant-level interactions.
Participant level
Interactions
07
Proportion / Prevalence Meta-Analysis
Pools proportions or prevalence across studies with a logit or mixed-model approach that keeps estimates between zero and one.
Prevalence
Transformation
08
SWiM (Synthesis Without Meta-Analysis)
Synthesizes results when studies are too different to pool, using explicit grouping, standardized metrics and transparent reporting.
No pooling
Structured grouping
METHOD COMPARISON
Choosing Between Methods in This Family
Start from the question and from whether the studies are comparable enough to combine; when they are not, a structured synthesis without pooling is a legitimate result, not a failure of the review.
| Research situation | Typical method |
|---|---|
| Pool one effect measure across comparable studies | Meta-Analysis |
| Identify and appraise all eligible studies for a question | Systematic Review |
| Explain between-study differences with study characteristics | Meta-Regression |
| Compare three or more treatments across a connected network | Network Meta-Analysis (NMA) |
| Pool the sensitivity and specificity of a diagnostic test | Diagnostic Test Accuracy Meta-Analysis |
| Participant-level data available from the included studies | Individual Participant Data (IPD) Meta-Analysis |
| Pool a proportion, rate or prevalence across studies | Proportion / Prevalence Meta-Analysis |
| Studies too different or too poorly reported to pool | SWiM (Synthesis Without Meta-Analysis) |
COMMON ANALYTICAL ISSUES
Common Issues We See in Evidence Synthesis & Meta-Research
Most weak syntheses go wrong before any model is fitted, in how comparability was judged, how data were extracted and when the analyses were planned.
Pooling Before Judging Comparability The question is whether studies are similar enough to combine, not how to force one estimate. Populations, interventions or outcome definitions that differ can make an average meaningless, however tidy the plot.
Extraction Errors Read as Heterogeneity A standard error entered as a standard deviation, or incompatible effect measures, can manufacture heterogeneity. Duplicate extraction and unit checks come before any model, and the differences are not explained away.
Model Chosen by a Heterogeneity Test Cochran's Q misses real heterogeneity among few studies and flags trivial differences among many. Choose the model by the inferential goal, and read a prediction interval for the likely range of true effects.
Unplanned Moderator and Bias Searches Moderator, small-study and sensitivity analyses should be prespecified where possible. With few studies they have little power, so a post hoc search through study characteristics mostly finds false positives.
Unsure whether your studies can be pooled, or which synthesis fits? We can review the question and the extracted data and confirm an appropriate approach.
ANALYTICAL SUPPORT
From Method Selection to Reportable Results
As part of our Meta-Analysis & Evidence Synthesis service, evidence synthesis consulting takes a project from the choice of synthesis to results reported with their uncertainty and certainty.
Method Specification We confirm the review question, eligibility criteria, effect measure and unit of analysis, and document why the chosen synthesis fits what the included studies report.
Assumptions & Diagnostics Comparability, heterogeneity, small-study effects and the influence of single studies are judged through plots, prediction intervals and sensitivity analyses, not one test statistic.
Effect Estimates & Uncertainty Pooled effects, between-study variance and prediction intervals are reported with confidence intervals, with the certainty of the evidence considered in the GRADE framework where it applies.
Publication-Ready Outputs Forest plots, flow diagrams and tables are prepared in line with PRISMA 2020 and the extension that applies, such as PRISMA-NMA, with scripts provided where applicable.
SOFTWARE
Statistical Software for Evidence Synthesis
R, Stata, SAS, IBM SPSS and Python fit the pooling models, Mplus the multivariate ones, and NVivo and MAXQDA support coding of qualitative evidence.
RESEARCH DISCIPLINES
Evidence Synthesis Across Research Fields
Research synthesis methods are used wherever several studies address one question, and each field brings its own study designs, effect measures and reporting conventions.
RESEARCHER VOICE
What Researchers Say
Feedback from two researchers, on deciding what could legitimately be pooled in a review, and on choosing methods and reporting that suit education and social research.
“We had reached the synthesis stage of a review and needed help deciding what could legitimately be pooled. The team looked at effect measures, heterogeneity, study comparability, and bias before choosing the synthesis approach, which was much more useful than simply producing a forest plot.”
“Our projects mix surveys, clustered samples, policy data, and qualitative material, so a one-size-fits-all statistical approach does not work. The team understood the design issues immediately and helped us choose methods and reporting that made sense for education and social research.”
OnlineSPSS provides the agreed methodological, analytical and statistical reporting support and claims no intellectual contribution. The review question, interpretation, conclusions and authorship remain with the researcher or research team.
FREQUENTLY ASKED QUESTIONS
Questions About Evidence Synthesis & Meta-Research
Short answers to what researchers ask before sharing a synthesis project, from choosing a method to confidentiality and the assessment process.
Which synthesis method fits my research question?
The question and the studies decide, and whether the included studies are similar enough to combine comes before which model to use. A review of all eligible evidence needs a protocol, comparable trials of one effect point to meta-analysis, three or more treatments to a network, and a diagnostic test to accuracy pooling. If a synthesis was already run and questioned, Independent Statistical Review re-examines it.
What if a meta-analysis is not appropriate for my studies?
Then not pooling is the correct decision, and it is a result in its own right. A synthesis without meta-analysis groups studies by a stated rule, uses standardized metrics where they exist and reports the direction and size of effects openly, in line with the SWiM guideline. If participant-level data can be obtained, an individual participant data analysis may answer what published aggregates cannot.
How do you protect my research data and confidentiality?
Extraction sheets, screening records, protocols and analysis scripts are handled confidentially, and project information is not published. An NDA is available where required and can be arranged before sensitive materials are shared. Read more about confidentiality and data security.
What happens when I request an assessment, and how long does it take?
You share your research question, design and relevant materials, and a statistically trained Research Coordinator reviews the request and responds the same day. The analytical work, deliverables, timeline and one fixed project fee are then confirmed before any work begins, so the timeline follows from the agreed scope rather than being quoted in advance.
Still have a question? Find more answers in our full FAQ.
NEXT STEP
Need Support Choosing the Right
Synthesis Method?
Share your review question, the studies or data you hold and any protocol. We review how comparable the evidence is and confirm the appropriate synthesis method before any work begins.
No Commitment Required
Strictly Confidential
Fixed Project Fee
In-House PhD-Level Statisticians
Same-Day Response
Serving Researchers Since 2014