RESEARCH DISCIPLINE
Statistical Analysis for Public Policy Research
Consulting for Program & Evaluation Studies
Public policy statistical analysis for program evaluations, administrative records and agency performance data, with the comparison behind every impact estimate made explicit and the results prepared for a journal, a funder or an oversight audience.
No Commitment Required
Strictly Confidential
In-House PhD-Level Statisticians
Fixed Project Fee
Same-Day Response
Statistical Analysis for Public Policy Research
Public policy statistical analysis is the statistical design, analysis and reporting of studies that estimate what a program, regulation or administrative reform changed, usually from agency records, program rosters and repeated surveys. Participation is rarely allocated at random. Service users, schools or municipalities sit inside agencies and regions, and outcomes are recorded by the organization that runs the program, so a regression with a policy indicator can mix the effect of the program with who chose to take part and understate uncertainty. OnlineSPSS, a specialist statistical consultancy for academic, clinical and professional research, provides policy evaluation statistics through in-house PhD-level statisticians, who settle the comparison group, the unit that was allocated and the estimand with you before any model is fitted.
WHEN TO ENGAGE
Statistical Support at Different Stages of Public Policy Research
Policy researchers usually reach out at a recognizable point, from the first plan for an evaluation to an oversight body's questions about a finished report.
Designing an Evaluation You are planning a program evaluation and need the comparison group, the unit that is allocated, the primary outcome and the precision your records can deliver settled before the program starts.
Analyzing Program Records Your agency or program records are assembled, and the treatment of selection into the program, the clustering by office or area and the impact model need to be set up, run and checked.
Preparing a Policy Paper Results are ready for a journal article, a funder report or a policy brief, and need effects in practical units with intervals and a methods account that the agency can defend.
Reviewing a Completed Evaluation A report relied on a controlled regression or a simple before-and-after comparison, and a reviewer, advisor or oversight body has questioned the comparison group, the clustering or the causal wording.
RESEARCH CONTEXT
Research Designs and Data We Work With in Public Policy Research
Public administration data analysis draws on agency records, rollout schedules and eligibility rules, and each source carries a structure that decides what the data can show.
01
Program Participation Records
Rosters of enrollees and nonparticipants where take-up depended on eligibility rules, caseworker discretion or the choice of the person.
02
Staggered Policy Rollouts
Municipalities, states or agencies observed before and after a law or rule took effect, often on different dates in different places.
03
Eligibility-Threshold Programs
Benefits, services or sanctions allocated by a score or an age cutoff, so that units just above and just below the line can be compared.
04
Randomized and Lottery-Based Pilots
Randomized pilots and lotteries for oversubscribed services, allocated by person or by site, with offers that not everyone accepts.
05
Administrative Outcome Records
Employment, benefit, enrollment or service records kept by agencies, where outcomes are observed only while people remain in the system.
06
Performance and Spending Indicators
Agency-level indicators and spending series across jurisdictions, with few units, changing definitions and many reported measures.
METHODOLOGICAL CONSIDERATIONS
Statistical Issues We Consider in Public Policy Research
These checks come first in a program evaluation, because how the program reached its participants can change an estimate more than the choice of model does.
How Units Selected the Program Who entered the program and why is documented, and balance on earlier outcomes and covariates is examined, since adjusting for what was measured cannot remove selection on what was not.
Parallel Trends and Anticipation A comparison-group design needs treated and comparison units to have moved together before the policy, so pre-period trends are examined, along with early adoption in anticipation of the rule.
Staggered Adoption Timing When places adopt at different dates, a single two-way fixed-effects coefficient can use already treated units as controls and misstate the effect, so the estimator is chosen for the timing pattern.
Few Clusters With a handful of states or agencies, conventional clustered standard errors can be too small, so the number of clusters is checked and small-sample or resampling corrections are applied where needed.
The Allocation Rule Sets the Estimand The rule that allocated the program decides what is estimated: the effect of an offer, a local effect at a cutoff, or an effect among those who comply, and each is worded as such.
Practical Size and Uncertainty Effects are reported in the outcome's own units with intervals and set against the smallest change that would matter to the agency, not only against whether a coefficient reaches statistical significance.
STATISTICAL METHODS
Methods Selected for the Research Question and Study Design
Each method family is chosen for the outcome and the data structure of your study, and the line under each title explains why it is used in policy evaluation.
Longitudinal & Multilevel Analysis
Jurisdictions observed before and after a rule support a difference-in-differences contrast, and service users nested in offices or regions call for multilevel or cluster-robust inference.
Clinical, Survival & Epidemiological
A policy with a known start date is judged against its own earlier trend, an eligibility cutoff supports a discontinuity design, and matching adjusts for observed selection only.
Regression & Predictive Modeling
Employment, reenrollment and other binary outcomes and graded service ratings are modeled with agency-level clustering carried into the standard errors, and program pathways examined through mediators.
Comparative & Hypothesis Testing
Randomized pilots are compared on baseline-adjusted outcomes and on take-up or employment rates by arm, since separate pre and post tests cannot show a program effect.
Bayesian & Computational Methods
A wild cluster bootstrap improves intervals when only a few states or agencies exist, and imputation handles outcomes lost when people leave the program records.
Multivariate, Scale & Reliability
Service-user satisfaction and service-quality surveys are scored as scales, with reliability checked before offices or sites are compared on the results.
DELIVERABLES
Statistical Outputs Prepared for Your Research
The outputs follow the structure of a policy evaluation, from the review of the program rules to the scripts that regenerate every reported estimate.
Data & Study Review The program rules, enrollment dates, outcome definitions, cluster identifiers and the pattern of missing outcomes are reviewed before any impact model is fitted.
Statistical Analysis The agreed comparison-group, discontinuity, interrupted-series or randomized-trial analysis, with balance, pre-trend and clustering checks and the planned sensitivity analyses.
Effect Estimates & Uncertainty Program effects in the outcome's units with confidence intervals that reflect the clustering, and the effect of an offer or at a cutoff stated as such.
Tables & Figures Balance tables, event-study and trend figures, effect tables by subgroup and the participant and site counts that reporting guidelines ask for.
Statistical Interpretation Each estimate is explained in policy terms within what the design supports, selection assumptions are stated, and nonrandomized findings are worded as associations or program effects only as far as justified.
Analysis Files SPSS syntax, R scripts, Stata do-files or SAS programs, with the program-rule variables and cluster structure documented so that every table can be regenerated.
REPORTING
Reporting Public Policy Research
The guideline follows the design of the evaluation, and each is applied only where it fits.
Randomized pilots and lottery-based studies report to CONSORT 2025, with the cluster extension where sites rather than people were randomized. Evaluations without randomization report to TREND, and analyses of administrative records that do not evaluate an intervention report to STROBE. Where a health or social care program is evaluated for cost-effectiveness, the economic evaluation follows CHEERS 2022. The program rules, the comparison group, the balance achieved, the number of clusters and the handling of missing outcomes are documented from the analysis output, so an agency reader can check each step. Effects are shown in practical units with confidence intervals, and funder, agency and journal requirements are checked on top.
ILLUSTRATIVE EXAMPLE
How the Analysis May Look in Practice
An illustration of how a program evaluation built on administrative records moves from a roster to a result whose assumptions are stated.
ILLUSTRATIVE EXAMPLE · STATE WORKFORCE TRAINING EVALUATION · UNITED STATES
A state workforce agency's evaluation unit wanted to know whether a subsidized job-training program was associated with higher employment a year later than among similar unemployed workers who did not enroll.
Enrollment was voluntary and depended on caseworker referral, so participants differed from nonparticipants, and workers were nested in a modest number of local offices. Before outcomes were examined, we fixed the comparison as nonenrolled claimants from the same offices, employment in the fourth quarter after program entry, or after the claim date for comparison claimants, as the outcome, and weighting on earnings history and prior unemployment spells, with balance tables as the check. Standard errors were clustered by office, and because the offices were few, intervals were checked with a wild cluster bootstrap, and a sensitivity analysis asked how strong an unmeasured referral factor would need to be to remove the estimate. The result was worded as an association under the stated selection assumptions.
Outcome: The unit received the adjusted difference in employment probability with its 95% confidence interval, balance tables before and after weighting, the sensitivity analysis, a TREND-aligned methods paragraph and the scripts.
This example does not describe a real client or project. We never share details of actual client work, in line with the confidentiality we promise every researcher. This illustrative example is included to show the type, scope, and standard of work involved in this kind of analysis.
RESEARCHER VOICE
What Researchers Say
Feedback from a policy analyst on support that understood the field, on clustering in the records and on reporting framed around how studies are reviewed.
“We needed statistical support from someone who understood Public Policy & Administration, not just someone who could run software. The main issue in our data was the clustered structure of the data rather than treating every row as independent. The analysis and reporting were framed around the way studies in our field are actually designed and reviewed, which made a noticeable difference.”
HOW WE ASSESS YOUR PROJECT
We Review Your Actual Research
Your assessment is based on a review of your actual research materials and analytical requirements, not a generic project category.
You Submit Your Project Details
Share whatever is available at the current stage, such as the research aim, protocol or proposal, dataset, variable list, manuscript or statistical requirements.
We Review Your Research and Define the Scope
Your statistically trained Research Coordinator reviews the materials to understand the study design, research questions, data structure, analytical requirements and expected deliverables. If clarification is needed, additional documents or information can be requested before the assessment is finalized.
Your Analysis Is Completed and Delivered
Once the scope and fixed project fee are agreed, the in-house statistician completes the work and the Research Coordinator remains the communication point through delivery, with the analysis, deliverables and timeline as agreed.
SOFTWARE
Statistical Software for Public Policy Research
Policy analyses are run in the package your agency or research group already uses, with syntax or scripts provided where applicable.
Statistical Support Within Your Research OnlineSPSS provides the agreed methodological, analytical, and statistical reporting support. The researcher or research team retains the scientific question, substantive interpretation, conclusions, and overall responsibility for the research.
FREQUENTLY ASKED QUESTIONS
Questions About Statistical Analysis for Public Policy Research
Short answers to what policy researchers ask before sharing an evaluation, from design choice and staggered adoption to few clusters, review and confidentiality.
Which statistical methods are used in public policy research?
Public policy research, and policy impact analysis in particular, most often uses comparison-group designs such as difference-in-differences, interrupted time series, regression discontinuity, matching or weighting for selection, and multilevel or cluster-robust models for outcomes recorded within agencies and regions. The choice follows how the program reached its participants and what the records contain. Our Statistical Methods pages describe each family.
Can a program be evaluated from administrative records alone?
Administrative records can support an evaluation when they identify participants, offer a credible comparison group and record the outcome before and after entry. What they usually lack is the reason people entered, so the analysis documents how selection occurred and tests how sensitive the estimate is to what is missing from the file.
How do you analyze a policy adopted in different places at different times?
Staggered adoption needs an estimator built for it, because a single two-way fixed-effects coefficient can compare later-treated places with places that were already treated. We examine the adoption pattern, use an estimator that compares each adopting group only with units not yet treated or never treated, and show event-study estimates with intervals so the timing of the effect can be read.
How many states or agencies do I need for clustered standard errors?
There is no single threshold, but inference becomes unreliable when the clusters are few, and especially when few of them are treated, and default standard errors are then often too small. We count the clusters and the treated clusters, and use small-sample corrections, a wild cluster bootstrap or randomization inference where the design calls for them.
How is confidential agency or program data handled?
Research data and project details are handled confidentially, and project information is not published. An NDA is available where required and can be arranged before sensitive materials are shared. Read more about confidentiality and data security.
What happens when I request an assessment, and how long does it take?
You send your evaluation question, the program rules and any data or plan you have. A statistically trained Research Coordinator reviews them and responds the same day, and the analysis, deliverables, timeline and one fixed project fee are confirmed before work begins, so the timeline follows from the agreed scope.
Still have a question? Find more answers in our full FAQ.
NEXT STEP
Discuss Your
Public Policy Research With Us
Share your evaluation question, the program rules and the records you have, even if only a draft plan or a roster exists. We review how participants were selected and confirm the approach before work begins.
No Commitment Required
Strictly Confidential
Fixed Project Fee
In-House PhD-Level Statisticians
Same-Day Response
Serving Researchers Since 2014