STATISTICAL METHOD
Difference-in-Differences Analysis
Statistical Consulting for Policy Evaluation
We apply difference-in-differences analysis within your evaluation design, comparing treated and comparison groups before and after the change, examining the parallel trends assumption, and reporting the effect on the treated with 95% confidence intervals.
No Commitment Required
Strictly Confidential
In-House PhD-Level Statisticians
Fixed Project Fee
Same-Day Response
METHOD OVERVIEW
What Is Difference-in-Differences?
Difference-in-differences compares the change in an outcome over time in a group exposed to a policy or program with the change in a comparison group that was not exposed, and estimates the average effect on the treated. The comparison group stands in for what would have happened to the treated group without the change, which holds only if both groups would have followed parallel trends. That assumption cannot be verified for the post-treatment period, so credibility rests on the design, the pre-treatment evidence and sensitivity checks. OnlineSPSS, a specialist statistical consultancy for academic, clinical and professional research, provides difference-in-differences analysis within your study design, with the relevant checks, effect estimates, uncertainty measures, interpretation and publication-ready reporting.
WHEN TO ENGAGE
When Is Difference-in-Differences Appropriate?
The design fits a change that reached some units and not others, observed before and after, and these are the research situations in which researchers most often ask us to apply or confirm it.
Policy Adopted by Some Units A law, program or fee change reached some states, clinics or firms and not others, and outcomes exist for both groups before and after the change.
Natural Experiment, Two Groups An external event exposed one group, such as a plant closure or a regional ban, and a similar unexposed group was observed over the same periods.
Staggered Adoption Across Sites Units adopted the policy in different periods, so a single before-and-after split does not apply and the estimator must handle varying treatment timing.
Reviewing a Completed DiD Model A DiD regression is finished, and a reviewer has asked about parallel trends, timing or clustering. Independent Statistical Review re-examines the design and the estimator.
ANALYTICAL APPROACH
How We Apply Difference-in-Differences
Within the family of Longitudinal & Multilevel Analysis, the design is judged first, because the estimate is only as credible as its comparison group.
Comparison Group and Timing We establish how and when treatment was assigned, whether the groups are comparable, and whether anticipation, spillovers or shifts in who is observed threaten the contrast.
Parallel Trends Evidence Parallel trends cannot be tested after treatment. We inspect pre-period trends in an event-study plot, read a pre-trend test as low-powered evidence, and add sensitivity analyses.
Estimator and Staggered Timing When effects differ across groups or over time, two-way fixed effects can use already-treated units as controls and bias the estimate, so we use staggered-timing estimators.
Inference and Functional Form We cluster standard errors at the level where treatment was set, use methods suited to few clusters, and fix the outcome scale, such as levels or logs, before results are seen.
DELIVERABLES
What You Receive
As part of our Quantitative Data Analysis service, the evaluation is delivered with the estimates, checks and files needed to report and defend it.
Data & Variable Review Group, period and treatment-timing coding, the unit of analysis, panel balance, attrition and changes in who is observed are reviewed before any model is run.
Method-Specific Checks Pre-period trends, event-study estimates, anticipation, clustering and sensitivity to the comparison group are examined, with a staggered-adoption estimator where timing varies.
Statistical Results The average effect on the treated with its 95% confidence interval, event-study estimates by period relative to treatment, and group-time effects where they are estimated.
Publication-Ready Tables & Figures An event-study plot with confidence intervals, a trend figure for both groups and a regression table, formatted to the journal's requirements, ready to insert.
Interpretation The effect is read as an average effect on the treated under stated assumptions, with the strength of the parallel trends case and the remaining threats discussed plainly.
Reproducible Analysis Files Stata do-files, R scripts, SAS programs or Python code for the models and figures are provided where applicable, so every estimate can be rerun if the data change.
ALTERNATIVE METHODS
When Another Method May Be More Appropriate
A comparison group and a believable parallel trends argument are required. These methods fit when no comparison group exists, when treatment follows a cutoff or when exposure is not a group-level change.
01
Interrupted Time Series
With one exposed series and no comparison group, a segmented model estimates the level and slope change, and the earlier trend becomes the only benchmark.
02
Propensity Score Matching (PSM)
When outcomes are measured once after exposure, matching on observed baseline traits balances the groups, but it cannot remove differences in unmeasured ones.
03
Regression Discontinuity (RD)
When eligibility follows a threshold on a test score or income, the jump in outcomes at that threshold estimates the effect locally, without any parallel trends assumption.
04
Panel Data Analysis (Fixed & Random Effects)
When a continuous exposure varies within units over time, rather than a policy switched on for one group, a fixed effects model with clustered standard errors fits.
CASE EXAMPLE
How We Work in Practice
An illustration of how a policy evaluation moves from a planned comparison to a reportable, defensible effect estimate.
ILLUSTRATIVE EXAMPLE · TRANSPORTATION POLICY RESEARCH · UNITED STATES
A regional transit authority cut fares on selected bus routes and wanted to evaluate whether weekly ridership changed relative to routes with unchanged fares.
Fares fell on the selected routes in one month, so we proposed difference-in-differences with unchanged routes as the comparison group. Before outcomes were examined, we fixed weekly boardings on the log scale, the route as the unit, standard errors clustered by route and an event-study specification. A planned redesign on some comparison routes threatened the contrast, so the primary analysis excluded them and a sensitivity analysis kept them. Pre-period trends were read from the event-study plot, a pre-trend test was treated as limited evidence, and comparison routes sharing stops with treated ones were dropped in a planned check for riders switching routes.
Outcome: The authority received the average effect on treated routes with its 95% confidence interval, the event-study plot, the sensitivity results, a methods paragraph and the analysis code.
This example does not describe a real client or project. We never share details of actual client work, in line with the confidentiality we promise every researcher. This illustrative example is included to show the type, scope, and standard of work involved in this kind of analysis.
SOFTWARE
Statistical Software for Policy Evaluation
Difference-in-differences models and event-study plots are run in the statistical software researchers already use, with syntax or scripts provided where applicable.
RESEARCH DISCIPLINES
Policy Evaluation Across Research Fields
Difference-in-differences designs appear across research fields, each with its own policies, comparison groups and reporting conventions.
A Defined Process for Your Analysis
We review your dataset, research question and design before any work begins, so the scope, deliverables, timeline and fixed project fee are clear from the outset.
01
Share Your Research Materials
Send your dataset, research question, study design and any existing analysis plan or output. We will let you know if anything else is needed to assess the project.
02
Receive Your Assessment
Your statistically trained Research Coordinator reviews the project and confirms the required analysis, deliverables, timeline and fixed project price before work begins.
03
Analysis & Delivery
Once you confirm the scope, an in-house statistician completes the agreed analysis and delivers the outputs, interpretation and reporting files.
RESEARCHER VOICE
What Researchers Say
Feedback from a policy researcher who needed the clustered structure of the data handled correctly in an evaluation.
“I had already started the Difference-in-Differences (DiD), but I was not confident about the clustered structure of the data rather than treating every row as independent. The team checked the setup against our study design, explained the decisions in plain language, and returned results we could actually use. The final output was concise, clearly interpreted, and ready to use in our research reporting.”
RESEARCH RESPONSIBILITY
Statistical Support Without Taking Over Your Research
We provide the statistical work for your policy evaluation, and the study, its meaning and its authorship stay entirely with you.
Agreed Statistical Support We provide the agreed methodological, analytical and statistical reporting support for your difference-in-differences analysis, within the scope confirmed before work begins.
No Intellectual Contribution OnlineSPSS claims no intellectual contribution to your research. Our work is statistical support, and authorship remains yours.
Your Research Decisions The research question, the choice of comparison group and your conclusions remain with you, and our interpretation stays within what the estimates and the design support.
Confidential Handling Research data and project details are handled confidentially and are not published. An NDA is available where required, and can be arranged before sensitive materials are shared.
FREQUENTLY ASKED QUESTIONS
Questions About Difference-in-Differences
Short answers to what researchers ask before evaluating a policy with a comparison group, from parallel trends to confidentiality and the assessment process.
Can difference-in-differences work without a comparison group?
No, the comparison group is what separates this design from a simple before-and-after comparison, because it shows how the outcome changed over time without the treatment. With one exposed series and many time points, an interrupted time series estimates level and slope changes, but it is open to events that coincided with the change. A comparison series, where one exists, strengthens that design.
Can parallel trends be tested with my pre-period data?
Only partly. Parallel trends concerns what would have happened to the treated group after treatment, which is never observed, so no test confirms it. Pre-period trends and event-study estimates give supporting evidence, but a pre-trend test has low power and passing it does not establish the assumption. We pair the plot with design arguments and sensitivity analysis, so the claim is stated at the strength the evidence allows.
Why does staggered adoption need newer DiD estimators?
When units adopt at different times and effects vary over time or by adoption group, the two-way fixed effects coefficient is a weighted average that can use already-treated units as controls, and some weights can be negative. The result may not match any sensible average effect. Callaway-Sant’Anna, Sun-Abraham and de Chaisemartin-D’Haultfoeuille estimators compare against never-treated or not-yet-treated units instead.
Can you review an analysis that has already been completed?
Yes, we can review a completed DiD analysis before submission or a response to reviewers. We check the comparison group, the parallel trends evidence, the treatment timing and estimator, the level at which standard errors were clustered and the reporting, and confirm what stands or needs to change.
How do you protect my dataset and research confidentiality?
Your dataset, output files and any protocol or analysis plan are handled confidentially, and project information is not published. An NDA is available where required and can be arranged before sensitive materials are shared. Read more about confidentiality and data security.
What happens when I request an assessment, and how long does it take?
You share your research question, design and relevant materials, and a statistically trained Research Coordinator reviews the request and responds the same day. The analysis, deliverables, timeline and one fixed project fee are then confirmed before any work begins, so the timeline follows from the agreed scope.
Still have a question? Find more answers in our full FAQ.
NEXT STEP
Need Support With
Difference-in-Differences?
Share your research question, the treatment timing and your data. We assess whether the design fits your evaluation and confirm the approach before any work begins.
No Commitment Required
Strictly Confidential
Fixed Project Fee
In-House PhD-Level Statisticians
Same-Day Response
Serving Researchers Since 2014