RESEARCH DISCIPLINE
Statistical Analysis for Experimental Psychology Research
Consulting for Laboratory & Online Experiments
Experimental psychology statistical analysis for factorial and within-participant designs, trial-level data and preregistered studies, with participants and stimuli modeled as the design requires and results prepared for the journals your field reads.
No Commitment Required
Strictly Confidential
In-House PhD-Level Statisticians
Fixed Project Fee
Same-Day Response
Statistical Analysis for Experimental Psychology Research
Experimental psychology statistical analysis is the statistical design, analysis and reporting of controlled studies in which participants are randomized to conditions or exposed to all of them, with accuracy, reaction times, ratings or choices recorded over many trials. Each participant contributes many correlated responses, the stimuli are a sample of a larger set that the conclusions are meant to cover, and a factorial design offers many interactions to test, so a workflow that averages over trials, ignores the stimuli and reports every significant effect in the ANOVA output produces findings that fail to replicate. OnlineSPSS, a specialist statistical consultancy for academic, clinical and professional research, provides psychology experiment data analysis through in-house PhD-level statisticians, who settle the random effects, the effect of interest and the power for it with you before data collection or analysis.
WHEN TO ENGAGE
Statistical Support at Different Stages of Experimental Psychology Research
Researchers usually contact us at a recognizable point, from the design of a new experiment to a reviewer's question about a finished one.
Planning a Factorial Design You are planning a factorial or within-participant experiment and need the design, the target effect, the numbers of participants and trials, and the exclusion rules settled before the first session.
Analyzing Condition Effects Responses are collected, and the model for participants and stimuli, the handling of condition order and the excluded trials need to be chosen, run and checked.
Preregistering or Reporting You are writing a preregistration or a manuscript and need the analysis plan, a sensitivity power analysis and effect estimates with intervals that are worded as the data allow.
Revisiting a Doubtful Result A significant interaction did not replicate, or a reviewer has asked whether participants and items were random, whether exclusions were flexible or whether the study was powered for its interaction.
RESEARCH CONTEXT
Research Designs and Data We Work With in Experimental Psychology
The datasets behind most psychology experiment data analysis consist of many responses per participant, and the structure of those responses shapes every model.
01
Within-Participant Experiments
Every participant completes all conditions over many trials, so responses are correlated within the person and within the condition.
02
Between-Participant Experiments
Participants randomized to one condition, often tested online, with attention checks, dropout after randomization and a manipulation check to analyze.
03
Mixed and Factorial Designs
Several manipulated factors, some within and some between participants, where the interaction is the effect the hypothesis actually predicts.
04
Multi-Session and Crossover Designs
Participants tested on several days or under several manipulations, such as sleep loss or stress, with session order counterbalanced and carryover to check.
05
Rating and Choice Tasks
Ratings or binary choices made to a sample of stimuli such as faces, words or scenarios, which themselves differ from one another.
06
Preregistered and Replication Studies
Studies with a written plan, a stated smallest effect of interest and exclusion rules fixed before data, including direct replications.
METHODOLOGICAL CONSIDERATIONS
Statistical Issues We Consider in Experimental Psychology Research
Before any effect is read, your treatment of participants and stimuli, your power and your exclusions are examined, since each can change a result more than the choice of test.
Random Slopes for Manipulated Factors When a factor varies within participants or within stimuli, its effect also varies across them. Leaving out the matching random slopes understates the standard error of that effect, so the slopes follow the design.
Power for the Interaction An interaction usually needs a larger sample than a main effect, so a study powered for the main effect tests its interaction with little power. The sample is judged against the smallest effect of interest.
Confirmatory and Exploratory Tests A factorial design offers many tests, and reporting only those that cross .05 inflates false positives. Confirmatory effects are kept apart from exploratory ones, and the number of tests is stated.
Order and Carryover Effects In within-participant designs, practice, fatigue and carryover between conditions can mimic a condition effect. Counterbalancing is checked in the data, and order enters the model where it was varied.
Contrasts Matched to the Hypothesis An omnibus F test shows that some cells differ, not which. The contrast or interaction pattern the hypothesis predicts is specified in advance and tested directly, with its estimate and interval.
Estimates and Null Results Each effect is reported with its confidence interval, and for a null result an equivalence test or an interval is used to show which effect sizes the data can rule out.
STATISTICAL METHODS
Methods Selected for the Research Question and Study Design
Each method family follows the design and the number of observations per participant, and the line under each title says why it suits experimental data.
Longitudinal & Multilevel Analysis
Factorial designs with factors that vary within participants and stimuli need random slopes for each factor, which mixed models estimate even with unbalanced or missing cells.
Comparative & Hypothesis Testing
Balanced designs with within-participant factors use repeated-measures or mixed ANOVA, and a paired comparison when only two conditions are contrasted.
Regression & Predictive Modeling
Binary choices and ratings are modeled on their own scale with logistic or ordinal models, since treating them as continuous can create or hide an interaction.
Bayesian & Computational Methods
Bayes factors weigh the evidence for a null against a stated alternative, and bootstrapping gives intervals for statistics without a simple standard error, such as medians.
Multivariate, Scale & Reliability
Outcome scales and manipulation-check items are tested for reliability in your sample before conditions are compared, since an unreliable outcome can hide a real effect.
Evidence Synthesis & Meta-Research
A direct replication or a series of related experiments can be combined into a pooled estimate with its heterogeneity, instead of a count of significant results.
DELIVERABLES
Statistical Outputs Prepared for Your Research
The outputs follow the structure of an experiment, from the review of trial files and exclusion rules to the scripts that regenerate every reported value.
Data & Study Review Trial files, condition codes, exclusion flags and the preregistration are reviewed, so every response can be traced to a participant, a stimulus and a condition.
Statistical Analysis The agreed mixed or repeated-measures models, with residual and random-effects diagnostics, the prespecified contrasts and the planned exclusion sensitivity analyses.
Effect Estimates & Uncertainty Condition effects and interactions with confidence intervals, standardized effects where meaningful, and equivalence or Bayesian estimates for null results.
Tables & Figures Condition means with intervals, model tables, interaction plots, condition-by-order plots and the participant counts remaining after each exclusion step.
Statistical Interpretation Results are worded as the manipulation supports, confirmatory and exploratory findings are marked apart, and the limits of the sample and the stimuli are stated for the discussion.
Analysis Files R scripts, SPSS syntax or Python code, with the trial-level data structure and exclusion rules documented so that every reported value can be regenerated.
REPORTING
Reporting Experimental Psychology Research
Experimental journals look at how the sample size, exclusions and confirmatory tests were decided, and each standard is applied only where it fits.
Experiments are reported to APA JARS-Quant, including how conditions were assigned or counterbalanced, how the sample size was decided and how many participants each exclusion rule removed. Counterbalancing orders and the number of trials per cell after exclusions are reported, so a reader can rerun the design. A preregistered study states each departure from its plan, and a Registered Report is written to the plan accepted at the first stage of review. Exclusion counts at each step and the distinction between confirmatory and exploratory analyses are tabulated from the analysis itself, so the manuscript and the output agree.
ILLUSTRATIVE EXAMPLE
How the Analysis May Look in Practice
An illustration of how a factorial experiment moves from a power question to a defensible, reportable result.
ILLUSTRATIVE EXAMPLE · JUDGMENT AND DECISION-MAKING RESEARCH · UNITED STATES
A decision-making laboratory tested whether time pressure changes the framing effect in risky choice, with framing varied within participants and time pressure between participants in an online study.
The hypothesis concerned the interaction, not either main effect, and each participant made many binary choices across counterbalanced frame orders. Before data collection, we fixed the plan in writing: a logistic mixed model for choices with a by-participant slope for framing, the interaction as the single confirmatory test, frame order entered as a factor to check for carryover, and every other contrast labeled exploratory. The sample was set by simulating the interaction at the smallest effect of interest from pilot variance components, and the attention-check exclusion rule was fixed in advance, with its rates compared across the time-pressure conditions.
Outcome: The laboratory received the interaction as a ratio of odds ratios with its 95% confidence interval, the simple framing effects under each time condition, the order check, the simulation-based power report, a methods paragraph and the scripts.
This example does not describe a real client or project. We never share details of actual client work, in line with the confidentiality we promise every researcher. This illustrative example is included to show the type, scope, and standard of work involved in this kind of analysis.
RESEARCHER VOICE
What Researchers Say
Feedback from an experimental psychology researcher on understanding the design and the reporting expectations, and on a repeated or paired structure handled from the start.
“What worked for me was that I did not have to translate the whole Experimental Psychology context before discussing the statistics. They understood the study design, the usual data problems, and the reporting expectations. We were particularly concerned about the repeated or paired structure of the measurements, and that was dealt with properly from the start.”
HOW WE ASSESS YOUR PROJECT
We Review Your Actual Research
Your assessment is based on a review of your actual research materials and analytical requirements, not a generic project category.
You Submit Your Project Details
Share whatever is available at the current stage, such as the research aim, protocol or proposal, dataset, variable list, manuscript or statistical requirements.
We Review Your Research and Define the Scope
Your statistically trained Research Coordinator reviews the materials to understand the study design, research questions, data structure, analytical requirements and expected deliverables. If clarification is needed, additional documents or information can be requested before the assessment is finalized.
Your Analysis Is Completed and Delivered
Once the scope and fixed project fee are agreed, the in-house statistician completes the work and the Research Coordinator remains the communication point through delivery, with the analysis, deliverables and timeline as agreed.
SOFTWARE
Statistical Software for Experimental Psychology Research
Experimental analyses are run in the software your laboratory already uses, with syntax or scripts provided where applicable.
Statistical Support Within Your Research OnlineSPSS provides the agreed methodological, analytical, and statistical reporting support. The researcher or research team retains the scientific question, substantive interpretation, conclusions, and overall responsibility for the research.
FREQUENTLY ASKED QUESTIONS
Questions About Statistical Analysis for Experimental Psychology
Short answers to what researchers ask before sharing an experiment, from ANOVA and mixed models to power, preregistration, confidentiality and the assessment.
What statistical methods are used in experimental psychology?
Experimental psychology most often uses factorial ANOVA with planned contrasts for balanced data, mixed-effects models for trial-level data, logistic models for accuracy, and effect sizes with confidence intervals, chosen by the design and by how many observations each participant contributes. Preregistered and replication studies add equivalence or Bayesian estimates for null results. Our Statistical Methods pages show how each family is applied.
Is ANOVA still appropriate for my experiment?
A repeated-measures or mixed ANOVA is appropriate when each participant contributes one value per cell and the design is balanced, and a mixed-effects model is preferable with many trials, missing cells or stimuli sampled from a larger set. With unequal cell sizes, the type of sums of squares changes the main-effect tests, so the choice is stated, and sphericity is checked or corrected for within-participant factors.
How is power calculated for an interaction in a factorial design?
Power is calculated for the effect the hypothesis predicts, which in a factorial design is usually the interaction, not a main effect. We use the smallest effect of interest and plausible variance components, simulate the design for mixed models, and report which effect sizes the sample could detect. Effect sizes from published studies are treated with caution because they are often inflated.
Can you prepare the analysis plan for a preregistration?
Yes. We write the analysis section of the plan: the primary model, the confirmatory tests and how multiple outcomes are handled, the exclusion rules and the power calculation. After data collection, the analysis follows the plan and each departure from it is reported, so a reader can tell confirmatory from exploratory results.
How is confidential research data handled?
Study datasets, protocols and analysis files are handled confidentially, and project information is not published. An NDA is available where required and can be arranged before sensitive materials are shared. Read more about confidentiality and data security.
What happens when I request an assessment, and how long does it take?
You share your research question, design and relevant materials, and a statistically trained Research Coordinator reviews the request and responds the same day. The analysis, deliverables, timeline and one fixed project fee are then confirmed before any work begins, so the timeline follows from the agreed scope.
Still have a question? Find more answers in our full FAQ.
NEXT STEP
Discuss Your
Experimental Psychology Research With Us
Share your research question, design and available data, even if only a design sketch or pilot data exist. We review how participants and stimuli are structured and confirm the approach before any work begins.
No Commitment Required
Strictly Confidential
Fixed Project Fee
In-House PhD-Level Statisticians
Same-Day Response
Serving Researchers Since 2014