ILLUSTRATIVE EXAMPLE
Biomarker Study for a
Disease Subtype Classifier
A penalized logistic classifier tuned in nested cross-validation, with a batch confounding check, AUC with a confidence interval and calibration, reported to TRIPOD+AI for about 150 plasma samples.
This example does not describe a real client or project. Every engagement is confidential, during the project and after it ends.
RESEARCH QUESTION
What the Study Set Out to Test
This biomarker study example follows a rheumatology research team that has measured plasma proteins by mass spectrometry in patients with two subtypes of a chronic disease. They want to know whether a small panel of proteins can distinguish the subtypes, and how well it would do in patients not used to build it.
The question is one of discrimination and calibration, not of cause or of a ready clinical test. The samples of one subtype were run in earlier weeks than the other, so a difference between subtypes could partly be a difference between run weeks. The plan says how far the design allows a conclusion before any model is fitted.
STUDY DESIGN AND DATA
A Case-Comparison Study With Batch Structure
A biomarker study with about 150 plasma samples, split between two disease subtypes. The dataset holds the abundance of several hundred proteins per sample, the subtype, run week, age and sex, and many abundances are missing because they fell below the detection limit. Run week overlaps with subtype, which is the main threat to the validity of the classifier.
- Design: case-comparison biomarker study, samples measured in batches
- Outcome: disease subtype (two classes), predicted from protein abundances
- Software requested: Python, with scripts and an environment file
METHOD AND ASSUMPTIONS
Why Validate Inside Nested Cross-Validation
With far more proteins than samples, an unpenalized model fits noise. A penalized logistic regression shrinks the coefficients and selects a sparse panel, and the penalty is tuned in an inner loop of cross-validation while performance is estimated in an outer loop. Selection, scaling and handling of missing values are all done inside the loops, so the held-out samples never influence the model, which is the main source of optimism in published classifiers.
Before any model is fitted, an ordination colored by run week shows how far batch separates the groups, and the batch-by-subtype table is reported. Missing values below detection are handled by a rule fixed in advance and compared with alternatives. The study is described as development with internal validation, because no external cohort is available, and the report says so in the terms of the TRIPOD+AI items.
OUTPUT AND INTERPRETATION
What the Analysis Delivers
The output has the area under the ROC curve from the outer loop with a 95% confidence interval, the ROC curve itself, and a calibration plot and slope that show whether predicted probabilities can be read as probabilities. The selected proteins are listed with their coefficients and the frequency with which each was selected across folds, so the stability of the panel is visible, not just its performance.
Sensitivity analyses repeat the estimate with run week included as a covariate and within the weeks where both subtypes appear. The write-up states what the estimates support: discrimination in these samples, limited by batch overlap and by internal validation only. It does not describe the panel as a validated clinical decision tool, and it names the external validation that would be needed.
REPORTING
What the Researcher Received
A written report structured to the TRIPOD+AI checklist for prediction model studies, with statistical wording for the methods, matched to the output, for the team to check and incorporate, and the validation scheme described so a reviewer can check it for leakage. The research question, biological interpretation and conclusions remain with the research team.