Diagnostic Accuracy Study for a
Caries Detection Device

RESEARCH QUESTION

What the Study Set Out to Ask

This diagnostic accuracy study example follows a dental research team that is evaluating a handheld optical device meant to flag early caries on chewing surfaces. The team wants to know how well the device reading agrees with a reference standard in the patients who would actually be examined in a dental clinic, and whether it could support a clinician in deciding where to look more closely.

The question is about accuracy: how often the device flags surfaces that have the disease and how often it stays quiet on surfaces that do not, with the uncertainty around each. It is not about whether using the device improves treatment outcomes, which needs a different study. The plan fixes the device threshold, the reference standard and the unit of analysis before any readings are compared.

STUDY DESIGN AND DATA

Surfaces Nested Within About 200 Patients

A prospective study of about 200 consecutive patients attending a dental clinic, with about 1,100 eligible tooth surfaces read by the device and by a reference standard of clinical examination and bitewing radiographs, judged by a panel blind to the device result. The device operators are blind to the panel, and both readings are recorded before they are compared. The dataset holds one row per surface, with patient, tooth and surface identifiers, the device reading, the reference result, any uninterpretable readings and patient characteristics.

METHOD AND ASSUMPTIONS

Why Clustering and Prevalence Matter

Sensitivity and specificity are estimated at the device threshold fixed in the plan, with confidence intervals that account for several surfaces from one patient, because surfaces in the same mouth are not independent observations. Treating 1,100 surfaces as 1,100 independent units gives intervals that are too narrow. The intervals come from resampling patients, and a patient-level estimate, where a patient counts as positive when any surface has disease, is reported beside the surface-level one because clinical decisions are often made for the patient. The plan also states that the reference standard is imperfect and discusses how that may bias the estimates.

Predictive values depend on how common the disease is, so the prevalence of caries in the study is not carried over to other settings: positive and negative predictive values are reported at the study prevalence and at stated lower and higher prevalences typical of settings where the device could be used. The ROC curve over the continuous device readings shows the trade-off between sensitivity and specificity, with the area under it and its interval, and any threshold chosen from this study is labeled optimistic because it is chosen on the same data. Uninterpretable or missing readings are counted and reported, not dropped.

OUTPUT AND INTERPRETATION

How the Results Are Read

The output has the participant flow diagram that STARD asks for, the cross-tabulation of device result against reference standard, sensitivity and specificity with 95% confidence intervals at surface and patient level, predictive values at the stated prevalences, and the ROC curve with its area and interval. Estimates are also shown by tooth type and by operator, to see where accuracy differs.

Sensitivity analyses treat uninterpretable readings as positive, as negative and as excluded, and use an alternative reference standard that counts only lesions reaching dentin. The interpretation states the accuracy in this patient spectrum and against this reference standard, and notes that estimates may differ in other populations or with other operators. It makes no claim about clinical benefit, which would need a study of patient outcomes.

REPORTING

What the Researcher Received

A written report structured to the STARD 2015 checklist, with statistical wording for the methods, matched to the output, for the team to check and incorporate. The research question, clinical interpretation and conclusions remain with the research team.