Item 5 — Prospective or retrospective study

Indicate if the study is prospective or retrospective. Evaluate AI models in a prospective setting, if possible.

Explanation and Elaboration: Prospective evaluation is preferred because it evaluates the model in a setting more similar to the expected clinical use of the model. Retrospective evaluation is usually faster and less expensive, but more likely to be subject to confounding and bias. In complex multimodal clinical datasets it is often difficult to establish what data would have been available at the time the model would have run – for example a single note or report may include data obtained both before and after an imaging study was performed. Where it is not feasible to perform prospective evaluation, there should generally be discussion of potential biases and confounders due to retrospective evaluation in the discussion section.

Examples
This retrospective study was classified as nonhuman subjects research by the institutional review board of the University of Arkansas for Medical Sciences (UAMS) (under institutional review board approval no. 262472), and formal review was waived. Three retrospective datasets were used to develop and test the proposed model: a local dataset to train and test models and two external datasets to further assess performance. [Material and Methods (14), CC-BY 4.0]
We prospectively recruited patients referred for clinical CMR exam for myocardial scar evaluation. Patients were approached before the CMR exam for willingness to allow additional research scanning at the end of their clinical exam, which added < 5 min to their clinical exam. [Material and Methods (15), CC-BY 4.0]