Item 8 — Inclusion and exclusion criteria

Specify inclusion and exclusion criteria, such as location, dates, patient-care setting, demographics (e.g., age, sex, race), pertinent follow-up, and results from prior tests. Define how, where, and when potentially eligible participants or studies were identified. Indicate whether a consecutive, random, or convenience series was selected.

Explanation and Elaboration: Understanding the composition of a dataset is key to evaluating the validity and transferability of machine learning models based on the dataset. In addition to explicit inclusion and exclusion criteria, which have obvious impact on dataset composition, any sampling procedure must be described as this can influence the composition. Sampling methods other than consecutive or fully randomized samples often introduce biases into a dataset.

Examples
The inclusion criteria were as follows: (a) imaging performed using a 3-T MRI system and (b) MRI fusion US-guided targeted biopsies were performed for MRI-suspicious lesions, followed by systematic biopsies. We excluded 262 patients from the analysis because of lack of biopsy results; 65 patients because of a longer than 2-week interval between screening and pathology; 551 patients because of a history of previous treatment for csPCa, such as antihormonal therapy, radiation therapy, focal therapy, cryotherapy, or surgery before MRI; 108 patients because of duplicate cases; and 76 patients because of incomplete sequences. Data from 2203 consecutive patients from Cs 1–3 were finally used as external testing set 1 to evaluate the performance of the DL model in a real clinical setting. Among these 2203 patients, 29.6% (652 of 2203) had rectal artifacts with varying severity (232, 222, and 198 patients had mild, moderate, and severe artifacts, respectively). Patients without visible artifacts (1551 of 2203) formed external testing set 2. To assess the impact of RAN interference on the DL model, subtle, invisible rectal artifact–like noises were intentionally introduced into external testing set 2 (referred to as external testing set 2*) to conduct a rectal artifact–pattern adversarial noise attack experiment. [Materials and methods, “patients” section (21), CC-BY 4.0]
The study was designed as a multicenter study of detection accuracy and feasibility on a retrospective cohort including all screening mammograms in the Region of Southern Denmark between August 4, 2014, and August 15, 2018, as previously reported…. In brief, a study population of 272008 consecutive screening mammograms was retrieved, including original binary assessments of normal or abnormal (recall) by breast radiologists through independent, blinded double reading with arbitration for disagreements. Exclusion criteria were insufficient image quality, missing images, lack of follow-up, or image data types not supported by the AI. [Materials and methods “Study Design and Sample” section (22), CC-BY 4.0]
Mammography and pathology data records for 915,334 screening and diagnostic examinations from 94,656 patients were obtained from a single site at US Dataset 1, acquired from 2008 to 2017. From this dataset, digital breast tomosynthesis (DBT) examinations and those in patients with breast implants were removed. This process resulted in a total of 117,844 screening examinations from 38,456 patients with a mean age of 57.4 years (standard deviation 10.2). The examinations were acquired on two scanner models: Hologic Selenia (HSE) (39,710 examinations, 33.7%) and Hologic Selenia Dimensions (SED) (78,134 examinations, 66.3%). The examinations were interpreted by one of 11 radiologists with subspecialties in breast imaging and experience ranging from 2 to 30 years. Among the examinations, 979 (0.8%) were cancer-positive confirmed by biopsy, 78 (0.07%) were interval cancers, and 1,638 (1.4%) were benign confirmed by biopsy. In addition to the screening examinations, 25,749 diagnostic examinations available from the same data source were utilized to augment training. These diagnostic examinations were gathered such that all images had image-level annotations localizing any findings provided retrospectively by a radiologist with breast imaging subspecialty and magnification factors no larger than 1.1, therefore removing the magnified views. These diagnostic examinations added 2,555 examinations that were cancer positive confirmed by biopsy. [Appendix S1, Data Acquisition and Labeling section (23), CC-BY 4.0]