Item 40 — Study limitations

Identify the study’s limitations, including those involving the study’s methods, materials, biases, statistical uncertainty, unexpected results, and generalizability. This discussion should follow succinct summarization of the results with appropriate context and explanation of how the current work advances our knowledge and the state of the art.

Explanation and Elaboration: Think broadly about the potential sources of bias and error in your investigation. What are the assumptions inherent in the methods, how likely are those assumptions to be incorrect, and what would the impact be if they were? Intelligent discussion of the limitations that are potentially unique to a study adds greatly to a manuscript; pro forma listing of the limitations that are inherent and common to all investigations in a field adds little.

Examples
While our AI system shows potential to impact LCS worldwide, the studies performed and the system both have limitations to consider. As in other recent studies, although realistic environments were used, the studies were retrospective on an enriched dataset. Similarly, this study did not include the full breadth of potential data that a radiologist may access in the electronic health record, such as smoking history. Any impact of having such variables available will need to be evaluated in future work. As our analysis principally focused on the performance of experienced radiologists on challenging cases, the impact of AI assistance on less experienced radiologists also needs to be examined in follow-up work. As positive cancer cases are more difficult to obtain, a total of 50 patients outside of NLST that were positive for cancer were used for evaluation. These cases were from different patient populations and involved selecting the earliest study within 2 years prior to biopsy. Subject to study feasibility, future studies could improve upon these limitations of retrospective enriched studies. [Discussion (39), CC BY 4.0]
Some limitations of the present study must be acknowledged. First, manual delineation of the brainstem and ventricular regions was performed by a single rater, although with more than 30 years of experience in the field of neuroradiology. Second, PD and PSP cases were not pathologically confirmed. Moreover, we considered only patients with PSP–Richardson syndrome, and further studies are needed to investigate the robustness and usefulness of our approach in other clinical subtypes, such as PSP parkinsonism, and/or considering patients at early disease stages. Third, during the models’ training phase, we did not adopt early stopping criteria and cross-validation. Although this can lead to overfitting issues, the optimal segmentation performance displayed by DL-based models in the independent external test datasets indicates their generalizability on new MR images. Fourth, a min-max intensity normalization method was used to transform the image intensity range of MR images used as input to DL-based models. While this approach is commonly used to transform the intensity histogram to a 0–1 range without affecting the image itself, it may be influenced by outliers in the raw intensity values. Finally, although our results showed clear-cut differences between patients with PD and patients with PSP in MP, MRPI, and MRPI 2.0, it is important to highlight that these indices are computed based on brainstem and ventricular morphologic features that are significantly correlated. A high degree of collinearity may increase the risk of observing statistically significant results by chance and make the determination of the individual contribution of each brain region to the indices challenging. [Discussion (36), CC BY 4.0]