Developing robust artificial intelligence (AI) models that generalize well to unseen datasets is challenging and usually requires large and variable datasets, preferably from multiple institutions. In federated learning (FL), a model is trained collaboratively at numerous sites that hold local datasets without exchanging them. So far, the impact of training strategy, i.e., local versus collaborative, on the diagnostic on-domain and off-domain performance of AI models interpreting chest radiographs has not been assessed.
A recent study published in Scientific Reports titled ‘Enhancing domain generalization in the AI-based analysis of chest radiographs with federated learning’ aimed to fill this gap by evaluating the diagnostic performance of AI models trained using different strategies, network architectures, dataset sizes, and diversity. The study utilized 610,000 chest radiographs from five institutions across the globe to assess the effectiveness of federated learning in improving the accuracy and reliability of AI models for chest radiograph analysis.
The results of the study revealed some interesting findings. Larger datasets did not necessarily lead to significant performance gains when using federated learning. In fact, in some cases, the performance even decreased. On the other hand, smaller datasets showed marked improvements with the collaborative training approach. This suggests that the size of the training data influenced on-domain performance, with larger datasets not always translating to better results.
However, when it comes to off-domain performance, which refers to the model’s ability to generalize to datasets from different institutions, training diversity played a more crucial role. AI models trained collaboratively across diverse external institutions consistently outperformed models trained locally. This highlights the potential of federated learning in leveraging data diversity to enhance the reliability and accuracy of AI models for off-domain tasks.
The study also compared the performance of different network architectures, specifically convolutional and transformer-based models. By utilizing the ResNet50 and Vision Transformer (ViT) base models, the researchers examined the potential influence of the underlying architecture on the AI models’ performance.
Overall, the findings of this study demonstrate the potential of federated learning to enhance the privacy, reproducibility, and off-domain reliability of AI models. By training collaboratively across different institutions without sharing sensitive data, federated learning allows for the development of more robust and generalizable AI models. The optimized performance of these models has the potential to significantly impact healthcare outcomes, contributing to improved diagnostic accuracy and patient care.
As AI continues to revolutionize the field of medical imaging, it is crucial to explore innovative approaches like federated learning that address the challenges of data privacy while maximizing the potential for AI model generalization. The findings of this study pave the way for further research and development in this domain, opening up new possibilities for optimizing AI-powered healthcare solutions.
