A Multi-scale Capsule Network for Improving Diagnostic Generalizability in Breast Cancer Diagnosis Using Ultrasonography

Recently, deep learning has shown promising results in medical image processing. However, computer-aided diagnosis (CAD) systems based on deep learning still struggle for the real-world deployment, due to its low generalizability and reliability. It is essential to improve the generalization performance to enable them to be used routinely in clinical practice. In this paper, we propose a capsule network with a multi-scale setting to achieve better generalization performance in the differential diagnosis of breast tumors using ultrasonography. The proposed network utilizes a Gaussian pyramid to learn multi-scale features of breast tumors and dynamic routing to improve its robustness against image quality with severe noises. To evaluate the generalizability of the proposed method, we collected breast ultrasound images from 4 different hospitals and used one dataset from 1 hospital as a train set and the rest as external validation sets. We compared the classification performance with other networks, which were employed for the ultrasound diagnosis in previous studies, on the external validation sets. We also conducted additional experiments: feature space visualization and robustness evaluation study with respect to the image noise. Our model showed better classification results than other networks, such as GoogLeNet and Inception-v3, in the external validation. Experimental results also indicate that the proposed network can learn more robust and noise-invariant features from breast ultrasound imaging.