Skip-StyleGAN: Skip-Connected Generative Adversarial Networks for Generating 3D Rendered Image of Hand Bone Complex

Computed tomography (CT) is commonly used for fracture diagnosis because it provides accurate visualization of shape with 3-dimensional(3D) structure. However, CT has some disadvantages such as the high dose of radiation involved in scanning, and relatively high expense compared to X-ray. Also, it is difficult to scan CT in the operation room despite it is necessary to check 3D structure during operation. On the other hand, X-ray is often used in operating rooms because it is relatively simple to scan. However, since X-ray only provides overlapped 2D images, surgeons should rely on 2D images to imagine 3D structure of a target shape. If we can create a 3D structure from a single 2D X-ray image, then it will be clinically valuable. Therefore, we propose Skip-StyleGAN that can efficiently generate rotated images of a given 2D image from 3D rendered shape. Based on the StyleGAN, we arrange training sequence and add skip-connection from the discriminator to the generator. Important discriminative information is transferred through this skip-connection, and it allows the generator to easily produce an appropriately rotated image by making a little variation during the training process. With the effect of skip-connection, Skip-StyleGAN can efficiently generate high-quality 3D rendered images even with small-sized data. Our experiments show that the proposed model successfully generates 3D rendered images of the hand bone complex.