Learning and inference in hierarchical models with singularities

When we infer the underlying rule which generates a large amount of data, we assume a family of hierarchical statistical models and estimate an appropriate model and its parameters. In this case, the parameter space of the model usually includes singularities, and interesting phenomena, different from those appearing in conventional inference theory, are observed. In this paper, we review the studies of singular models in learning and inference which are being extensively developed in Japan, and elucidate the mechanisms of strange behavior by using simple models.