TY - GEN
T1 - Geometrical singularities in the neuromanifold of multilayer perceptrons
AU - Amari, Shun Ichi
AU - Park, Hyeyoung
AU - Ozoki, Tomoko
PY - 2002
Y1 - 2002
N2 - Singularities are ubiquitous in the parameter space of hierarchical models such as multilayer perceptrons. At singularities, the Fisher information matrix degenerates, and the Cramer-Rao paradigm does no more hold, implying that the classical model selection theory such as AIC and MDL cannot be applied. It is important to study the relation between the generalization error and the training error at singularities. The present paper demonstrates a method of analyzing these errors both for the maximum likelihood estimator and the Bayesian predictive distribution in terms of Gaussian random fields, by using simple models.
AB - Singularities are ubiquitous in the parameter space of hierarchical models such as multilayer perceptrons. At singularities, the Fisher information matrix degenerates, and the Cramer-Rao paradigm does no more hold, implying that the classical model selection theory such as AIC and MDL cannot be applied. It is important to study the relation between the generalization error and the training error at singularities. The present paper demonstrates a method of analyzing these errors both for the maximum likelihood estimator and the Bayesian predictive distribution in terms of Gaussian random fields, by using simple models.
UR - http://www.scopus.com/inward/record.url?scp=84898934178&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84898934178
SN - 0262042088
SN - 9780262042086
T3 - Advances in Neural Information Processing Systems
BT - Advances in Neural Information Processing Systems 14 - Proceedings of the 2001 Conference, NIPS 2001
PB - Neural information processing systems foundation
T2 - 15th Annual Neural Information Processing Systems Conference, NIPS 2001
Y2 - 3 December 2001 through 8 December 2001
ER -