Dynamics of the adaptive natural gradient descent method for soft committee machines

Masato Inoue, Hyeyoung Park, Masato Okada

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

The learning efficiency of a simplified version of adaptive natural gradient descent (ANGD) for soft committee machines was evaluated. Statistical-mechanical techniques, which extract order parameters and make the stochastic learning dynamics converge towards deterministic at the large limit of the input dimension N [1,2], were employed. ANGD was found to perform as well as natural gradient descent (NGD). The key condition affecting the learning plateau in ANGD were also revealed.

Original languageEnglish
Article number056120
Pages (from-to)056120-1-056120-14
JournalPhysical Review E
Volume69
Issue number5 1
StatePublished - May 2004

Cite this