Bifurcations of Renormalization Dynamics in Self-organizing Neural Networks

Peter Tino, M Ishikawa, K Doya, H Miyamoto, T Yamakawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Self-organizing neural networks (SONN) driven by softmax weight renormalization are capable of finding high quality solutions of difficult assignment optimization problems. The renormalization is shaped by a temperature parameter - as the system cools down the assignment weights become increasingly crisp. It has been reported that SONN search process can exhibit complex adaptation patterns as the system cools down. Moreover, there exists a critical temperature setting at which SONN is capable of powerful intermittent search through a multitude of high quality solutions represented as meta-stable states. To shed light on such observed phenomena, we present a detailed bifurcation study of the renormalization process. As SONN cools down, new renormalization equilibria emerge in a strong structure leading to a complex skeleton of saddle type equilibria surrounding an unstable maximum entropy point, with decision enforcing “one-hot” stable equilibria. This, in synergy with the SONN input driving process, can lead to sensitivity to annealing schedules and adaptation dynamics exhibiting signatures of complex dynamical behavior. We also show that (as hypothesized in earlier studies) the intermittent search by SONN can occur only at temperatures close to the first (symmetry breaking) bifurcation temperature.
Original languageEnglish
Title of host publicationNeural Information Processing
Subtitle of host publicationLecture Notes in Computer Science
PublisherSpringer
Pages405-414
Number of pages10
Volume4984
ISBN (Print)978-3-540-69154-9
DOIs
Publication statusPublished - 1 Jan 2008
EventNeural Information Processing: Proceedings of 14th International Conference (ICONIP 2007), Revised Selected Papers, Part I -
Duration: 1 Jan 2008 → …

Conference

ConferenceNeural Information Processing: Proceedings of 14th International Conference (ICONIP 2007), Revised Selected Papers, Part I
Period1/01/08 → …

Fingerprint

Dive into the research topics of 'Bifurcations of Renormalization Dynamics in Self-organizing Neural Networks'. Together they form a unique fingerprint.

Cite this