Modeling of Chaotic Processes by Means of Antisymmetric Neural ODEs

Vasiliy Ye. Belozyorov, Danylo V. Dantsev

Анотація


The main goal of this work is to construct an algorithm for modeling chaotic processes using special neural ODEs with antisymmetric matrices (antisymmetric neural ODEs) and power activation functions (PAFs). The central part of this algorithm is to design a neural ODEs architecture that would guarantee the generation of a stable limit cycle for a known time series. Then, one neuron is added to each equation of the created system until the  approximating properties of this system satisfy the well-known Kolmogorov theorem on the approximation of a continuous function of many variables. In
addition, as a result of such an addition of neurons, the cascade of bifurcations that allows generating a chaotic attractor from stable limit cycles is launched. We also consider the possibility of generating a homoclinic orbit whose bifurcations lead to the appearance of a chaotic attractor of another type. In conclusion, the conditions under which the found attractor adequately simulates the chaotic process are discussed. Examples are given.


Ключові слова


system of ordinary autonomous differential equations; neural network; antisymmetric matrix; power activation function; Lyapunov stability; limit cycle; homoclinic orbit; strange non-chaotic attractor, search algorithm

Повний текст:

PDF (English)

Посилання


S. Haykin, Neural Networks. A Comprehensive Foundation, Second Edition, Pearson Education, Prentice Hall, 2005.

E. M. Izhikevich, Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting, The MIT Press Cambridge, Massachusetts, London, England, 2007.

B. Cessac, A view of neural networks as dynamical systems, arXiv preprint arXiv: 0901.2203v2 [nlin.AO], (2019), 1–62.

I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, The MIT Press Cambridge, Massachusetts, London, England, 2017.

D. Krotov, J. Hopfield, Large associative memory problem in neurobiology and machine learning, arXiv preprint arXiv:2008.06996v1[q-bio. NC], (2020), 1–8.

Q. Li, Dynamical Systems and Machine Learning, Summer School, Peking University, 2020.

C. - L. Yang, C. S. Suh, A general framework for dynamic complex networks, Journal of Vibration Testing and System Dynamics, 5 (1), (2021), 87–111.

S. Sonoda, N. Murata, Neural network with unbounded activation functions is universal approximator, Applied and Computational Harmonic Analysis, 43, 2017, 233–268.

R. T. Q. Chen, Y. Rubanova, J. Bettencourt, D. Duvenaud, Neural ordinary differential equations, arXiv preprint arXiv:1806.07366v5[cs.LG], (2019), 1–18.

R. T. Q. Chen, D. Duvenaud, Neural networks with cheap differential operators, arXiv preprint arXiv:1912.03579v1[cs.LG], (2019), 1–11.

J. Kelly, J. Bettencourt, M. Johnson, D. Duvenaud, Learning differential equations that are easy to solve, arXiv preprint arXiv:2007.04504v2[cs.LG], (2020), 1–18.

J. Jia, A. R. Benson, Neural jump stochastic differential equations, arXiv preprint arXiv:1905.10403v3[cs.LG], (2020), 1–14.

B. Chang, M. Chen, E. Haber, E.D. Chi, Antisymmetric RNN: A Dynamical System Viev on Recurrent Neural Networks, In conference ICLR, May 6 - May 9, New Orleans, Louisiana, USA, (2019), 1–15.

J. Lorraine, P. Vicol, D. Duvenaud, Optimizing millions of hyperparameters by implicit differentiation, arXiv preprint arXiv:1911.02590v1[cs.LG], (2019), 1–18.

S. Massaroli, M. Poli, J. Park, A. Yamashita, H. Asama, Dissecting neural ODEs, arXiv preprint arXiv:2002.08071v3[cs.LG], (2020), 1–23.

A. F. Queiruga, N. B. Erichson, D. Taylor, M.W. Mahoney, Continuousin- depth neural networks, arXiv preprint arXiv:2008.02389v1[cs.LG], (2020), 1–29.

V. Ye. Belozyorov, D. V. Dantsev, S. A. Volkova. On Equivalence of Linear Control Systems and Its Usage to Verification of the Adequacy of Different Models for A Real Dynamic Process, Journal of Optimization, Differential Equations and Their Applications (JODEA), 28(1)(2020), 43–97.

N. A. Magnitskii, Universal theory of dynamical chaos in nonlinear dissipative systems of differential equations, Commun. Nonlinear Sci. Numer. Simul., 13, (2008), 416–433.

N. A. Magnitskii, Universality of transition to chaos in all kinds of nonlinear differential equations, Nonlinearity, Bifurcation and Chaos – Theory and Applications, Chapter 6. Intech, (2012), 133–174.

N. A. Magnitskii, Bifurcation theory of dynamical chaos, Chaos Theory, Chapter 11. Intech, (2018), 197–215.

V. Ye. Belozyorov, Universal approach to the problem of emergence of chaos in autonomous dynamical systems, Nonlinear Dynamics, 95 (1), (2019), 579–595.

N.-E. Tatar, Hopfield neural networks with unbounded monotone activation functions, Advances in Artificial Neural Systems, Hindawi Publishing Corporation, 2012, 2012, 571358-1–5.

S. M. Richards, F. Berkenkamp, A. Krause, The Lyapunov neural network: adaptive stability certification for safe learning of dynamical systems, arXiv preprint arXiv:1808.009241v2[cs.SY], (2018), 1–11.

E. Haber, L. Ruthotto, Stable Architectures for Deep Neural Networks, arXiv preprint arXiv: 1705.03341v1[cs.LG], (2019), 1–23.

V. Ye. Belozyorov, D. V. Dantsev, Stability of neural ordinary differential equations with power nonlinearities, Journal of optimization, differential equations and their applications (JODEA), 28 (2), (2020), 21–46.

V. Ye. Belozyorov, Ye. M. Kosariev, M. M. Pulin, V. G. Sychenko, V. G. Zaytsev. A new mathematical model of dynamic process in direct current traction power supply system, Journal of Optimization, Differential Equations and Their Applications (JODEA), 27(1) (2019), 21–55.

Z. Wang, J. Liang, Y. Liu, Mathematical problems for complex networks, Mathematical Problems in Engineering, Hindawi Publishing Corporation, 2012, (2012), ID 934680-1–5.

A. N. Gorban, V. L. Dunin-Barkovsky, A. N. Kirdin, E. M. Mirkes, A.Yu. Novokhodko, D. A. Rossiev, S. A. Terekhov, M. Yu. Senashova, V. G. Tsaregorodtsev, Neuroinformatics, Nauka, Novosibirsk, 1998 (in rus).

H. Zhang, X. Gao, J. Unterman, T. Arodz, Approximation capabilities of neural ordinary differential equations, arXiv preprint arXiv: 1907.12998v1[cs.LG], (2019), 1–11.

Q. Li, T. Lin, Z. Shen, Deep learning via dynamical systems: an approximation perspective, arXiv preprint arXiv:1912.10382v1[cs.LG], (2019), 1–30.

H. K. Khalil, Nonlinear Systems – 2nd Edition, (Prentice Hall/New-Jersy), 1996.

V. Ye. Belozyorov, A novel search method of chaotic autonomous quadratic dynamical systems without equilibrium points, Nonlinear Dyn., 86, (2016), 835–860.

V. Ye. Belozyorov, S. A. Volkova, Role of logistic and Ricker’s maps in appearance of chaos in autonomous quadratic dynamical systems, Nonlinear Dyn., 83, (2016), 719–729.

V. Ye. Belozyorov, On novel conditions of chaotic attractors existence in autonomous polynomial dynamical systems, Nonlinear Dyn., 91, (2018), 2435–2452.

V. Ye. Belozyorov, On existence of homoclinic orbits for some types of autonomous quadratic systems of differential equations, Applied Mathematics and Computation, 217(9), (2011), 4582–4595.

S. Jafary, J. C. Sprott, S. Mohammad Reza Hashemi Golpayegani, Elementary quadratic chaotic flows no equilibria, Physics Letters A, 377, (2013), 699–702.

X. Wang, G. Chen, A gallery Lorenz-like and Chen-like attractors, Int. J. Bifurc. Chaos, 23(4), (2013), ID 1330011-1–20.

D. Dantsev, A novel type of chaotic attractor for quadratic systems without equilibriums, Int. J. Bifurc. Chaos, 28(1), (2018), ID 1850001-1–7 .




DOI: http://dx.doi.org/10.15421/142201

Посилання

  • Поки немає зовнішніх посилань.



Індексування журналу

Журнал розміщено у наукометричних базах, репозитаріях та пошукових системах:

                 


Адреса редколегії: 49050, Україна, Дніпровський національний університет імені Олеся Гончара, вул. Козакова 18, корп. 14, механіко-математичний факультет, д-р фіз.-мат. наук, проф. Когут П.І. 

email: p.kogut@i.ua

www.dnu.dp.ua


Free counters! 

Лицензия Creative Commons
Це видання має доступ за ліцензією Creative Commons «Attribution» («Атрибуция») 4.0 Всемирная.


Open Science in Ukraine - website development