Intro

Can machines predict deterministic yet chaotic system? Can we distinguish chaotic structure from noisy data? Can machines recognize patterns from highly patternless system?
To answer these questions, one needs to understand and define what patternless is. Many mathematicians defined the notion of patternless as chaos. Good definitions

Notations and Prerequisites

def) x is fixed point of f if $ f(x) = x $

Proposition

Let p be a fixed point of f. If $|f’(p)| < 1$, then there exists a neighbourhood U of p such that, for all x in U, $\lim_{n \rightarrow \infty} f^{n}(x) = p.$ Such fixed point p is called attracting fixed point and the set U is a stable set. When $|f’(p)| > 1$, then p is called repelling fixed point. When $|f’(p)| = 1$, p is saddle point.

Ergodic Theory

“When can we expect the average of the data over time in the first experiment to be the same as the average of the data over the replicates at a fixed time?”
$ (X,F,\mu) $ is probability space. $ f: X \rightarrow X $ is measure preserving with respect to $ \mu $ if for any A in F, $ \mu(f^{-1}(A)) = \mu(A) $
A in F is invariant if $ f^{-1}(A) = A $ for invertible transformation f and $ f^{t}(A) = A $ for any $ t > 0 $.
f is ergodic if for any invariant subset A in F, $ \mu (A) = 0 or 1 $.
So if f is ergodic, the sample path is not trapped in the subset of domain.

Ergodic Theorem

If f is measure preserving and ergodic on $ (X,F, \mu) $, and Y is any random variable ( $ E[|Y|] < \infty $) then $ \frac{1}{n} \sum_{i=1}^{n} Y(f^{i}(x_{0})) = E_{P}(Y) $ almost surely


Chaos

How can we define the measure of paternless?
Most definitions of Chaos involve these two.

  1. Sensitivity to inditial conditions : Small perturbation can induce big difference in results.
  2. Recurrence / Mixing / Topological Transitivity : Some paths of the dynamical system generated by f eventually visit all regions of D.

    Formally, $f : D \rightarrow D $ is sensitive to intial conditions if there exists $\delta > 0$ such that for any x in D and any neighborhood V of x, there exists y in V and $n \geq 0$ such that $|f^{n}(x) - f^{n}(y)| > delta$
    $f:D \rightarrow D$ is topologically transtive if for any pair of open sets U, V in D there exists $ n > 0 $ such that $f^{n}(U) \cap V$ is not empty.


Measure of Chaos

1. Lyapunov exponents

definition : $ \lambda (x_{0}) = \lim_{n \rightarrow \infty} \frac{1}{n} log[ | \frac{dx_{n}}{dx_{0}} |] $
To estimate lyapunov exponents from noisy dataset, refer Nychka, McCAffrey, Ellner and Gallant.

2.

Reference

Statistics, Probability and Chaos (Berliner, 1992) Chaos and Intrinsic Unpredictability (Heninger and Johnson, 2023)