Add sofic.algebra: Krohn-Rhodes theory, holonomy decomposition, and DeDeo (2011) tools

CHANGELOG.md

@@ -135,6 +135,32 @@ brute-force reference or a closed-form value and has a regression test.
 
 ### New
 
+- `sofic.algebra`: algebraic automata theory for deterministic presentations.
+  `TransformationSemigroup` (lazy closure, shortest words, sink completion of
+  partial maps), Green's relations and maximal subgroups, `syntactic_monoid`
+  and `is_star_free`, and the Krohn-Rhodes holonomy decomposition
+  (`holonomy_decomposition`) with an explicit `CascadeProduct` whose
+  `emulates()` checks the covering. `krohn_rhodes_complexity` returns
+  `ComplexityBounds(lower, upper, exact)`, and `prime_decomposition` lists the
+  simple-group factors. `holonomy_information` splits `C_mu` across the
+  holonomy levels. DFAs, unifilar HMMs / ε-machines and sofic shifts gain
+  `transition_semigroup()` and `krohn_rhodes_complexity()`, and the shift
+  invariants gain `factor_syntactic_monoid` and
+  `factor_krohn_rhodes_complexity`. Wheeler-presentation failures now report
+  whether the language is star-free.
+- `sofic.algebra`, following DeDeo's "Effective Theories for Circuits and
+  Automata" (2011): `HolonomyDecomposition.effective_theory(k)` and
+  `CascadeProduct.truncate(k)` give coarse-grained machines on overlapping
+  superstates; `independent_random_maps` / `coupled_random_maps` /
+  `random_maps` rewrite probabilistic automata as random choices of
+  deterministic maps; `element_walk` and `classify_elements` follow the walk
+  over semigroup elements; and `walk_information` splits lost information into
+  irreversibility and dissipation. Also new: language varieties
+  (`is_piecewise_testable`, `is_locally_testable`, `is_definite`, ...),
+  `minimal_ideal` / `minimal_rank` / `is_synchronizing` /
+  `schutzenberger_group`, belief-map semigroups (`projective_semigroup`,
+  `mixed_state_semigroup`), and `is_full_transformation_monoid`, which makes
+  the complexity of `T_n` exact without enumerating it.
 - `sofic.generators.capacity`: capacity bounds for finite ε-transducers
   (unifilar finite-state channels). `EpsilonTransducer.feedback_capacity()`
   brackets the feedback capacity with Q-graph bounds (convex upper bound,

docs/algebra/algebra.rst

@@ -0,0 +1,39 @@
+.. algebra.rst
+.. py:module:: sofic.algebra
+
+*******
+Algebra
+*******
+
+The :mod:`sofic.algebra` package brings algebraic automata theory to every
+deterministic presentation in ``sofic``. A DFA, a unifilar automaton, a
+unifilar HMM or ε-machine, or a right-resolving shift presentation has one
+partial map of its states per symbol. Those maps generate a finite
+*transformation semigroup*, and the Krohn-Rhodes theorem breaks that
+semigroup into a cascade of simple groups and flip-flops
+:cite:`KrohnRhodes1965` :cite:`RhodesNehanivHirsch2010`.
+
+* :doc:`semigroups` covers transformation semigroups, Green's relations,
+  syntactic monoids, and star-freeness.
+* :doc:`holonomy` covers the holonomy decomposition as an explicit cascade
+  product that can be checked against the original machine.
+* :doc:`complexity` covers Krohn-Rhodes complexity bounds, prime factors,
+  how statistical complexity splits across the holonomy levels, and the
+  semigroup of an HMM's belief updates.
+* :doc:`effective_theories` covers truncating the cascade into coarse-grained
+  machines on overlapping superstates :cite:`DeDeo2011`.
+* :doc:`noise` covers probabilistic automata: rewriting them as random choices
+  of deterministic maps, the walk over semigroup elements, and how much
+  information is erased versus dissipated.
+
+The group-theoretic outputs need the optional ``sympy`` dependency, which the
+``sofic[symbolic]`` extra installs.
+
+.. toctree::
+   :maxdepth: 1
+
+   semigroups
+   holonomy
+   complexity
+   effective_theories
+   noise

docs/algebra/complexity.rst

@@ -0,0 +1,123 @@
+.. complexity.rst
+.. py:module:: sofic.algebra.complexity
+
+***********************
+Krohn-Rhodes Complexity
+***********************
+
+The *group complexity* of a finite semigroup is the least number of group
+levels, separated by aperiodic levels, in a cascade that emulates it
+:cite:`KrohnRhodes1965` :cite:`RhodesSteinberg2009`. It is zero exactly for
+aperiodic semigroups, which for syntactic monoids means star-free languages.
+It equals ``n - 1`` for the full transformation monoid ``T_n``. Whether it is
+computable in general is open; only complexity one is known to be decidable
+:cite:`HenckellRhodesSteinberg2010`.
+
+:func:`krohn_rhodes_complexity` therefore returns
+:class:`ComplexityBounds`:
+
+* The upper bound counts the holonomy levels with a nontrivial group.
+* The lower bound is ``0`` for an aperiodic semigroup and at least ``1``
+  otherwise. It is raised to ``k - 1`` when the elements that map some
+  ``k``-element image set into itself act on it as all of ``T_k``, because
+  complexity cannot increase under division.
+
+``exact`` is filled in when the two bounds meet. That happens for every
+aperiodic semigroup, for every semigroup with a single nontrivial holonomy
+level, and for ``T_n``. The sharper Rhodes-Tilson lower bounds are not
+implemented.
+
+.. ipython::
+
+   In [1]: from sofic.algebra import TransformationSemigroup, krohn_rhodes_complexity, prime_decomposition
+
+   In [2]: t4 = TransformationSemigroup({"t": (1, 0, 2, 3), "c": (1, 2, 3, 0), "e": (0, 0, 2, 3)})
+
+   @doctest
+   In [3]: krohn_rhodes_complexity(t4)
+   Out[3]: ComplexityBounds(lower=3, upper=3, exact=3)
+
+   @doctest
+   In [4]: prime_decomposition(t4).simple_group_orders()
+   Out[4]: (2, 3)
+
+:func:`prime_decomposition` lists the Jordan-Hölder factors of every
+holonomy group. These are the simple groups the Krohn-Rhodes theorem
+requires, and every simple group divisor of ``S`` divides one of them. It
+also reports whether flip-flops (resets) are needed. A nonabelian simple
+factor is reported by its order only, because order alone does not identify
+it.
+
+.. autofunction:: krohn_rhodes_complexity
+.. autoclass:: ComplexityBounds
+.. autofunction:: prime_decomposition
+.. autoclass:: PrimeDecomposition
+   :members:
+.. autoclass:: PrimeFactor
+
+Holonomy levels and statistical complexity
+==========================================
+
+.. py:module:: sofic.algebra.information
+
+Lifting each causal state to its holonomy coordinates turns the stationary
+state distribution into a joint distribution over the level variables
+``L_1, ..., L_h``. Distinct states get distinct coordinates, so
+``H[L_1, ..., L_h] = C_mu``. The chain rule then splits the statistical
+complexity into ``H[L_i | L_<i]``, the information resolved at each level,
+from the coarsest approximation at the top to individual states at the
+bottom. The split depends on the chosen lift; :func:`holonomy_information`
+uses the canonical one.
+
+.. note:: Original, uncited: the literature does not pair holonomy
+   coordinates with a stationary measure.
+
+.. ipython::
+
+   In [1]: from sofic.algebra import holonomy_information
+
+   In [2]: from sofic.examples import even_process
+
+   In [3]: info = holonomy_information(even_process())
+
+   @doctest
+   In [4]: [round(h, 6) for h in info.level_entropies]
+   Out[4]: [0.918296, 0.0]
+
+For the even process the top level, which separates the two causal states,
+carries all of ``C_mu``.
+
+.. autofunction:: holonomy_information
+.. autoclass:: HolonomyInformation
+
+Belief-map semigroups
+=====================
+
+.. py:module:: sofic.algebra.belief
+
+An observer of an HMM updates its belief by ``eta -> eta T^(x) / (eta T^(x) 1)``.
+:func:`projective_semigroup` enumerates the word matrices ``T^(w)`` up to a
+positive scalar, up to a cap, and returns the result as a transformation
+monoid, so the rest of :mod:`sofic.algebra` applies. If the enumeration
+closes, every initial belief has a finite orbit. Finiteness of matrix
+semigroups is decidable :cite:`MandelSimon1977` :cite:`Jacob1977`, but this
+is only a capped enumeration. It is also only a sufficient condition: the
+mixed-state presentation from one particular belief can be finite even when
+this semigroup is infinite. :func:`mixed_state_semigroup` takes the
+transition semigroup of the mixed-state presentation directly.
+
+.. ipython::
+
+   In [1]: from sofic.algebra import projective_semigroup
+
+   @doctest
+   In [2]: len(projective_semigroup(even_process()).states)
+   Out[2]: 7
+
+For a unifilar machine like the even process this matches the transition
+monoid. The simple nonunifilar source, whose mixed-state presentation is
+infinite, never closes and raises
+:class:`~sofic.exceptions.SemigroupTooLargeError`.
+
+.. autofunction:: projective_semigroup
+.. autofunction:: mixed_state_semigroup

docs/algebra/effective_theories.rst

@@ -0,0 +1,59 @@
+.. effective_theories.rst
+
+******************
+Effective Theories
+******************
+
+Each level of a holonomy cascade reads only the levels above it. So keeping
+the top ``k`` levels gives a well-defined machine of its own, a
+coarse-grained "effective theory" that tracks the large-scale logic of the
+computation while ignoring the details below :cite:`DeDeo2011`. Its states
+are *superstates*: sets of original states that agree on the top ``k``
+coordinates of some lift. Superstates can overlap.
+
+:meth:`~sofic.algebra.holonomy.HolonomyDecomposition.effective_theory`
+returns the truncated machine as a DFA, together with its superstates and
+the projection from full coordinates. The projection commutes with every
+input. :meth:`~sofic.algebra.cascade.CascadeProduct.truncate` does the same
+for a cascade.
+
+The example is Fig. 4 of :cite:`DeDeo2011`: an eight-state machine with two
+inputs, built from XOR gates on a three-bit shift register, here transcribed
+from the paper's TeX source.
+
+.. ipython::
+
+   In [1]: from sofic.algebra import TransformationSemigroup, holonomy_decomposition
+
+   In [2]: a = {"A": "B", "B": "E", "C": "F", "D": "A", "E": "H", "F": "C", "G": "D", "H": "G"}
+
+   In [3]: b = {"A": "B", "B": "C", "C": "F", "D": "B", "E": "G", "F": "C", "G": "F", "H": "G"}
+
+   In [4]: hd = holonomy_decomposition(TransformationSemigroup.from_maps({"a": a, "b": b}, states="ABCDEFGH"))
+
+   @doctest
+   In [5]: [level.order for level in hd.levels]
+   Out[5]: [3, 2, 2]
+
+   In [6]: top = hd.effective_theory(1)
+
+   @doctest
+   In [7]: sorted("".join(sorted(s)) for s in top.superstates.values())
+   Out[7]: ['ACFH', 'BCFG', 'CDEF']
+
+The top level is a counter modulo three on ``a`` and a reset on ``b``. Its
+three superstates are AH, BG, and DE, and each one also contains C and F,
+exactly as in the paper's Fig. 4b. The two lower levels are cyclic groups of
+order two.
+
+Given a stochastic machine, ``effective_theory`` also returns the stationary
+mass of each superstate, grouping states by their canonical lift. It reports
+whether that grouping is strongly lumpable
+(:func:`~sofic.generators.lumping.is_lumpable`), that is, whether the
+coarse-grained process is itself Markov on the superstates.
+
+.. autoclass:: sofic.algebra.holonomy.EffectiveTheory
+.. automethod:: sofic.algebra.holonomy.HolonomyDecomposition.effective_theory
+   :noindex:
+.. automethod:: sofic.algebra.cascade.CascadeProduct.truncate
+   :noindex:

docs/algebra/holonomy.rst

@@ -0,0 +1,77 @@
+.. holonomy.rst
+.. py:module:: sofic.algebra.holonomy
+
+**********************
+Holonomy Decomposition
+**********************
+
+The holonomy method of Zeiger :cite:`Zeiger1967` and Eilenberg
+:cite:`Eilenberg1976` builds a cascade product that emulates a
+transformation semigroup. Each level of the cascade is a permutation group
+(a *holonomy group*) together with constant maps (resets). The implementation
+follows the constructive proof of Egri-Nagy & Nehaniv
+:cite:`EgriNagyNehaniv2015`:
+
+1. The **extended image set** contains the images ``Q · s`` of the full state
+   set, plus all singletons. A set ``P`` approximates any state ``x ∈ P``.
+2. Sets that are images of each other are **equivalent**, and the classes are
+   the strongly connected components of ``S`` acting on image sets.
+   **Subduction** (``P ⊆ R · s`` for some ``s``) orders the classes. The
+   **height** of a set is the length of the longest strict subduction chain of
+   non-singletons ending at it, and the cascade has one level per height,
+   numbered by depth from the top.
+3. The **tiles** of a set are its maximal proper subsets in the image set. A
+   class representative's **holonomy group** is the permutation group its
+   setwise stabilizer induces on its tiles. It is generated by round trips
+   through the class, so it is computed from the generators alone, even when
+   ``S`` is too large to enumerate.
+
+A state is lifted to a maximal chain of tiles from ``Q`` down to the state's
+singleton. Each tile in the chain, recorded relative to its parent's class
+representative, becomes one cascade coordinate. An input acts on a chain
+level by level, from the top. Each level either applies a holonomy
+permutation or resets, and the choice depends only on the coordinates above
+it.
+
+.. ipython::
+
+   In [1]: from sofic.algebra import TransformationSemigroup, holonomy_decomposition
+
+   In [2]: sg = TransformationSemigroup({"s1": (1, 0, 2), "s2": (0, 1, 1)})
+
+   In [3]: hd = holonomy_decomposition(sg)
+
+   @doctest
+   In [4]: [level.order for level in hd.levels]
+   Out[4]: [1, 2]
+
+   In [5]: cascade = hd.to_cascade()
+
+   @doctest
+   In [6]: cascade.emulates(sg)
+   Out[6]: True
+
+This is Example 7.1 of :cite:`EgriNagyNehaniv2015`. ``{0, 1}`` is swapped by
+``s1``, which gives a cyclic group of order two at depth 2. The top level
+only records whether the state lies in ``{0, 1}`` or ``{2}``.
+:meth:`HolonomyDecomposition.describe` summarizes the levels.
+:meth:`~sofic.algebra.cascade.CascadeProduct.emulates` checks that decoding
+commutes with every generator, so each decomposition verifies itself.
+:meth:`~sofic.algebra.cascade.CascadeProduct.as_dfa` returns the cascade as
+an ordinary :class:`~sofic.automata.dfa.DFA` on coordinate tuples.
+
+.. autofunction:: holonomy_decomposition
+.. autoclass:: HolonomyDecomposition
+   :members:
+.. autoclass:: HolonomyLevel
+   :members:
+
+Cascade products
+================
+
+.. py:module:: sofic.algebra.cascade
+
+.. autoclass:: CascadeProduct
+   :members:
+.. autoclass:: LevelAction
+   :members: