Use public vo-regular-bp dependency

requirements.txt

@@ -6,4 +6,4 @@ numpy>=2.2
 torch>=2.3
 verovio>=6.2
 ./vendor/muses
-./vendor/vo_regular_bp
+git+https://github.com/fpachet/vo-regular-bp.git

vendor/vo_regular_bp/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2026 Francois Pachet
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

vendor/vo_regular_bp/README.md

@@ -1,570 +0,0 @@
-# vo-regular-bp
-
-`vo-regular-bp` is a Python library for exact constrained generation from
-sparse variable-order context models, with support for positional masks, meter
-constraints, forbidden-substring constraints, and reusable order-stack
-backends.
-
-The core algorithm runs backward dynamic programming on the reachable product
-of a context graph and a deterministic acceptor. Sampling then chooses each next
-symbol proportionally to its model probability times the downstream beta value,
-so generated sequences are exact samples from the source model conditioned on
-the regular language whenever the constrained mass is nonzero.
-
-The repository also contains specialized engines and experiments for positional
-constraints and Continuator-style order-stack backoff policies.
-
-## Install
-
-The package has no declared runtime dependencies and requires Python 3.10 or
-newer.
-
-For use from another application, install directly from GitHub:
-
-```bash
-python -m pip install "vo-regular-bp @ git+https://github.com/fpachet/vo-regular-bp.git"
-```
-
-To pin an application to a stable revision, append a branch, tag, or commit:
-
-```bash
-python -m pip install "vo-regular-bp @ git+https://github.com/fpachet/vo-regular-bp.git@main"
-```
-
-For local development:
-
-```bash
-python -m pip install -e ".[test]"
-python -m pytest -q
-```
-
-Library-oriented integration notes are in
-[`docs/library_usage.md`](docs/library_usage.md). The paper evaluation scripts
-are kept separately under [`paper/variable_order_regular_bp/`](paper/variable_order_regular_bp/).
-Optimization status and future performance ideas are tracked in
-[`docs/optimization_roadmap.md`](docs/optimization_roadmap.md).
-Virtual transformed-corpus augmentation is described in
-[`docs/virtual_data_augmentation.md`](docs/virtual_data_augmentation.md).
-
-## License
-
-This project is released under the MIT License. See [`LICENSE`](LICENSE).
-
-## Optimization Status
-
-The library includes several exactness-preserving optimizations: positional
-constraints are kept as time masks instead of DFA product state, MAXORDER /
-forbidden-substring constraints use a dense DFA when possible, order-stack
-sampling caches feasible candidate sets, and non-trace sampling avoids trace
-object allocation. See [`docs/optimization_roadmap.md`](docs/optimization_roadmap.md)
-for measured results, discarded experiments, and future optimization options.
-
-## Quick Start
-
-```python
-from vo_regular_bp import ContextGraph, positional_acceptor, run_bp
-
-graph = ContextGraph.from_counts({(): {"a": 10, "b": 1}}, max_order=0)
-acceptor = positional_acceptor(length=1, alphabet=graph.alphabet)
-
-bp = run_bp(graph, acceptor, length=1)
-
-print(bp.partition_function)             # 1.0
-print(bp.conditional_probability(("a",))) # 10 / 11
-print(bp.sample(rng=0))                   # exact constrained sample
-```
-
-Regular constraints can be intersected. This example samples three iid symbols,
-forces the final symbol to be `"a"`, and rejects the substring `"bb"`.
-
-```python
-from vo_regular_bp import (
-    ContextGraph,
-    all_of,
-    forbidden_substring_acceptor,
-    positional_acceptor,
-    run_bp,
-)
-
-graph = ContextGraph.from_counts({(): {"a": 1, "b": 1}}, max_order=0)
-acceptor = all_of(
-    positional_acceptor(3, {2: {"a"}}, alphabet=graph.alphabet),
-    forbidden_substring_acceptor([("b", "b")], alphabet=graph.alphabet),
-)
-
-bp = run_bp(graph, acceptor, length=3)
-samples = bp.sample_many(5, rng=123)
-
-assert all(sample[-1] == "a" for sample in samples)
-assert all(("b", "b") not in zip(sample, sample[1:]) for sample in samples)
-```
-
-## Main Concepts
-
-`ContextGraph` stores a sparse probabilistic context model. It can be built from
-explicit counts/probabilities, unweighted or weighted sequences, or an explicit
-backoff mixture:
-
-- `ContextGraph.from_counts(...)`
-- `ContextGraph.from_probabilities(...)`
-- `ContextGraph.from_sequences(...)`
-- `ContextGraph.from_weighted_sequences(...)`
-- `ContextGraph.from_backoff_sequences(...)`
-
-`DFA` is the deterministic acceptor interface used by product BP. A transition
-returning `None` rejects that symbol from the current acceptor state. Acceptors
-may also expose `transition_weight(state, symbol) -> float` for soft regular
-constraints: BP multiplies the VOMM transition probability by this weight before
-normalizing. The default weight is `1.0`; a weight of `0.0` acts as a hard ban.
-
-Available acceptor helpers include:
-
-- `WeightedDFA`: convenience subclass for weighted regular constraints.
-- `positional_acceptor`: zero-based per-position allowed symbols.
-- `meter_acceptor`: finite meter/class patterns.
-- `cumulative_meter_acceptor`: cumulative-cost meter predicates.
-- `forbidden_substring_acceptor`: reject any forbidden substring.
-- `dense_forbidden_substring_acceptor`: finite-alphabet optimized variant.
-- `max_order_acceptor`: reject reference substrings of length `max_order + 1`.
-- `all_of`: intersect acceptors.
-- `true_acceptor`: accept every finite sequence.
-
-## Engines
-
-### Product BP
-
-`run_bp(graph, acceptor, length=...)` is the general exact engine. It performs
-backward DP over reachable `(context_state, acceptor_state)` product states and
-returns a `ProductBPResult` with:
-
-- `partition_function`: constrained probability mass.
-- `sample(...)` and `sample_many(...)`: exact conditional samples.
-- `conditional_probability(sequence)`: probability under the constrained
-  distribution.
-- product-state and edge-count diagnostics for scalability studies.
-
-`sample_exact(...)` is a convenience wrapper that runs BP and draws one sample.
-
-### Source Graph Minimization And Experimental Merging
-
-Exact source minimization is available as an optional compiled view. It is
-semantics-preserving and never enabled by default:
-
-```python
-from vo_regular_bp import exact_context_graph_quotient_stats, minimize_context_graph
-
-stats = exact_context_graph_quotient_stats(graph)
-minimized = minimize_context_graph(graph)
-```
-
-Order-stack preparation also accepts `minimize_source_graphs=True`, which
-minimizes the source graphs before running the ordinary exact BP backend. The
-training/count model remains uncompressed, and minimized fixed-order graphs are
-cached on the model for reuse.
-
-Approximate source merging is explicitly experimental and changes the source
-model. It lives outside the safe top-level API:
-
-```python
-from vo_regular_bp.experimental import alergia_merge
-
-merged = alergia_merge(
-    graph,
-    alpha=0.01,
-    min_support=10,
-    recursive=True,
-    transition_projection=lambda state, symbol, edge: (
-        symbol - state[-1]
-        if state and isinstance(state[-1], int) and isinstance(symbol, int)
-        else symbol
-    ),
-)
-```
-
-The returned object is still a `ContextGraph`, so `run_bp(...)` remains exact
-with respect to the merged source model. This is not constrained-product
-minimization and is not applied automatically. The optional
-`symbol_projection` and `transition_projection` arguments are where a client
-supplies domain semantics for similarity; `transition_projection(state, symbol,
-edge)` handles context-relative abstractions and takes precedence when both are
-provided. The default compares raw emitted symbols.
-
-Order-stack models can opt into the same experiment by merging each fixed-order
-source graph independently, then using the ordinary backend:
-
-```python
-from vo_regular_bp.experimental import alergia_merge_order_stack_model
-
-abstract_model = alergia_merge_order_stack_model(
-    model,
-    alpha=0.01,
-    min_support=10,
-    recursive=True,
-    transition_projection=lambda state, symbol, edge: (
-        symbol - state[-1]
-        if state and isinstance(state[-1], int) and isinstance(symbol, int)
-        else symbol
-    ),
-)
-
-backend = prepare_constrained_order_stack(
-    abstract_model,
-    constraints,
-    length=8,
-    prefix=prefix,
-)
-```
-
-This preserves the public order-stack sampling path and trace shape. It changes
-the source model explicitly: BP and order selection remain exact for the merged
-fixed-order graphs that are passed in.
-
-The ALERGIA implementation is optimized for projected comparisons by caching
-projected continuation counts, projected successor labels, and recursive
-pair-compatibility decisions within one merge call. It also tracks active
-merge classes incrementally instead of rebuilding class members during every
-candidate comparison. These are internal optimizations only: they do not change
-the explicit API or make approximate merging automatic.
-
-### Positional BP
-
-`run_positional_bp(...)` is a no-DFA specialization for fixed-horizon
-time-indexed symbol masks. It is useful when the only constraints are
-positional, because the recursion ranges over context states rather than full
-context-acceptor products.
-
-`LazyBackoffContextModel` matches `ContextGraph.from_backoff_sequences(...)`
-semantically, but materializes outgoing edges only when reached by BP.
-
-### Order-Stack BP
-
-`OrderStackModel` and the `run_order_stack_*` functions implement
-Continuator-style constrained policy backoff over a stack of fixed-order
-models:
-
-- `run_order_stack_bp`: positional constraints only.
-- `run_order_stack_dfa_bp`: regular DFA constraints, including weighted transitions.
-- `run_order_stack_masked_dfa_bp`: regular DFA plus positional masks.
-
-This is a generation policy, not exact conditioning of one fixed stochastic
-source. At each step, the engine computes feasible future mass for each order,
-then an order policy such as `LongestFeasiblePolicy` or
-`SingletonAvoidingBackoffPolicy` chooses among feasible candidate orders.
-
-The main result object reports `success_mass`, order-specific start masses,
-samples, and optional order traces.
-
-### Public Constraint Backend
-
-`prepare_constrained_order_stack(...)` is the recommended library-facing entry
-point for order-stack generation. It accepts a model and a `ConstraintSet`,
-compiles positional constraints as time masks, compiles regular constraints as
-acceptors, runs the backend once, and returns a reusable sampler.
-
-```python
-from vo_regular_bp import (
-    ConstraintSet,
-    LongestFeasiblePolicy,
-    OrderStackModel,
-    prepare_constrained_order_stack,
-    prepare_constrained_order_stack_plan,
-)
-
-sequence = (60, 64, 67, 72, 76, 67, 71, 72)
-model = OrderStackModel.from_sequences([sequence], max_order=2)
-final_c = {pitch for pitch in sequence if pitch % 12 == 0}
-
-backend = prepare_constrained_order_stack(
-    model,
-    ConstraintSet(positional={3: final_c}),
-    length=4,
-    prefix=sequence[:2],
-    policy=LongestFeasiblePolicy(),
-)
-
-generated = backend.sample_with_orders(rng=0)
-sample = generated.sequence
-orders = generated.orders
-assert sample[-1] % 12 == 0
-```
-
-For convenience, `prepare_constrained_order_stack_from_sequences(...)` builds
-the `OrderStackModel` and prepares the backend in one call. The returned
-`ConstrainedOrderStackBackend` exposes `sample(...)`, `sample_many(...)`,
-`sample_with_orders(...)`, `sample_many_with_orders(...)`, `sample_with_trace(...)`,
-and a stable `diagnostics` object with context/product sizes and success mass
-when a regular backend is used.
-
-`run_constrained_order_stack(...)` remains available when callers need the raw
-internal BP result object.
-
-When the model, horizon, and constraints are fixed but many prefixes will be
-sampled, prepare a prefix-independent plan once and bind prefixes later:
-
-```python
-plan = prepare_constrained_order_stack_plan(
-    model,
-    ConstraintSet(positional={3: final_c}),
-    length=4,
-    policy=LongestFeasiblePolicy(),
-)
-
-backend_a = plan.for_prefix(sequence[:2])
-backend_b = plan.for_prefix(sequence[2:4])
-
-sample_a = backend_a.sample(rng=0)
-sample_b = backend_b.sample(rng=1)
-```
-
-For regular constraints, the plan owns the shared graph and backward-message
-caches. Binding a prefix warms the lazy beta messages reachable from that prefix,
-and later prefixes reuse overlapping cached product states. The old
-`prepare_constrained_order_stack(...)` API is now a convenience wrapper around
-this plan path when the model supports prefix-independent graph preparation.
-
-For variable-length Continuator-style suffixes, use
-`prepare_until_order_stack(...)`. It prepares one fixed-length constrained
-backend for each feasible length in `[min_length, max_length]`, samples a
-length by the sum of its positive start-order masses, then samples the suffix
-from that length backend. If only a backend success indicator is available, the
-length receives unit weight. The returned suffix does not include the prefix:
-the prefix is conditioning context only.
-
-```python
-from vo_regular_bp import OrderStackModel, prepare_until_order_stack
-
-model = OrderStackModel.from_sequences([("A", "B", "C")], max_order=1)
-backend = prepare_until_order_stack(
-    model,
-    prefix=("A",),
-    stop="C",
-    min_length=1,
-    max_length=3,
-)
-
-suffix = backend.sample(rng=0)
-assert suffix == ("B", "C")
-```
-
-`stop` may be one symbol, an iterable/set of symbols, or a predicate
-`stop(symbol) -> bool`. The returned suffix includes the first generated stop
-symbol at the final position, and earlier positions are constrained not to
-satisfy `stop`. `prepare_until_end_order_stack(..., end_symbol=...)` is a
-convenience alias for learned END/sentinel stops. END remains forbidden in
-ordinary fixed-length sampling, but first-hit END generation explicitly allows
-the sentinel at the final stop position.
-
-`ConstraintSet` supports:
-
-- `positional`: per-time symbol masks or predicates.
-- `forbidden_substrings`: exact forbidden substring / MAXORDER constraints,
-  compiled to a dense DFA when possible.
-- `regular_acceptors`: caller-supplied `DFA` or `WeightedDFA` instances. Soft
-  transition weights are multiplied into BP; missing transitions or weight
-  `0.0` reject the symbol.
-- `meter`: a `MeterConstraint` for finite per-symbol meter/class patterns.
-- `cumulative_meter`: a `CumulativeMeterConstraint` for duration/cost
-  accumulation, bar-boundary predicates, final total cost, and optional padding
-  symbols.
-
-The core API is intentionally not Continuator-specific; adapters for other
-projects can map their own event objects to symbols, meter classes, costs, or
-regular acceptors.
-
-All high-level constraints are over the generated suffix. Positional indices are
-zero-based within the generated sequence, not within the training corpus and not
-within `prefix + generated`.
-
-### Constraint Builders
-
-Common constraints can be built and combined without manually constructing
-`ConstraintSet` objects:
-
-```python
-from vo_regular_bp import (
-    combine_constraints,
-    final_pitch_class,
-    avoid_copied_ngrams,
-)
-
-constraints = combine_constraints(
-    final_pitch_class(0, length=32),
-    avoid_copied_ngrams(reference_pitches, 5),
-)
-```
-
-Builder helpers include:
-
-- `at_position(...)`
-- `final_symbol(...)` and `final_symbols(...)`
-- `final_pitch_class(...)`
-- `avoid_copied_ngrams(...)`
-- `meter_pattern(...)`
-- `cumulative_meter(...)`
-- `padded_duration_total(...)`
-- `combine_constraints(...)`
-
-### Event Adapters
-
-External projects can keep rich event objects at their boundary and encode them
-as hashable symbols for the backend:
-
-```python
-from dataclasses import dataclass
-from vo_regular_bp import (
-    EventCodec,
-    combine_constraints,
-    final_pitch_class,
-    prepare_constrained_order_stack_from_events,
-)
-
-@dataclass(frozen=True)
-class Note:
-    pitch: int
-    duration: int
-
-codec = EventCodec(
-    event_to_symbol=lambda note: (note.pitch, note.duration),
-    symbol_to_event=lambda symbol: Note(symbol[0], symbol[1]),
-)
-
-backend = prepare_constrained_order_stack_from_events(
-    [training_notes],
-    combine_constraints(
-        final_pitch_class(0, length=8, symbol_to_pitch=lambda symbol: symbol[0]),
-    ),
-    codec=codec,
-    max_order=3,
-    length=8,
-    prefix=prefix_notes,
-)
-
-generated = backend.sample_events_with_orders(rng=0)
-print(generated.events)
-print(generated.orders)
-```
-
-See `examples/event_order_stack_backend.py` for a complete small example.
-
-### Continuator-Style Facade
-
-For projects that want a Continuator-shaped entry point, the dependency-free
-facade in `vo_regular_bp.continuator` uses event sequences, a prefix, a horizon,
-and constraints:
-
-```python
-from vo_regular_bp import (
-    combine_constraints,
-    duration_total_constraint,
-    final_pitch_class_constraint,
-    prepare_continuation_backend,
-    prepare_continuation_plan,
-)
-
-constraints = combine_constraints(
-    final_pitch_class_constraint(
-        0,
-        horizon=8,
-        symbol_to_pitch=lambda symbol: symbol[0],
-    ),
-    duration_total_constraint(
-        8,
-        horizon=8,
-        symbol_to_duration=lambda symbol: symbol[1],
-    ),
-)
-
-backend = prepare_continuation_backend(
-    [training_events],
-    prefix=prefix_events,
-    horizon=8,
-    max_order=4,
-    constraints=constraints,
-    event_to_symbol=lambda event: (event.pitch, event.duration),
-)
-
-generated = backend.sample_events_with_orders(rng=0)
-print(generated.events)
-print(backend.diagnostics.as_dict())
-```
-
-The facade defaults to `SingletonAvoidingBackoffPolicy`, matching the
-Continuator-style policy-backoff interpretation. Pass `policy=...` to use a
-different order-selection policy. See `examples/continuator_style_backend.py`
-for a complete dependency-free example.
-
-When making many calls with the same training material, horizon, and constraints,
-use `prepare_continuation_plan(...)` once and bind each prefix later:
-
-```python
-plan = prepare_continuation_plan(
-    [training_events],
-    horizon=8,
-    max_order=4,
-    constraints=constraints,
-    event_to_symbol=lambda event: (event.pitch, event.duration),
-)
-
-backend = plan.for_prefix(prefix_events)
-generated = backend.sample_events_with_orders(rng=0)
-```
-
-For variable musical length in a fixed BP horizon, use
-`padded_duration_total_constraint(...)` with explicit zero-duration PAD symbols
-added to phrase/bar training sequences via `append_padding(...)`. See
-`examples/padded_duration_order_stack_backend.py`.
-
-## Brute Force and Metrics
-
-For small examples, the package includes exact enumeration helpers:
-
-- `brute_force_distribution(...)`
-- `brute_force_partition_function(...)`
-- `conditional_distribution(...)`
-
-For empirical checks:
-
-- `empirical_distribution(...)`
-- `total_variation(...)`
-
-These are intended for tests, toy examples, and exactness validation.
-
-## Paper Experiments
-
-The reusable library lives in `vo_regular_bp/`.  Paper-specific validation and
-benchmark implementations are kept under `paper/variable_order_regular_bp/`.
-The top-level `scripts/` files remain as compatibility launchers, so existing
-commands still work:
-
-```bash
-python scripts/eval_tiny_exactness.py
-python scripts/eval_scalability.py --help
-python scripts/eval_bach_scalability.py --help
-python scripts/eval_neurips_ablation.py --help
-python scripts/eval_virtual_augmentation.py --help
-python scripts/eval_bach_continuator_compare.py --help
-python scripts/eval_bach_positional_direct.py --help
-```
-
-The Bach data lives in `data/bach_prelude_c_major_pitches.txt`.
-
-Rendered experiment notes are in `reports/`, including:
-
-- `reports/evaluation_report.md`
-- `reports/optimization_report.md`
-- `reports/bach_contextbp_final_compare.md`
-- `reports/bach_policy_backoff_paper_results.md`
-
-## Semantics Note
-
-The direct product-BP path (`ContextGraph` plus `run_bp`) samples exactly from a
-single fixed stochastic source conditioned on a regular language:
-
-```text
-P_source(x | x in L(A))
-```
-
-The order-stack path is exact for a different object: a constrained generation
-policy that prefers higher orders when they have positive feasible future mass
-and backs off only when needed. Its `success_mass` is therefore a policy success
-indicator/mass, not the partition function of one fixed source graph.

vendor/vo_regular_bp/pyproject.toml

@@ -1,51 +0,0 @@
-[build-system]
-requires = ["setuptools>=68"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "vo-regular-bp"
-version = "0.1.1"
-description = "Exact constrained sampling from sparse variable-order context models under regular constraints."
-readme = "README.md"
-requires-python = ">=3.10"
-license = { file = "LICENSE" }
-authors = [
-  { name = "Francois Pachet" }
-]
-dependencies = []
-keywords = [
-  "constrained-sampling",
-  "variable-order-markov",
-  "belief-propagation",
-  "regular-constraints",
-  "music-generation",
-]
-classifiers = [
-  "Development Status :: 3 - Alpha",
-  "Intended Audience :: Developers",
-  "Intended Audience :: Science/Research",
-  "License :: OSI Approved :: MIT License",
-  "Programming Language :: Python :: 3",
-  "Programming Language :: Python :: 3 :: Only",
-  "Programming Language :: Python :: 3.10",
-  "Programming Language :: Python :: 3.11",
-  "Programming Language :: Python :: 3.12",
-  "Programming Language :: Python :: 3.13",
-  "Topic :: Scientific/Engineering",
-  "Topic :: Software Development :: Libraries :: Python Modules",
-]
-
-[project.optional-dependencies]
-test = [
-  "pytest>=8",
-]
-
-[project.urls]
-Repository = "https://github.com/fpachet/vo-regular-bp"
-
-[tool.setuptools.packages.find]
-include = ["vo_regular_bp*"]
-
-[tool.pytest.ini_options]
-testpaths = ["tests"]
-pythonpath = ["."]

vendor/vo_regular_bp/vo_regular_bp/__init__.py

@@ -1,256 +0,0 @@
-"""Exact constrained sampling for sparse variable-order context models.
-
-The top-level package exports the stable library surface: generic product BP,
-Continuator-style order-stack backends, event adapters, constraint builders, and
-diagnostic helpers.  Lower-level modules remain importable for experiments, but
-external projects should prefer the names exported here.
-"""
-
-from importlib.metadata import PackageNotFoundError as _PackageNotFoundError
-from importlib.metadata import version as _metadata_version
-
-try:
-    __version__ = _metadata_version("vo-regular-bp")
-except _PackageNotFoundError:
-    __version__ = "0+unknown"
-
-from .acceptors import (
-    DFA,
-    DenseForbiddenSubstringDFA,
-    SupportsWeightedTransitions,
-    WeightedDFA,
-    all_of,
-    cumulative_meter_acceptor,
-    dense_forbidden_substring_acceptor,
-    forbidden_substring_acceptor,
-    max_order_acceptor,
-    meter_acceptor,
-    positional_acceptor,
-    true_acceptor,
-)
-from .adapters import (
-    EventCodec,
-    EventOrderStackBackend,
-    EventOrderStackPlan,
-    GeneratedEvents,
-    infer_symbol_to_event,
-    prepare_constrained_order_stack_from_events,
-    prepare_constrained_order_stack_plan_from_events,
-)
-from .augmentation import (
-    SymbolTransform,
-    VirtualAugmentedOrderStackModel,
-    integer_shift_transform,
-    integer_shift_transforms,
-    materialize_transformed_sequences,
-)
-from .brute_force import (
-    brute_force_distribution,
-    brute_force_partition_function,
-    conditional_distribution,
-)
-from .backend import (
-    BackendDiagnostics,
-    ConstrainedOrderStackBackend,
-    ConstrainedOrderStackPlan,
-    ConstrainedOrderStackSupportPlan,
-    GeneratedSequence,
-    UntilLengthDiagnostics,
-    UntilOrderStackBackend,
-    UntilOrderStackDiagnostics,
-    prepare_constrained_order_stack,
-    prepare_constrained_order_stack_plan,
-    prepare_constrained_order_stack_plan_from_sequences,
-    prepare_constrained_order_stack_support_plan,
-    prepare_constrained_order_stack_support_plan_from_sequences,
-    prepare_constrained_order_stack_from_sequences,
-    prepare_until_end_order_stack,
-    prepare_until_order_stack,
-    run_constrained_order_stack,
-)
-from .constraint_builders import (
-    at_position,
-    avoid_copied_ngrams,
-    combine_constraints,
-    cumulative_meter,
-    final_pitch_class,
-    final_symbol,
-    final_symbols,
-    meter_pattern,
-    padded_duration_total,
-)
-from .continuator import (
-    duration_total_constraint,
-    final_pitch_class_constraint,
-    meter_cycle_constraint,
-    padded_duration_total_constraint,
-    prepare_continuation_backend,
-    prepare_continuation_plan,
-)
-from .constraints import (
-    CompiledConstraints,
-    ConstraintSet,
-    CumulativeMeterConstraint,
-    MeterConstraint,
-    compile_constraints,
-)
-from .context import Context, ContextGraph, Edge, Symbol
-from .metrics import empirical_distribution, total_variation
-from .minimization import (
-    ExactQuotientStats,
-    exact_context_graph_quotient_stats,
-    exact_fixed_order_graph_quotient_stats,
-    minimize_context_graph,
-    minimize_fixed_order_graph,
-    minimize_fixed_order_graphs,
-)
-from .order_stack_bp import (
-    DurationViewQuotientDiagnostics,
-    DurationViewQuotientOrderStats,
-    LongestFeasiblePolicy,
-    OrderStackBPPlan,
-    OrderStackBPResult,
-    OrderStackModel,
-    RegularOrderStackBPPlan,
-    RegularOrderStackBPResult,
-    SingletonAvoidingBackoffPolicy,
-    padded_melody_duration_view_quotient_diagnostics,
-    prepare_order_stack_bp,
-    prepare_order_stack_dfa_bp,
-    prepare_order_stack_masked_dfa_bp,
-    run_order_stack_dfa_bp,
-    run_order_stack_masked_dfa_bp,
-    run_order_stack_bp,
-)
-from .orbit_diagnostics import (
-    ForbiddenPatternOrbitStats,
-    RegularProductOrbitStats,
-    RegularRowSignatureStats,
-    canonical_integer_shift_key,
-    forbidden_pattern_orbit_stats,
-    regular_product_orbit_stats,
-    regular_row_signature_stats,
-)
-from .padding import append_padding
-from .positional_bp import LazyBackoffContextModel, PositionalBPResult, run_positional_bp
-from .product_bp import ProductBPResult, run_bp, sample_exact
-
-__all__ = [
-    "__version__",
-    # Core symbolic graph and DFA API.
-    "Context",
-    "ContextGraph",
-    "DFA",
-    "DenseForbiddenSubstringDFA",
-    "Edge",
-    "SupportsWeightedTransitions",
-    "Symbol",
-    "WeightedDFA",
-    "all_of",
-    "cumulative_meter_acceptor",
-    "dense_forbidden_substring_acceptor",
-    "forbidden_substring_acceptor",
-    "max_order_acceptor",
-    "meter_acceptor",
-    "positional_acceptor",
-    "true_acceptor",
-    # General exact product BP.
-    "LazyBackoffContextModel",
-    "PositionalBPResult",
-    "ProductBPResult",
-    "run_bp",
-    "run_positional_bp",
-    "sample_exact",
-    # Library-facing order-stack backend.
-    "BackendDiagnostics",
-    "ConstrainedOrderStackBackend",
-    "ConstrainedOrderStackPlan",
-    "ConstrainedOrderStackSupportPlan",
-    "DurationViewQuotientDiagnostics",
-    "DurationViewQuotientOrderStats",
-    "GeneratedSequence",
-    "LongestFeasiblePolicy",
-    "OrderStackBPPlan",
-    "OrderStackBPResult",
-    "OrderStackModel",
-    "RegularOrderStackBPPlan",
-    "RegularOrderStackBPResult",
-    "SingletonAvoidingBackoffPolicy",
-    "UntilLengthDiagnostics",
-    "UntilOrderStackBackend",
-    "UntilOrderStackDiagnostics",
-    "prepare_constrained_order_stack",
-    "prepare_constrained_order_stack_plan",
-    "prepare_constrained_order_stack_plan_from_sequences",
-    "prepare_constrained_order_stack_support_plan",
-    "prepare_constrained_order_stack_support_plan_from_sequences",
-    "prepare_constrained_order_stack_from_sequences",
-    "prepare_order_stack_bp",
-    "prepare_order_stack_dfa_bp",
-    "prepare_order_stack_masked_dfa_bp",
-    "padded_melody_duration_view_quotient_diagnostics",
-    "prepare_until_end_order_stack",
-    "prepare_until_order_stack",
-    "run_constrained_order_stack",
-    "run_order_stack_bp",
-    "run_order_stack_dfa_bp",
-    "run_order_stack_masked_dfa_bp",
-    # Exact source graph minimization.
-    "ExactQuotientStats",
-    "exact_context_graph_quotient_stats",
-    "exact_fixed_order_graph_quotient_stats",
-    "minimize_context_graph",
-    "minimize_fixed_order_graph",
-    "minimize_fixed_order_graphs",
-    # Transformation-orbit diagnostics.
-    "ForbiddenPatternOrbitStats",
-    "RegularProductOrbitStats",
-    "RegularRowSignatureStats",
-    "canonical_integer_shift_key",
-    "forbidden_pattern_orbit_stats",
-    "regular_product_orbit_stats",
-    "regular_row_signature_stats",
-    # Virtual data augmentation.
-    "SymbolTransform",
-    "VirtualAugmentedOrderStackModel",
-    "integer_shift_transform",
-    "integer_shift_transforms",
-    "materialize_transformed_sequences",
-    # Fixed-horizon padding helpers.
-    "append_padding",
-    # Constraint specifications and builders.
-    "CompiledConstraints",
-    "ConstraintSet",
-    "CumulativeMeterConstraint",
-    "MeterConstraint",
-    "at_position",
-    "avoid_copied_ngrams",
-    "combine_constraints",
-    "compile_constraints",
-    "cumulative_meter",
-    "final_pitch_class",
-    "final_symbol",
-    "final_symbols",
-    "meter_pattern",
-    "padded_duration_total",
-    # Event and Continuator-style adapters.
-    "EventCodec",
-    "EventOrderStackBackend",
-    "EventOrderStackPlan",
-    "GeneratedEvents",
-    "duration_total_constraint",
-    "final_pitch_class_constraint",
-    "infer_symbol_to_event",
-    "meter_cycle_constraint",
-    "padded_duration_total_constraint",
-    "prepare_constrained_order_stack_from_events",
-    "prepare_constrained_order_stack_plan_from_events",
-    "prepare_continuation_backend",
-    "prepare_continuation_plan",
-    # Analysis/test helpers that are useful for exactness checks.
-    "brute_force_distribution",
-    "brute_force_partition_function",
-    "conditional_distribution",
-    "empirical_distribution",
-    "total_variation",
-]

vendor/vo_regular_bp/vo_regular_bp/acceptors.py

@@ -1,660 +0,0 @@
-"""Deterministic regular acceptors used by product BP."""
-
-from __future__ import annotations
-
-import math
-from itertools import product
-from typing import Callable, Hashable, Iterable, Mapping, Protocol, Sequence
-
-Symbol = Hashable
-State = Hashable
-
-
-class SupportsWeightedTransitions(Protocol):
-    """Structural acceptor interface with optional soft transition weights."""
-
-    start_state: State
-
-    def next_state(self, state: State, symbol: Symbol) -> State | None:
-        ...
-
-    def is_accepting(self, state: State) -> bool:
-        ...
-
-    def transition_weight(self, state: State, symbol: Symbol) -> float:
-        ...
-
-
-class DFA:
-    """Small deterministic acceptor interface.
-
-    A transition returning ``None`` means that the emitted symbol is rejected
-    from that state. Legal transitions can optionally carry nonnegative
-    multiplicative weights through ``transition_weight``; the default is
-    ``1.0``, so ordinary hard DFAs are unchanged.
-    """
-
-    def __init__(
-        self,
-        *,
-        start_state: State,
-        accept_states: Iterable[State] | None = None,
-        transitions: Mapping[State, Mapping[Symbol, State]] | None = None,
-        states: Iterable[State] | None = None,
-        alphabet: Iterable[Symbol] | None = None,
-        transition_func: Callable[[State, Symbol], State | None] | None = None,
-        transition_weights: Mapping[State, Mapping[Symbol, float]] | None = None,
-        transition_weight_func: Callable[[State, Symbol], float] | None = None,
-        accept_func: Callable[[State], bool] | None = None,
-        name: str = "dfa",
-    ) -> None:
-        if accept_states is None and accept_func is None:
-            raise ValueError("either accept_states or accept_func is required")
-        if transitions is None and transition_func is None:
-            raise ValueError("either transitions or transition_func is required")
-
-        self.start_state = start_state
-        self.accept_states = frozenset(accept_states or ())
-        self.transitions = {
-            state: dict(symbols)
-            for state, symbols in (transitions or {}).items()
-        }
-        self._transition_weights = {
-            state: dict(symbols)
-            for state, symbols in (transition_weights or {}).items()
-        }
-        self.states = None if states is None else frozenset(states)
-        self.alphabet = None if alphabet is None else frozenset(alphabet)
-        self._transition_func = transition_func
-        self._transition_weight_func = transition_weight_func
-        self._accept_func = accept_func
-        self.name = name
-
-    def next_state(self, state: State, symbol: Symbol) -> State | None:
-        if self._transition_func is not None:
-            return self._transition_func(state, symbol)
-        return self.transitions.get(state, {}).get(symbol)
-
-    def transition_weight(self, state: State, symbol: Symbol) -> float:
-        transition_weight_func = getattr(self, "_transition_weight_func", None)
-        if transition_weight_func is not None:
-            return float(transition_weight_func(state, symbol))
-        transition_weights = getattr(self, "_transition_weights", {})
-        return float(transition_weights.get(state, {}).get(symbol, 1.0))
-
-    def is_accepting(self, state: State) -> bool:
-        if self._accept_func is not None:
-            return bool(self._accept_func(state))
-        return state in self.accept_states
-
-    def accepts(self, sequence: Sequence[Symbol]) -> bool:
-        state = self.start_state
-        for symbol in sequence:
-            current_state = state
-            state = self.next_state(state, symbol)
-            if state is None:
-                return False
-            if transition_weight(self, current_state, symbol) <= 0.0:
-                return False
-        return self.is_accepting(state)
-
-    def state_count(self) -> int | None:
-        if self.states is None:
-            return None
-        return len(self.states)
-
-
-class WeightedDFA(DFA):
-    """DFA subclass documenting weighted regular-constraint semantics.
-
-    Subclasses may override ``transition_weight``. The inherited implementation
-    supports either a transition-weight mapping or a callable passed to
-    ``DFA.__init__``.
-    """
-
-
-class DenseForbiddenSubstringDFA(DFA):
-    """Finite-alphabet forbidden-substring DFA with dense integer states.
-
-    States are integer ids for the same proper-prefix states used by
-    :func:`forbidden_substring_acceptor`. A transition value of ``-1`` means
-    rejection. The public methods mirror :class:`DFA`, while
-    ``dense_transition_by_symbol`` gives hot loops a direct table lookup path.
-    """
-
-    rejected_state = -1
-
-    def __init__(
-        self,
-        *,
-        prefixes: Sequence[tuple[Symbol, ...]],
-        alphabet: Iterable[Symbol],
-        transition_table: Sequence[Sequence[int]],
-        name: str = "dense_forbidden_substring",
-    ) -> None:
-        alphabet_tuple = tuple(alphabet)
-        self.start_state = 0
-        self.prefixes = tuple(prefixes)
-        self.states = frozenset(range(len(self.prefixes)))
-        self.accept_states = self.states
-        self.alphabet = frozenset(alphabet_tuple)
-        self.transition_table = tuple(tuple(int(next_state) for next_state in row) for row in transition_table)
-        self.symbol_to_index = {symbol: index for index, symbol in enumerate(alphabet_tuple)}
-        self.dense_transition_by_symbol = {
-            symbol: tuple(row[index] for row in self.transition_table)
-            for symbol, index in self.symbol_to_index.items()
-        }
-        self.name = name
-
-    def next_state(self, state: State, symbol: Symbol) -> State | None:
-        if not isinstance(state, int):
-            raise TypeError("dense forbidden-substring states must be integers")
-        transitions = self.dense_transition_by_symbol.get(symbol)
-        if transitions is None:
-            return None
-        next_state = transitions[state]
-        if next_state == self.rejected_state:
-            return None
-        return next_state
-
-    def is_accepting(self, state: State) -> bool:
-        return isinstance(state, int) and 0 <= state < len(self.prefixes)
-
-    def accepts(self, sequence: Sequence[Symbol]) -> bool:
-        state = self.start_state
-        for symbol in sequence:
-            next_state = self.next_state(state, symbol)
-            if next_state is None:
-                return False
-            if transition_weight(self, state, symbol) <= 0.0:
-                return False
-            state = next_state
-        return self.is_accepting(state)
-
-    def state_count(self) -> int:
-        return len(self.prefixes)
-
-    def edge_count(self) -> int:
-        return sum(
-            1
-            for row in self.transition_table
-            for next_state in row
-            if next_state != self.rejected_state
-        )
-
-
-def true_acceptor(*, name: str = "true") -> DFA:
-    """Accept every finite sequence over any alphabet."""
-
-    return DFA(
-        start_state=0,
-        accept_states={0},
-        states={0},
-        transition_func=lambda state, symbol: 0,
-        name=name,
-    )
-
-
-def all_of(*acceptors: DFA, name: str = "all_of") -> DFA:
-    """Intersection of deterministic acceptors."""
-
-    if not acceptors:
-        return true_acceptor(name=name)
-
-    start = tuple(acceptor.start_state for acceptor in acceptors)
-
-    def transition(state: State, symbol: Symbol) -> State | None:
-        next_parts = []
-        for acceptor, part in zip(acceptors, state):
-            next_part = acceptor.next_state(part, symbol)
-            if next_part is None:
-                return None
-            next_parts.append(next_part)
-        return tuple(next_parts)
-
-    def accepting(state: State) -> bool:
-        return all(acceptor.is_accepting(part) for acceptor, part in zip(acceptors, state))
-
-    def weight(state: State, symbol: Symbol) -> float:
-        total = 1.0
-        for acceptor, part in zip(acceptors, state):
-            total *= transition_weight(acceptor, part, symbol)
-        return total
-
-    states = None
-    if all(acceptor.states is not None for acceptor in acceptors):
-        sizes = [len(acceptor.states or ()) for acceptor in acceptors]
-        product_size = 1
-        for size in sizes:
-            product_size *= size
-        if product_size <= 100_000:
-            state_sets = [acceptor.states or () for acceptor in acceptors]
-            states = set(product(*state_sets))
-
-    alphabets = [acceptor.alphabet for acceptor in acceptors if acceptor.alphabet is not None]
-    alphabet = set.intersection(*map(set, alphabets)) if alphabets else None
-
-    dfa = DFA(
-        start_state=start,
-        states=states,
-        alphabet=alphabet,
-        transition_func=transition,
-        transition_weight_func=weight,
-        accept_func=accepting,
-        name=name,
-    )
-    dfa.component_acceptors = tuple(acceptors)  # type: ignore[attr-defined]
-    return dfa
-
-
-def transition_weight(acceptor: object, state: State, symbol: Symbol) -> float:
-    """Return a validated multiplicative weight for a legal transition.
-
-    Acceptors without ``transition_weight`` are treated as ordinary hard DFAs
-    with weight ``1.0``. A weight of ``0.0`` is allowed and acts as hard
-    rejection in BP; negative or non-finite weights are invalid.
-    """
-
-    method = getattr(acceptor, "transition_weight", None)
-    raw_weight = 1.0 if method is None else method(state, symbol)
-    weight = float(raw_weight)
-    if not math.isfinite(weight) or weight < 0.0:
-        raise ValueError(
-            f"transition weight for state {state!r} and symbol {symbol!r} "
-            f"must be a finite nonnegative number, got {raw_weight!r}"
-        )
-    return weight
-
-
-def has_custom_transition_weights(acceptor: object) -> bool:
-    method = getattr(type(acceptor), "transition_weight", None)
-    default_method = getattr(DFA, "transition_weight")
-    if method is not None and method is not default_method:
-        return True
-    return bool(
-        getattr(acceptor, "_transition_weights", None)
-        or getattr(acceptor, "_transition_weight_func", None) is not None
-    )
-
-
-def positional_acceptor(
-    length: int,
-    constraints: Mapping[int, Iterable[Symbol] | Symbol] | None = None,
-    *,
-    alphabet: Iterable[Symbol] | None = None,
-    name: str = "positional",
-) -> DFA:
-    """Accept length-``n`` strings satisfying per-position allowed sets.
-
-    Positions are zero-based. Constraint values may be a single symbol or an
-    iterable of allowed symbols.
-    """
-
-    if length < 0:
-        raise ValueError("length must be non-negative")
-
-    allowed_by_pos: dict[int, set[Symbol]] = {}
-    for position, allowed in (constraints or {}).items():
-        if position < 0 or position >= length:
-            raise ValueError(f"position {position} is outside length {length}")
-        if isinstance(allowed, (str, bytes)):
-            allowed_by_pos[position] = {allowed}
-        else:
-            try:
-                allowed_by_pos[position] = set(allowed)  # type: ignore[arg-type]
-            except TypeError:
-                allowed_by_pos[position] = {allowed}  # type: ignore[list-item]
-
-    def transition(position: State, symbol: Symbol) -> State | None:
-        if not isinstance(position, int) or position >= length:
-            return None
-        allowed = allowed_by_pos.get(position)
-        if allowed is not None and symbol not in allowed:
-            return None
-        return position + 1
-
-    return DFA(
-        start_state=0,
-        accept_states={length},
-        states=set(range(length + 1)),
-        alphabet=alphabet,
-        transition_func=transition,
-        name=name,
-    )
-
-
-def meter_acceptor(
-    pattern: Sequence[Hashable | Iterable[Hashable] | None],
-    symbol_to_meter: Mapping[Symbol, Hashable] | Callable[[Symbol], Hashable],
-    *,
-    alphabet: Iterable[Symbol] | None = None,
-    name: str = "meter",
-) -> DFA:
-    """Accept strings whose per-symbol meter classes match ``pattern``.
-
-    Each pattern entry may be a single class, an iterable of allowed classes, or
-    ``None`` as a wildcard. This intentionally keeps meter generic: callers can
-    map symbols to stress, duration, pitch class, or any other finite class.
-    """
-
-    allowed_by_pos: list[set[Hashable] | None] = []
-    for expected in pattern:
-        if expected is None:
-            allowed_by_pos.append(None)
-        elif isinstance(expected, (str, bytes)):
-            allowed_by_pos.append({expected})
-        else:
-            try:
-                allowed_by_pos.append(set(expected))  # type: ignore[arg-type]
-            except TypeError:
-                allowed_by_pos.append({expected})  # type: ignore[list-item]
-
-    if callable(symbol_to_meter):
-        meter_of = symbol_to_meter
-    else:
-        meter_map = dict(symbol_to_meter)
-        meter_of = lambda symbol: meter_map[symbol]
-
-    length = len(pattern)
-
-    def transition(position: State, symbol: Symbol) -> State | None:
-        if not isinstance(position, int) or position >= length:
-            return None
-        allowed = allowed_by_pos[position]
-        if allowed is not None and meter_of(symbol) not in allowed:
-            return None
-        return position + 1
-
-    return DFA(
-        start_state=0,
-        accept_states={length},
-        states=set(range(length + 1)),
-        alphabet=alphabet,
-        transition_func=transition,
-        name=name,
-    )
-
-
-def cumulative_meter_acceptor(
-    length: int,
-    cost: Mapping[Symbol, int] | Callable[[Symbol], int],
-    predicate: Callable[[int, Symbol, int], bool] | None = None,
-    *,
-    alphabet: Iterable[Symbol] | None = None,
-    max_cost: int | None = None,
-    accept_costs: Iterable[int] | Callable[[int], bool] | None = None,
-    end_symbol: Symbol | None = None,
-    name: str = "cumulative_meter",
-) -> DFA:
-    """Accept strings satisfying a cumulative meter predicate.
-
-    This is the regular-constraint form of the paper-style meter predicate
-    ``pi(c, x, k)``: before emitting symbol ``x`` at 1-based position ``k``,
-    the DFA state stores the cumulative cost ``c`` of previous symbols. The
-    transition is accepted iff ``predicate(c, x, k)`` holds, then the symbol
-    cost is added to the state.
-
-    ``accept_costs`` can be used for final total-cost requirements. If
-    ``end_symbol`` is provided, once it appears, all later symbols must be the
-    same padding symbol. Passing ``max_cost`` bounds transitions and lets the
-    DFA expose an explicit finite state set.
-    """
-
-    if length < 0:
-        raise ValueError("length must be non-negative")
-    if max_cost is not None and max_cost < 0:
-        raise ValueError("max_cost must be non-negative")
-
-    if callable(cost):
-        cost_of = cost
-    else:
-        cost_map = dict(cost)
-
-        def cost_of(symbol: Symbol) -> int:
-            return cost_map[symbol]
-
-    if accept_costs is None:
-        accepts_cost = lambda total: True
-    elif callable(accept_costs):
-        accepts_cost = accept_costs
-    else:
-        accepted_totals = frozenset(int(total) for total in accept_costs)
-        accepts_cost = lambda total: total in accepted_totals
-
-    def transition(state: State, symbol: Symbol) -> State | None:
-        if not (
-            isinstance(state, tuple)
-            and len(state) == 3
-            and isinstance(state[0], int)
-            and isinstance(state[1], int)
-            and isinstance(state[2], bool)
-        ):
-            raise TypeError("cumulative-meter states must be (position, total_cost, ended)")
-
-        position, total_cost, ended = state
-        if position >= length:
-            return None
-        if ended and symbol != end_symbol:
-            return None
-
-        symbol_cost = _nonnegative_int_cost(cost_of(symbol), symbol)
-        next_position = position + 1
-        next_total = total_cost + symbol_cost
-        if max_cost is not None and next_total > max_cost:
-            return None
-        if predicate is not None and not predicate(total_cost, symbol, next_position):
-            return None
-
-        next_ended = ended or (end_symbol is not None and symbol == end_symbol)
-        return (next_position, next_total, next_ended)
-
-    def accepting(state: State) -> bool:
-        return (
-            isinstance(state, tuple)
-            and len(state) == 3
-            and state[0] == length
-            and isinstance(state[1], int)
-            and bool(accepts_cost(state[1]))
-        )
-
-    states = None
-    if max_cost is not None:
-        ended_values = (False, True) if end_symbol is not None else (False,)
-        states = {
-            (position, total, ended)
-            for position in range(length + 1)
-            for total in range(max_cost + 1)
-            for ended in ended_values
-        }
-
-    return DFA(
-        start_state=(0, 0, False),
-        states=states,
-        alphabet=alphabet,
-        transition_func=transition,
-        accept_func=accepting,
-        name=name,
-    )
-
-
-def forbidden_substring_acceptor(
-    forbidden_patterns: Iterable[Sequence[Symbol]],
-    *,
-    alphabet: Iterable[Symbol] | None = None,
-    name: str = "forbidden_substring",
-) -> DFA:
-    """Accept strings that contain none of the forbidden substrings.
-
-    The states are Aho-Corasick-style prefixes: the current state stores the
-    longest suffix of the generated sequence that is a proper prefix of a
-    forbidden pattern. Completing a forbidden pattern rejects the transition.
-    """
-
-    patterns = tuple(tuple(pattern) for pattern in forbidden_patterns)
-    if any(len(pattern) == 0 for pattern in patterns):
-        raise ValueError("empty forbidden patterns would reject every sequence")
-    if not patterns:
-        return true_acceptor(name=name)
-
-    prefixes: set[tuple[Symbol, ...]] = {()}
-    for pattern in patterns:
-        for prefix_len in range(1, len(pattern)):
-            prefixes.add(pattern[:prefix_len])
-
-    max_prefix_len = max((len(prefix) for prefix in prefixes), default=0)
-    patterns_by_len: dict[int, set[tuple[Symbol, ...]]] = {}
-    for pattern in patterns:
-        patterns_by_len.setdefault(len(pattern), set()).add(pattern)
-
-    if len(patterns_by_len) == 1:
-        forbidden_len, forbidden_set = next(iter(patterns_by_len.items()))
-
-        def completes_forbidden(candidate: tuple[Symbol, ...]) -> bool:
-            return len(candidate) >= forbidden_len and candidate[-forbidden_len:] in forbidden_set
-
-    else:
-
-        def completes_forbidden(candidate: tuple[Symbol, ...]) -> bool:
-            return any(
-                len(candidate) >= pattern_len and candidate[-pattern_len:] in pattern_set
-                for pattern_len, pattern_set in patterns_by_len.items()
-            )
-
-    def next_prefix(state: State, symbol: Symbol) -> State | None:
-        if not isinstance(state, tuple):
-            raise TypeError("forbidden-substring states must be tuples")
-        candidate = state + (symbol,)
-        if completes_forbidden(candidate):
-            return None
-        limit = min(len(candidate), max_prefix_len)
-        for size in range(limit, -1, -1):
-            suffix = candidate[-size:] if size else ()
-            if suffix in prefixes:
-                return suffix
-        return ()
-
-    transitions = None
-    if alphabet is not None:
-        alphabet_set = frozenset(alphabet)
-        transitions = {
-            state: {
-                symbol: next_state
-                for symbol in alphabet_set
-                if (next_state := next_prefix(state, symbol)) is not None
-            }
-            for state in prefixes
-        }
-    else:
-        alphabet_set = None
-
-    return DFA(
-        start_state=(),
-        accept_states=prefixes,
-        transitions=transitions,
-        states=prefixes,
-        alphabet=alphabet_set,
-        transition_func=None if transitions is not None else next_prefix,
-        name=name,
-    )
-
-
-def dense_forbidden_substring_acceptor(
-    forbidden_patterns: Iterable[Sequence[Symbol]],
-    *,
-    alphabet: Iterable[Symbol],
-    name: str = "dense_forbidden_substring",
-) -> DenseForbiddenSubstringDFA:
-    """Dense finite-alphabet acceptor for strings with no forbidden substring."""
-
-    patterns = tuple(tuple(pattern) for pattern in forbidden_patterns)
-    if any(len(pattern) == 0 for pattern in patterns):
-        raise ValueError("empty forbidden patterns would reject every sequence")
-
-    alphabet_tuple = tuple(alphabet)
-    prefix_to_id: dict[tuple[Symbol, ...], int] = {(): 0}
-    prefixes: list[tuple[Symbol, ...]] = [()]
-    for pattern in patterns:
-        for prefix_len in range(1, len(pattern)):
-            prefix = pattern[:prefix_len]
-            if prefix not in prefix_to_id:
-                prefix_to_id[prefix] = len(prefixes)
-                prefixes.append(prefix)
-
-    max_prefix_len = max((len(prefix) for prefix in prefixes), default=0)
-    patterns_by_len: dict[int, set[tuple[Symbol, ...]]] = {}
-    for pattern in patterns:
-        patterns_by_len.setdefault(len(pattern), set()).add(pattern)
-
-    if len(patterns_by_len) == 1:
-        forbidden_len, forbidden_set = next(iter(patterns_by_len.items()))
-
-        def completes_forbidden(candidate: tuple[Symbol, ...]) -> bool:
-            return len(candidate) >= forbidden_len and candidate[-forbidden_len:] in forbidden_set
-
-    else:
-
-        def completes_forbidden(candidate: tuple[Symbol, ...]) -> bool:
-            return any(
-                len(candidate) >= pattern_len and candidate[-pattern_len:] in pattern_set
-                for pattern_len, pattern_set in patterns_by_len.items()
-            )
-
-    def next_prefix(state: tuple[Symbol, ...], symbol: Symbol) -> tuple[Symbol, ...] | None:
-        candidate = state + (symbol,)
-        if completes_forbidden(candidate):
-            return None
-        limit = min(len(candidate), max_prefix_len)
-        for size in range(limit, -1, -1):
-            suffix = candidate[-size:] if size else ()
-            if suffix in prefix_to_id:
-                return suffix
-        return ()
-
-    transition_table: list[list[int]] = []
-    for prefix in prefixes:
-        row: list[int] = []
-        for symbol in alphabet_tuple:
-            next_state = next_prefix(prefix, symbol)
-            row.append(DenseForbiddenSubstringDFA.rejected_state if next_state is None else prefix_to_id[next_state])
-        transition_table.append(row)
-
-    return DenseForbiddenSubstringDFA(
-        prefixes=prefixes,
-        alphabet=alphabet_tuple,
-        transition_table=transition_table,
-        name=name,
-    )
-
-
-def max_order_acceptor(
-    reference_sequences: Iterable[Sequence[Symbol]],
-    max_order: int,
-    *,
-    alphabet: Iterable[Symbol] | None = None,
-    name: str = "max_order",
-) -> DFA:
-    """Forbid every reference substring of length ``max_order + 1``.
-
-    This is the standard regular way to enforce "do not copy beyond max order":
-    any generated window longer than ``max_order`` that appears in the reference
-    material is rejected.
-    """
-
-    if max_order < 0:
-        raise ValueError("max_order must be non-negative")
-
-    window = max_order + 1
-    forbidden: set[tuple[Symbol, ...]] = set()
-    for sequence in reference_sequences:
-        tokens = tuple(sequence)
-        for index in range(0, len(tokens) - window + 1):
-            forbidden.add(tokens[index : index + window])
-    return forbidden_substring_acceptor(forbidden, alphabet=alphabet, name=name)
-
-
-def _nonnegative_int_cost(value: int, symbol: Symbol) -> int:
-    if not isinstance(value, int):
-        raise TypeError(f"cost for symbol {symbol!r} must be an integer")
-    if value < 0:
-        raise ValueError(f"cost for symbol {symbol!r} must be non-negative")
-    return value

vendor/vo_regular_bp/vo_regular_bp/adapters.py

@@ -1,270 +0,0 @@
-"""Adapters for projects that use rich event objects instead of symbols."""
-
-from __future__ import annotations
-
-from collections.abc import Callable, Iterable, Mapping, Sequence
-from dataclasses import dataclass
-import random
-from typing import Generic, TypeVar
-
-from .backend import (
-    BackendDiagnostics,
-    ConstrainedOrderStackBackend,
-    ConstrainedOrderStackPlan,
-    prepare_constrained_order_stack_plan,
-)
-from .constraints import ConstraintSet
-from .context import Symbol
-from .order_stack_bp import OrderPolicy, OrderStackModel
-
-
-EventT = TypeVar("EventT")
-
-
-@dataclass(frozen=True)
-class EventCodec(Generic[EventT]):
-    """Encode project-specific events as hashable symbols.
-
-    ``symbol_to_event`` is optional.  It is only required when callers want to
-    decode generated symbols back into event objects.
-    """
-
-    event_to_symbol: Callable[[EventT], Symbol]
-    symbol_to_event: Mapping[Symbol, EventT] | Callable[[Symbol], EventT] | None = None
-
-    @classmethod
-    def identity(cls) -> "EventCodec[Symbol]":
-        return cls(lambda event: event, lambda symbol: symbol)
-
-    def encode_event(self, event: EventT) -> Symbol:
-        return self.event_to_symbol(event)
-
-    def encode_sequence(self, sequence: Sequence[EventT]) -> tuple[Symbol, ...]:
-        return tuple(self.encode_event(event) for event in sequence)
-
-    def encode_sequences(
-        self,
-        sequences: Iterable[Sequence[EventT]],
-    ) -> tuple[tuple[Symbol, ...], ...]:
-        return tuple(self.encode_sequence(sequence) for sequence in sequences)
-
-    def decode_symbol(self, symbol: Symbol) -> EventT:
-        decoder = self.symbol_to_event
-        if decoder is None:
-            raise ValueError("EventCodec has no symbol_to_event decoder")
-        if isinstance(decoder, Mapping):
-            return decoder[symbol]
-        return decoder(symbol)
-
-    def decode_sequence(self, sequence: Sequence[Symbol]) -> tuple[EventT, ...]:
-        return tuple(self.decode_symbol(symbol) for symbol in sequence)
-
-    def symbol_property(
-        self,
-        event_property: Callable[[EventT], object],
-    ) -> Callable[[Symbol], object]:
-        """Lift an event property function to generated symbols."""
-
-        def property_from_symbol(symbol: Symbol) -> object:
-            return event_property(self.decode_symbol(symbol))
-
-        return property_from_symbol
-
-    def with_decoder(
-        self,
-        symbol_to_event: Mapping[Symbol, EventT] | Callable[[Symbol], EventT],
-    ) -> "EventCodec[EventT]":
-        return EventCodec(self.event_to_symbol, symbol_to_event)
-
-
-@dataclass(frozen=True)
-class GeneratedEvents(Generic[EventT]):
-    """Generated events with their encoded symbols and optional order trace."""
-
-    events: tuple[EventT, ...]
-    symbols: tuple[Symbol, ...]
-    orders: tuple[int, ...] = ()
-
-
-@dataclass(frozen=True)
-class EventOrderStackBackend(Generic[EventT]):
-    """Prepared constrained order-stack sampler for event objects."""
-
-    backend: ConstrainedOrderStackBackend
-    codec: EventCodec[EventT]
-
-    @property
-    def diagnostics(self) -> BackendDiagnostics:
-        return self.backend.diagnostics
-
-    def sample_symbols(self, *, rng: random.Random | int | None = None) -> tuple[Symbol, ...]:
-        return self.backend.sample(rng=rng)
-
-    def sample_events(self, *, rng: random.Random | int | None = None) -> tuple[EventT, ...]:
-        return self.codec.decode_sequence(self.sample_symbols(rng=rng))
-
-    def sample(self, *, rng: random.Random | int | None = None) -> tuple[EventT, ...]:
-        return self.sample_events(rng=rng)
-
-    def sample_events_with_orders(
-        self,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> GeneratedEvents[EventT]:
-        generated = self.backend.sample_with_orders(rng=rng)
-        return GeneratedEvents(
-            events=self.codec.decode_sequence(generated.sequence),
-            symbols=generated.sequence,
-            orders=generated.orders,
-        )
-
-    def sample_many_events(
-        self,
-        count: int,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> list[tuple[EventT, ...]]:
-        return [
-            self.codec.decode_sequence(sequence)
-            for sequence in self.backend.sample_many(count, rng=rng)
-        ]
-
-    def sample_many_events_with_orders(
-        self,
-        count: int,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> list[GeneratedEvents[EventT]]:
-        return [
-            GeneratedEvents(
-                events=self.codec.decode_sequence(generated.sequence),
-                symbols=generated.sequence,
-                orders=generated.orders,
-            )
-            for generated in self.backend.sample_many_with_orders(count, rng=rng)
-        ]
-
-
-@dataclass(frozen=True)
-class EventOrderStackPlan(Generic[EventT]):
-    """Prefix-independent event sampler plan with encode/decode glue."""
-
-    plan: ConstrainedOrderStackPlan
-    codec: EventCodec[EventT]
-
-    @property
-    def length(self) -> int:
-        return self.plan.length
-
-    def for_prefix(self, prefix: Sequence[EventT]) -> EventOrderStackBackend[EventT]:
-        backend = self.plan.for_prefix(self.codec.encode_sequence(prefix))
-        return EventOrderStackBackend(backend=backend, codec=self.codec)
-
-    def sample_events(
-        self,
-        *,
-        prefix: Sequence[EventT],
-        rng: random.Random | int | None = None,
-    ) -> tuple[EventT, ...]:
-        return self.for_prefix(prefix).sample_events(rng=rng)
-
-    def sample_events_with_orders(
-        self,
-        *,
-        prefix: Sequence[EventT],
-        rng: random.Random | int | None = None,
-    ) -> GeneratedEvents[EventT]:
-        return self.for_prefix(prefix).sample_events_with_orders(rng=rng)
-
-
-def prepare_constrained_order_stack_from_events(
-    sequences: Iterable[Sequence[EventT]],
-    constraints: ConstraintSet | None = None,
-    *,
-    codec: EventCodec[EventT],
-    max_order: int,
-    length: int,
-    prefix: Sequence[EventT],
-    policy: OrderPolicy | None = None,
-    start_event: EventT | None = None,
-    end_event: EventT | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> EventOrderStackBackend[EventT]:
-    """Build and prepare a constrained order-stack backend from event sequences."""
-
-    plan = prepare_constrained_order_stack_plan_from_events(
-        sequences,
-        constraints,
-        codec=codec,
-        max_order=max_order,
-        length=length,
-        policy=policy,
-        start_event=start_event,
-        end_event=end_event,
-        alphabet=alphabet,
-        prefer_dense_forbidden=prefer_dense_forbidden,
-        minimize_source_graphs=minimize_source_graphs,
-    )
-    return plan.for_prefix(prefix)
-
-
-def prepare_constrained_order_stack_plan_from_events(
-    sequences: Iterable[Sequence[EventT]],
-    constraints: ConstraintSet | None = None,
-    *,
-    codec: EventCodec[EventT],
-    max_order: int,
-    length: int,
-    policy: OrderPolicy | None = None,
-    start_event: EventT | None = None,
-    end_event: EventT | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> EventOrderStackPlan[EventT]:
-    """Build an order-stack model from events and prepare a prefixless plan."""
-
-    encoded_sequences = codec.encode_sequences(sequences)
-    model = OrderStackModel.from_sequences(
-        encoded_sequences,
-        max_order=max_order,
-        start_symbol=None if start_event is None else codec.encode_event(start_event),
-        end_symbol=None if end_event is None else codec.encode_event(end_event),
-    )
-    plan = prepare_constrained_order_stack_plan(
-        model,
-        constraints,
-        length=length,
-        policy=policy,
-        alphabet=alphabet,
-        prefer_dense_forbidden=prefer_dense_forbidden,
-        minimize_source_graphs=minimize_source_graphs,
-    )
-    return EventOrderStackPlan(plan=plan, codec=codec)
-
-
-def infer_symbol_to_event(
-    sequences: Iterable[Sequence[EventT]],
-    event_to_symbol: Callable[[EventT], Symbol],
-    *,
-    strict: bool = True,
-) -> dict[Symbol, EventT]:
-    """Infer a decoder lookup from training events.
-
-    In strict mode, two unequal events mapping to the same symbol raise because
-    decoding would be ambiguous.  With ``strict=False``, the first event seen for
-    each symbol is kept.
-    """
-
-    lookup: dict[Symbol, EventT] = {}
-    for sequence in sequences:
-        for event in sequence:
-            symbol = event_to_symbol(event)
-            if symbol in lookup:
-                if strict and lookup[symbol] != event:
-                    raise ValueError(f"symbol {symbol!r} maps to multiple events")
-                continue
-            lookup[symbol] = event
-    return lookup

vendor/vo_regular_bp/vo_regular_bp/augmentation.py

@@ -1,505 +0,0 @@
-"""Virtual data augmentation for order-stack models.
-
-The classes here keep the generated symbol space absolute.  Transformations are
-used only to evaluate continuation counts as if transformed copies of the
-training sequences had been materialized.
-"""
-
-from __future__ import annotations
-
-from collections import Counter
-from collections.abc import Callable, Iterable, Iterator, Sequence
-from dataclasses import dataclass, field
-import time
-
-from .context import Context, Symbol, _as_context
-from .order_stack_bp import FixedOrderContextGraph, OrderStackModel
-from .order_stack_bp import StackEdge
-
-
-@dataclass(frozen=True)
-class SymbolTransform:
-    """A deterministic finite augmentation transform over symbols."""
-
-    name: str
-    apply_symbol: Callable[[Symbol], Symbol]
-    inverse_symbol: Callable[[Symbol], Symbol]
-    weight: float = 1.0
-
-    def apply_context(self, context: Sequence[Symbol]) -> Context:
-        return tuple(self.apply_symbol(symbol) for symbol in context)
-
-    def inverse_context(self, context: Sequence[Symbol]) -> Context:
-        return tuple(self.inverse_symbol(symbol) for symbol in context)
-
-
-def integer_shift_transform(
-    offset: int,
-    *,
-    fixed_symbols: Iterable[Symbol] = (),
-    name: str | None = None,
-) -> SymbolTransform:
-    """Return a symbol transform that adds an integer offset.
-
-    ``fixed_symbols`` are left unchanged.  This is useful for start/end sentinel
-    symbols in pitch models.
-    """
-
-    fixed = frozenset(fixed_symbols)
-    shift = int(offset)
-
-    def apply_symbol(symbol: Symbol) -> Symbol:
-        if symbol in fixed:
-            return symbol
-        return int(symbol) + shift
-
-    def inverse_symbol(symbol: Symbol) -> Symbol:
-        if symbol in fixed:
-            return symbol
-        return int(symbol) - shift
-
-    return SymbolTransform(
-        name=name or f"shift_{shift:+d}",
-        apply_symbol=apply_symbol,
-        inverse_symbol=inverse_symbol,
-    )
-
-
-def integer_shift_transforms(
-    offsets: Iterable[int],
-    *,
-    fixed_symbols: Iterable[Symbol] = (),
-) -> tuple[SymbolTransform, ...]:
-    """Return integer-shift transforms for all requested offsets."""
-
-    return tuple(
-        integer_shift_transform(offset, fixed_symbols=fixed_symbols)
-        for offset in offsets
-    )
-
-
-def materialize_transformed_sequences(
-    sequences: Iterable[Sequence[Symbol]],
-    transforms: Iterable[SymbolTransform],
-) -> tuple[tuple[Symbol, ...], ...]:
-    """Materialize transformed copies for validation and comparison."""
-
-    material = tuple(tuple(sequence) for sequence in sequences)
-    return tuple(
-        transform.apply_context(sequence)
-        for transform in transforms
-        for sequence in material
-    )
-
-
-@dataclass
-class VirtualAugmentedOrderStackModel:
-    """Order-stack model with exact virtual transformed-corpus counts.
-
-    The model answers continuation queries with the same counts as an explicit
-    augmented corpus:
-
-    ``C_aug(c -> y) = sum_g C_base(g^-1(c) -> g^-1(y))``.
-    """
-
-    base_model: OrderStackModel
-    transforms: tuple[SymbolTransform, ...]
-    _counts_cache: dict[Context, Counter[Symbol]] = field(default_factory=dict, init=False)
-    _distribution_cache: dict[tuple[Context, int | None], tuple[tuple[tuple[Symbol, float], ...], int | None]] = field(
-        default_factory=dict,
-        init=False,
-        repr=False,
-    )
-    _graph_cache: dict[int, FixedOrderContextGraph] = field(default_factory=dict, init=False)
-    _contexts_cache: dict[int, tuple[Context, ...]] = field(default_factory=dict, init=False)
-    _lazy_graph_cache: dict[tuple[Context, int], dict[int, "LazyVirtualFixedOrderContextGraph"]] = field(
-        default_factory=dict,
-        init=False,
-        repr=False,
-    )
-    _lazy_plan_graph_cache: dict[int, dict[int, "LazyVirtualFixedOrderContextGraph"]] = field(
-        default_factory=dict,
-        init=False,
-        repr=False,
-    )
-    _apply_symbol_cache: dict[tuple[int, Symbol], Symbol] = field(
-        default_factory=dict,
-        init=False,
-        repr=False,
-    )
-    _inverse_symbol_cache: dict[tuple[int, Symbol], Symbol] = field(
-        default_factory=dict,
-        init=False,
-        repr=False,
-    )
-    _inverse_context_cache: dict[tuple[int, Context], Context] = field(
-        default_factory=dict,
-        init=False,
-        repr=False,
-    )
-    _context_materialization_seconds: float = field(default=0.0, init=False, repr=False)
-    _context_materialization_calls: int = field(default=0, init=False, repr=False)
-    _context_materialization_cache_hits: int = field(default=0, init=False, repr=False)
-    _context_materialization_cache_misses: int = field(default=0, init=False, repr=False)
-    _augmented_count_calls: int = field(default=0, init=False, repr=False)
-    _augmented_count_cache_hits: int = field(default=0, init=False, repr=False)
-    _augmented_count_cache_misses: int = field(default=0, init=False, repr=False)
-
-    def __post_init__(self) -> None:
-        if not self.transforms:
-            raise ValueError("at least one transform is required")
-        if any(transform.weight <= 0.0 for transform in self.transforms):
-            raise ValueError("transform weights must be positive")
-        self.max_order = self.base_model.max_order
-        self.forbidden_symbols = frozenset(
-            self._apply_symbol(transform_index, symbol)
-            for transform_index, _transform in enumerate(self.transforms)
-            for symbol in self.base_model.forbidden_symbols
-        )
-        self.alphabet = frozenset(
-            self._apply_symbol(transform_index, symbol)
-            for transform_index, _transform in enumerate(self.transforms)
-            for symbol in self.base_model.alphabet
-        )
-
-    @classmethod
-    def from_sequences(
-        cls,
-        sequences: Iterable[Sequence[Symbol]],
-        *,
-        max_order: int,
-        transforms: Iterable[SymbolTransform],
-        start_symbol: Symbol | None = None,
-        end_symbol: Symbol | None = None,
-    ) -> "VirtualAugmentedOrderStackModel":
-        base_model = OrderStackModel.from_sequences(
-            sequences,
-            max_order=max_order,
-            start_symbol=start_symbol,
-            end_symbol=end_symbol,
-        )
-        return cls(base_model=base_model, transforms=tuple(transforms))
-
-    @property
-    def base_context_count(self) -> int:
-        return len(self.base_model.counts)
-
-    @property
-    def virtual_event_multiplier(self) -> int:
-        return len(self.transforms)
-
-    def virtual_context_count(self, order: int | None = None) -> int:
-        if order is None:
-            order = self.max_order
-        return sum(1 for _context in self.iter_contexts(order))
-
-    def iter_contexts(self, order: int) -> Iterable[Context]:
-        self._context_materialization_calls += 1
-        cached = self._contexts_cache.get(order)
-        if cached is not None:
-            self._context_materialization_cache_hits += 1
-            return cached
-        self._context_materialization_cache_misses += 1
-        started = time.perf_counter()
-        contexts: set[Context] = set()
-        for context in self.base_model.counts:
-            if len(context) > order:
-                continue
-            for transform_index, _transform in enumerate(self.transforms):
-                contexts.add(
-                    tuple(
-                        self._apply_symbol(transform_index, symbol)
-                        for symbol in context
-                    )
-                )
-        result = tuple(contexts)
-        self._contexts_cache[order] = result
-        self._context_materialization_seconds += time.perf_counter() - started
-        return result
-
-    def augmented_counts(self, context: Iterable[Symbol] | Context) -> Counter[Symbol]:
-        self._augmented_count_calls += 1
-        state = _as_context(context)
-        cached = self._counts_cache.get(state)
-        if cached is not None:
-            self._augmented_count_cache_hits += 1
-            return cached
-
-        self._augmented_count_cache_misses += 1
-        counts: Counter[Symbol] = Counter()
-        for transform_index, _transform in enumerate(self.transforms):
-            inverse_context = self._inverse_context(transform_index, state)
-            base_counts = self.base_model.counts.get(inverse_context)
-            if not base_counts:
-                continue
-            for base_symbol, count in base_counts.items():
-                counts[self._apply_symbol(transform_index, base_symbol)] += (
-                    count * self.transforms[transform_index].weight
-                )
-        self._counts_cache[state] = counts
-        return counts
-
-    def virtual_diagnostics(self) -> dict[str, int | float]:
-        return {
-            "virtual_context_materialization_seconds": self._context_materialization_seconds,
-            "virtual_context_materialization_calls": self._context_materialization_calls,
-            "virtual_context_materialization_cache_hits": (
-                self._context_materialization_cache_hits
-            ),
-            "virtual_context_materialization_cache_misses": (
-                self._context_materialization_cache_misses
-            ),
-            "augmented_count_calls": self._augmented_count_calls,
-            "augmented_count_cache_hits": self._augmented_count_cache_hits,
-            "augmented_count_cache_misses": self._augmented_count_cache_misses,
-        }
-
-    def _apply_symbol(self, transform_index: int, symbol: Symbol) -> Symbol:
-        key = (transform_index, symbol)
-        cached = self._apply_symbol_cache.get(key)
-        if cached is not None:
-            return cached
-        transformed = self.transforms[transform_index].apply_symbol(symbol)
-        self._apply_symbol_cache[key] = transformed
-        return transformed
-
-    def _inverse_symbol(self, transform_index: int, symbol: Symbol) -> Symbol:
-        key = (transform_index, symbol)
-        cached = self._inverse_symbol_cache.get(key)
-        if cached is not None:
-            return cached
-        transformed = self.transforms[transform_index].inverse_symbol(symbol)
-        self._inverse_symbol_cache[key] = transformed
-        return transformed
-
-    def _inverse_context(self, transform_index: int, context: Sequence[Symbol]) -> Context:
-        state = _as_context(context)
-        key = (transform_index, state)
-        cached = self._inverse_context_cache.get(key)
-        if cached is not None:
-            return cached
-        transformed = tuple(self._inverse_symbol(transform_index, symbol) for symbol in state)
-        self._inverse_context_cache[key] = transformed
-        return transformed
-
-    def longest_available_suffix(
-        self,
-        context: Iterable[Symbol] | Context,
-        *,
-        max_order: int | None = None,
-    ) -> Context | None:
-        state = _as_context(context)
-        order_limit = min(self.max_order if max_order is None else max_order, len(state))
-        for order in range(order_limit, 0, -1):
-            suffix = state[-order:]
-            if self.augmented_counts(suffix):
-                return suffix
-        return None
-
-    def continuation_distribution_with_order(
-        self,
-        context: Iterable[Symbol] | Context,
-        *,
-        max_order: int | None = None,
-    ) -> tuple[tuple[tuple[Symbol, float], ...], int | None]:
-        state = _as_context(context)
-        cache_key = (state, max_order)
-        cached = self._distribution_cache.get(cache_key)
-        if cached is not None:
-            return cached
-
-        suffix = self.longest_available_suffix(state, max_order=max_order)
-        if suffix is None:
-            result = ((), None)
-            self._distribution_cache[cache_key] = result
-            return result
-
-        counts = self.augmented_counts(suffix)
-        total = float(sum(counts.values()))
-        if total <= 0.0:
-            result = ((), None)
-        else:
-            result = (
-                tuple((symbol, float(count) / total) for symbol, count in counts.items()),
-                len(suffix),
-            )
-        self._distribution_cache[cache_key] = result
-        return result
-
-    def compile_graph(self, order: int) -> FixedOrderContextGraph:
-        cached = self._graph_cache.get(order)
-        if cached is not None:
-            return cached
-        graph = FixedOrderContextGraph.from_model(self, order=order)
-        self._graph_cache[order] = graph
-        return graph
-
-    def compile_graphs_for_plan(
-        self,
-        *,
-        length: int,
-    ) -> dict[int, "LazyVirtualFixedOrderContextGraph"]:
-        if length < 0:
-            raise ValueError("length must be non-negative")
-        cache_key = int(length)
-        cached = self._lazy_plan_graph_cache.get(cache_key)
-        if cached is not None:
-            return cached
-
-        graphs = self._new_lazy_graphs()
-        self._lazy_plan_graph_cache[cache_key] = graphs
-        return graphs
-
-    def compile_graphs_for_prefix(
-        self,
-        *,
-        prefix: Sequence[Symbol],
-        length: int,
-    ) -> dict[int, "LazyVirtualFixedOrderContextGraph"]:
-        if length < 0:
-            raise ValueError("length must be non-negative")
-        prefix_context = tuple(prefix)
-        cache_key = (prefix_context, int(length))
-        cached = self._lazy_graph_cache.get(cache_key)
-        if cached is not None:
-            return cached
-
-        graphs = self._new_lazy_graphs()
-        self._lazy_graph_cache[cache_key] = graphs
-        return graphs
-
-    def _new_lazy_graphs(self) -> dict[int, "LazyVirtualFixedOrderContextGraph"]:
-        return {
-            order: LazyVirtualFixedOrderContextGraph(
-                self,
-                order=order,
-            )
-            for order in range(1, self.max_order + 1)
-        }
-
-
-class _LazyOutgoing:
-    def __init__(self, graph: "LazyVirtualFixedOrderContextGraph") -> None:
-        self.graph = graph
-
-    def __getitem__(self, state: int) -> list[StackEdge]:
-        return self.graph.outgoing_for_state(state)
-
-    def __iter__(self) -> Iterator[list[StackEdge]]:
-        for state in range(len(self.graph.contexts)):
-            yield self.graph.outgoing_for_state(state)
-
-    def __len__(self) -> int:
-        return len(self.graph.contexts)
-
-
-class LazyVirtualFixedOrderContextGraph:
-    """Lazily materialized fixed-order graph for virtual augmentation."""
-
-    def __init__(
-        self,
-        model: VirtualAugmentedOrderStackModel,
-        *,
-        order: int,
-        allowed_contexts: Iterable[Context] | None = None,
-    ) -> None:
-        if order < 1 or order > model.max_order:
-            raise ValueError(f"order must be between 1 and {model.max_order}")
-        self.model = model
-        self.order = int(order)
-        self._allowed_contexts_source = (
-            None if allowed_contexts is None else tuple(allowed_contexts)
-        )
-        self._allowed_contexts: set[Context] | None = None
-        self.contexts: list[Context] = []
-        self.context_to_id: dict[Context, int] = {}
-        self._outgoing_cache: dict[int, list[StackEdge]] = {}
-        self.outgoing = _LazyOutgoing(self)
-        self.outgoing_row_calls = 0
-        self.outgoing_row_cache_hits = 0
-        self.outgoing_row_cache_misses = 0
-        self.outgoing_row_materialization_seconds = 0.0
-
-    @property
-    def allowed_contexts(self) -> set[Context]:
-        return self._ensure_allowed_contexts()
-
-    def truncate_context(self, context: Iterable[Symbol] | Context) -> Context:
-        state = _as_context(context)
-        if len(state) <= self.order:
-            return state
-        return state[-self.order:]
-
-    def next_context(self, context: Iterable[Symbol] | Context, symbol: Symbol) -> Context:
-        return self.truncate_context(_as_context(context) + (symbol,))
-
-    def state_id(self, context: Iterable[Symbol] | Context) -> int | None:
-        state = self.truncate_context(context)
-        if state not in self._ensure_allowed_contexts():
-            return None
-        return self._add_context(state)
-
-    @property
-    def edge_count(self) -> int:
-        return sum(len(edges) for edges in self.outgoing)
-
-    def outgoing_for_state(self, state: int) -> list[StackEdge]:
-        self.outgoing_row_calls += 1
-        cached = self._outgoing_cache.get(state)
-        if cached is not None:
-            self.outgoing_row_cache_hits += 1
-            return cached
-        self.outgoing_row_cache_misses += 1
-        started = time.perf_counter()
-        context = self.contexts[state]
-        distribution, effective_order = self.model.continuation_distribution_with_order(
-            context,
-            max_order=self.order,
-        )
-        edges: list[StackEdge] = []
-        order = self.order
-        allowed_contexts = self._ensure_allowed_contexts()
-        for symbol, probability in distribution:
-            candidate_context = context + (symbol,)
-            dst_context = (
-                candidate_context
-                if len(candidate_context) <= order
-                else candidate_context[-order:]
-            )
-            if dst_context not in allowed_contexts:
-                continue
-            dst = self._add_context(dst_context)
-            edges.append(
-                StackEdge(
-                    src=state,
-                    dst=dst,
-                    symbol=symbol,
-                    probability=float(probability),
-                    order=effective_order or 0,
-                )
-            )
-        self._outgoing_cache[state] = edges
-        self.outgoing_row_materialization_seconds += time.perf_counter() - started
-        return edges
-
-    def _ensure_allowed_contexts(self) -> set[Context]:
-        allowed_contexts = self._allowed_contexts
-        if allowed_contexts is not None:
-            return allowed_contexts
-        source = (
-            self.model.iter_contexts(self.order)
-            if self._allowed_contexts_source is None
-            else self._allowed_contexts_source
-        )
-        allowed_contexts = {self.truncate_context(context) for context in source}
-        self._allowed_contexts = allowed_contexts
-        return allowed_contexts
-
-    def _add_context(self, context: Context) -> int:
-        found = self.context_to_id.get(context)
-        if found is not None:
-            return found
-        context_id = len(self.contexts)
-        self.context_to_id[context] = context_id
-        self.contexts.append(context)
-        return context_id

vendor/vo_regular_bp/vo_regular_bp/backend.py

@@ -1,1066 +0,0 @@
-"""Stable public backend entry points.
-
-This module is the library-facing layer.  It compiles generic constraint
-specifications and delegates to the current BP engines.  Future optimized
-engines should be wired behind these functions without changing the public
-surface.
-"""
-
-from __future__ import annotations
-
-import bisect
-from collections.abc import Callable, Hashable, Iterable, Sequence
-from dataclasses import dataclass
-import random
-
-from .acceptors import DFA
-from .constraint_builders import combine_constraints
-from .constraints import ConstraintSet, compile_constraints
-from .context import Symbol
-from .order_stack_bp import (
-    OrderPolicy,
-    OrderSampleStep,
-    OrderStackBPPlan,
-    OrderStackModel,
-    RegularOrderStackBPResult,
-    RegularOrderStackBPPlan,
-    OrderStackBPResult,
-    prepare_order_stack_bp,
-    prepare_order_stack_masked_dfa_bp,
-    run_order_stack_bp,
-    run_order_stack_masked_dfa_bp,
-)
-from .positional_bp import AllowedForbiddenSymbols
-
-
-@dataclass(frozen=True)
-class GeneratedSequence:
-    """A generated sequence plus optional order diagnostics."""
-
-    sequence: tuple[Symbol, ...]
-    orders: tuple[int, ...] = ()
-
-
-@dataclass(frozen=True)
-class BackendDiagnostics:
-    """Runtime-independent diagnostics exposed by prepared backends."""
-
-    backend: str
-    length: int
-    max_order: int
-    context_states: int
-    context_edges: int
-    regular_product_states: int | None = None
-    regular_product_states_time_indexed: int | None = None
-    regular_product_edges: int | None = None
-    regular_transition_rows: int | None = None
-    regular_transition_row_cache_hits: int | None = None
-    regular_transition_row_cache_misses: int | None = None
-    regular_accepted_transitions: int | None = None
-    regular_beta_state_expansions: int | None = None
-    regular_beta_cache_hits: int | None = None
-    regular_beta_cache_misses: int | None = None
-    regular_acceptor_symbol_transition_cache_hits: int | None = None
-    regular_acceptor_symbol_transition_cache_misses: int | None = None
-    duration_view_quotient_states: int | None = None
-    duration_view_quotient_classes: int | None = None
-    duration_view_quotient_state_reduction: float | None = None
-    duration_view_quotient_edges: int | None = None
-    duration_view_quotient_projected_edges: int | None = None
-    duration_view_quotient_ignored_edges: int | None = None
-    duration_view_quotient_quotient_edges: int | None = None
-    duration_view_quotient_projected_edge_reduction: float | None = None
-    duration_view_quotient_max_class_size: int | None = None
-    duration_view_quotient_refinement_rounds: int | None = None
-    duration_view_quotient_seconds: float | None = None
-    duration_view_quotient_order_stats: tuple[dict[str, object], ...] = ()
-    virtual_context_materialization_seconds: float | None = None
-    virtual_context_materialization_calls: int | None = None
-    virtual_context_materialization_cache_hits: int | None = None
-    virtual_context_materialization_cache_misses: int | None = None
-    augmented_count_calls: int | None = None
-    augmented_count_cache_hits: int | None = None
-    augmented_count_cache_misses: int | None = None
-    virtual_outgoing_row_calls: int | None = None
-    virtual_outgoing_row_cache_hits: int | None = None
-    virtual_outgoing_row_cache_misses: int | None = None
-    success_mass: float | None = None
-    start_order_masses: tuple[tuple[int, float], ...] = ()
-
-    def as_dict(self) -> dict[str, object]:
-        return {
-            "backend": self.backend,
-            "length": self.length,
-            "max_order": self.max_order,
-            "context_states": self.context_states,
-            "context_edges": self.context_edges,
-            "regular_product_states": self.regular_product_states,
-            "regular_product_states_time_indexed": self.regular_product_states_time_indexed,
-            "regular_product_edges": self.regular_product_edges,
-            "regular_transition_rows": self.regular_transition_rows,
-            "regular_transition_row_cache_hits": self.regular_transition_row_cache_hits,
-            "regular_transition_row_cache_misses": self.regular_transition_row_cache_misses,
-            "regular_accepted_transitions": self.regular_accepted_transitions,
-            "regular_beta_state_expansions": self.regular_beta_state_expansions,
-            "regular_beta_cache_hits": self.regular_beta_cache_hits,
-            "regular_beta_cache_misses": self.regular_beta_cache_misses,
-            "regular_acceptor_symbol_transition_cache_hits": (
-                self.regular_acceptor_symbol_transition_cache_hits
-            ),
-            "regular_acceptor_symbol_transition_cache_misses": (
-                self.regular_acceptor_symbol_transition_cache_misses
-            ),
-            "duration_view_quotient_states": self.duration_view_quotient_states,
-            "duration_view_quotient_classes": self.duration_view_quotient_classes,
-            "duration_view_quotient_state_reduction": (
-                self.duration_view_quotient_state_reduction
-            ),
-            "duration_view_quotient_edges": self.duration_view_quotient_edges,
-            "duration_view_quotient_projected_edges": (
-                self.duration_view_quotient_projected_edges
-            ),
-            "duration_view_quotient_ignored_edges": (
-                self.duration_view_quotient_ignored_edges
-            ),
-            "duration_view_quotient_quotient_edges": (
-                self.duration_view_quotient_quotient_edges
-            ),
-            "duration_view_quotient_projected_edge_reduction": (
-                self.duration_view_quotient_projected_edge_reduction
-            ),
-            "duration_view_quotient_max_class_size": (
-                self.duration_view_quotient_max_class_size
-            ),
-            "duration_view_quotient_refinement_rounds": (
-                self.duration_view_quotient_refinement_rounds
-            ),
-            "duration_view_quotient_seconds": self.duration_view_quotient_seconds,
-            "duration_view_quotient_order_stats": (
-                self.duration_view_quotient_order_stats
-            ),
-            "virtual_context_materialization_seconds": (
-                self.virtual_context_materialization_seconds
-            ),
-            "virtual_context_materialization_calls": self.virtual_context_materialization_calls,
-            "virtual_context_materialization_cache_hits": (
-                self.virtual_context_materialization_cache_hits
-            ),
-            "virtual_context_materialization_cache_misses": (
-                self.virtual_context_materialization_cache_misses
-            ),
-            "augmented_count_calls": self.augmented_count_calls,
-            "augmented_count_cache_hits": self.augmented_count_cache_hits,
-            "augmented_count_cache_misses": self.augmented_count_cache_misses,
-            "virtual_outgoing_row_calls": self.virtual_outgoing_row_calls,
-            "virtual_outgoing_row_cache_hits": self.virtual_outgoing_row_cache_hits,
-            "virtual_outgoing_row_cache_misses": self.virtual_outgoing_row_cache_misses,
-            "success_mass": self.success_mass,
-            "start_order_masses": self.start_order_masses,
-        }
-
-
-@dataclass(frozen=True)
-class UntilLengthDiagnostics:
-    """Diagnostics for one feasible first-hit continuation length."""
-
-    length: int
-    weight: float
-    backend: BackendDiagnostics
-
-    def as_dict(self) -> dict[str, object]:
-        return {
-            "length": self.length,
-            "weight": self.weight,
-            "backend": self.backend.as_dict(),
-        }
-
-
-@dataclass(frozen=True)
-class UntilOrderStackDiagnostics:
-    """Diagnostics exposed by variable-length first-hit backends."""
-
-    backend: str
-    min_length: int
-    max_length: int
-    feasible_lengths: tuple[int, ...]
-    length_weights: tuple[tuple[int, float], ...]
-    length_diagnostics: tuple[UntilLengthDiagnostics, ...]
-
-    def as_dict(self) -> dict[str, object]:
-        return {
-            "backend": self.backend,
-            "min_length": self.min_length,
-            "max_length": self.max_length,
-            "feasible_lengths": self.feasible_lengths,
-            "length_weights": self.length_weights,
-            "length_diagnostics": tuple(
-                diagnostic.as_dict() for diagnostic in self.length_diagnostics
-            ),
-        }
-
-
-def _virtual_model_diagnostics(model: object) -> dict[str, object]:
-    diagnostics = getattr(model, "virtual_diagnostics", None)
-    if callable(diagnostics):
-        return dict(diagnostics())
-    return {}
-
-
-def _virtual_graph_outgoing_diagnostics(
-    graphs: object,
-) -> dict[str, int]:
-    values = tuple(graphs.values()) if isinstance(graphs, dict) else ()
-    return {
-        "virtual_outgoing_row_calls": sum(
-            int(getattr(graph, "outgoing_row_calls", 0)) for graph in values
-        ),
-        "virtual_outgoing_row_cache_hits": sum(
-            int(getattr(graph, "outgoing_row_cache_hits", 0))
-            for graph in values
-        ),
-        "virtual_outgoing_row_cache_misses": sum(
-            int(getattr(graph, "outgoing_row_cache_misses", 0))
-            for graph in values
-        ),
-    }
-
-
-@dataclass(frozen=True)
-class ConstrainedOrderStackBackend:
-    """Prepared reusable constrained order-stack sampler.
-
-    External projects can keep this object around after compilation/BP and call
-    the sampling methods repeatedly.  The concrete result remains available for
-    advanced inspection, but regular users should prefer this stable wrapper.
-    """
-
-    result: OrderStackBPResult | RegularOrderStackBPResult
-
-    @property
-    def diagnostics(self) -> BackendDiagnostics:
-        is_regular = isinstance(self.result, RegularOrderStackBPResult)
-        virtual_diagnostics = _virtual_model_diagnostics(self.result.model)
-        virtual_outgoing_diagnostics = _virtual_graph_outgoing_diagnostics(
-            self.result.graphs,
-        )
-        duration_view = (
-            self.result.duration_view_quotient_diagnostics
-            if is_regular
-            else None
-        )
-        return BackendDiagnostics(
-            backend="order_stack_regular" if is_regular else "order_stack_positional",
-            length=self.result.length,
-            max_order=self.result.model.max_order,
-            context_states=self.result.context_state_count,
-            context_edges=self.result.context_edge_count,
-            regular_product_states=self.result.product_state_count if is_regular else None,
-            regular_product_states_time_indexed=(
-                self.result.time_indexed_product_state_count if is_regular else None
-            ),
-            regular_product_edges=self.result.product_edge_count if is_regular else None,
-            regular_transition_rows=(
-                self.result.regular_transition_row_count if is_regular else None
-            ),
-            regular_transition_row_cache_hits=(
-                self.result.regular_transition_row_cache_hits if is_regular else None
-            ),
-            regular_transition_row_cache_misses=(
-                self.result.regular_transition_row_cache_misses if is_regular else None
-            ),
-            regular_accepted_transitions=(
-                self.result.regular_accepted_transition_count if is_regular else None
-            ),
-            regular_beta_state_expansions=(
-                self.result.regular_beta_state_expansions if is_regular else None
-            ),
-            regular_beta_cache_hits=(
-                self.result.regular_beta_cache_hits if is_regular else None
-            ),
-            regular_beta_cache_misses=(
-                self.result.regular_beta_cache_misses if is_regular else None
-            ),
-            regular_acceptor_symbol_transition_cache_hits=(
-                self.result.regular_acceptor_symbol_transition_cache_hits
-                if is_regular
-                else None
-            ),
-            regular_acceptor_symbol_transition_cache_misses=(
-                self.result.regular_acceptor_symbol_transition_cache_misses
-                if is_regular
-                else None
-            ),
-            duration_view_quotient_states=(
-                duration_view.states if duration_view is not None else None
-            ),
-            duration_view_quotient_classes=(
-                duration_view.classes if duration_view is not None else None
-            ),
-            duration_view_quotient_state_reduction=(
-                duration_view.state_reduction if duration_view is not None else None
-            ),
-            duration_view_quotient_edges=(
-                duration_view.edges if duration_view is not None else None
-            ),
-            duration_view_quotient_projected_edges=(
-                duration_view.projected_edges if duration_view is not None else None
-            ),
-            duration_view_quotient_ignored_edges=(
-                duration_view.ignored_edges if duration_view is not None else None
-            ),
-            duration_view_quotient_quotient_edges=(
-                duration_view.quotient_edges if duration_view is not None else None
-            ),
-            duration_view_quotient_projected_edge_reduction=(
-                duration_view.projected_edge_reduction
-                if duration_view is not None
-                else None
-            ),
-            duration_view_quotient_max_class_size=(
-                duration_view.max_class_size if duration_view is not None else None
-            ),
-            duration_view_quotient_refinement_rounds=(
-                duration_view.refinement_rounds if duration_view is not None else None
-            ),
-            duration_view_quotient_seconds=(
-                duration_view.seconds if duration_view is not None else None
-            ),
-            duration_view_quotient_order_stats=(
-                tuple(
-                    stats.as_dict()
-                    for stats in duration_view.orders
-                )
-                if duration_view is not None
-                else ()
-            ),
-            virtual_context_materialization_seconds=virtual_diagnostics.get(
-                "virtual_context_materialization_seconds",
-            ),
-            virtual_context_materialization_calls=virtual_diagnostics.get(
-                "virtual_context_materialization_calls",
-            ),
-            virtual_context_materialization_cache_hits=virtual_diagnostics.get(
-                "virtual_context_materialization_cache_hits",
-            ),
-            virtual_context_materialization_cache_misses=virtual_diagnostics.get(
-                "virtual_context_materialization_cache_misses",
-            ),
-            augmented_count_calls=virtual_diagnostics.get("augmented_count_calls"),
-            augmented_count_cache_hits=virtual_diagnostics.get("augmented_count_cache_hits"),
-            augmented_count_cache_misses=virtual_diagnostics.get(
-                "augmented_count_cache_misses",
-            ),
-            virtual_outgoing_row_calls=virtual_outgoing_diagnostics[
-                "virtual_outgoing_row_calls"
-            ],
-            virtual_outgoing_row_cache_hits=virtual_outgoing_diagnostics[
-                "virtual_outgoing_row_cache_hits"
-            ],
-            virtual_outgoing_row_cache_misses=virtual_outgoing_diagnostics[
-                "virtual_outgoing_row_cache_misses"
-            ],
-            success_mass=self.result.success_mass,
-            start_order_masses=self.result.start_order_masses(),
-        )
-
-    def sample(self, *, rng: random.Random | int | None = None) -> tuple[Symbol, ...]:
-        return self.result.sample(rng=rng)
-
-    def sample_many(
-        self,
-        count: int,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> list[tuple[Symbol, ...]]:
-        return [
-            sequence
-            for sequence, _orders in self.result.sample_many_with_orders(count, rng=rng)
-        ]
-
-    def sample_with_orders(self, *, rng: random.Random | int | None = None) -> GeneratedSequence:
-        sequence, orders = self.result.sample_with_orders(rng=rng)
-        return GeneratedSequence(sequence=sequence, orders=orders)
-
-    def sample_many_with_orders(
-        self,
-        count: int,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> list[GeneratedSequence]:
-        return [
-            GeneratedSequence(sequence=sequence, orders=orders)
-            for sequence, orders in self.result.sample_many_with_orders(count, rng=rng)
-        ]
-
-    def sample_with_trace(
-        self,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> tuple[tuple[Symbol, ...], tuple[OrderSampleStep, ...]]:
-        return self.result.sample_with_trace(rng=rng)
-
-
-@dataclass(frozen=True)
-class ConstrainedOrderStackPlan:
-    """Prefix-independent constrained order-stack preparation.
-
-    Keep this object around when the model, horizon, and constraints are fixed
-    but callers need to sample from many different prefixes. ``for_prefix`` is
-    cheap and returns the existing prefix-bound backend wrapper.
-    """
-
-    plan: OrderStackBPPlan | RegularOrderStackBPPlan
-
-    @property
-    def length(self) -> int:
-        return self.plan.length
-
-    @property
-    def is_regular(self) -> bool:
-        return isinstance(self.plan, RegularOrderStackBPPlan)
-
-    def for_prefix(self, prefix: Sequence[Symbol]) -> ConstrainedOrderStackBackend:
-        return ConstrainedOrderStackBackend(self.plan.for_prefix(prefix))
-
-    def sample(
-        self,
-        *,
-        prefix: Sequence[Symbol],
-        rng: random.Random | int | None = None,
-    ) -> tuple[Symbol, ...]:
-        return self.for_prefix(prefix).sample(rng=rng)
-
-    def sample_with_orders(
-        self,
-        *,
-        prefix: Sequence[Symbol],
-        rng: random.Random | int | None = None,
-    ) -> GeneratedSequence:
-        return self.for_prefix(prefix).sample_with_orders(rng=rng)
-
-
-@dataclass(frozen=True)
-class ConstrainedOrderStackSupportPlan:
-    """Reusable hard-support plan for dynamic soft regular acceptors.
-
-    The wrapped hard plan owns the compiled source/order-stack graphs and stable
-    constraints. ``with_soft_acceptor`` creates a prefix-bound backend with fresh
-    regular caches so changing soft weights or topology cannot reuse stale beta
-    values.
-    """
-
-    plan: ConstrainedOrderStackPlan
-
-    @property
-    def length(self) -> int:
-        return self.plan.length
-
-    @property
-    def is_regular(self) -> bool:
-        return self.plan.is_regular
-
-    def for_prefix(self, prefix: Sequence[Symbol]) -> ConstrainedOrderStackBackend:
-        """Bind the hard-only support plan to a prefix."""
-
-        return self.plan.for_prefix(prefix)
-
-    def with_soft_acceptor(
-        self,
-        soft_acceptor: DFA | None,
-        *,
-        prefix: Sequence[Symbol],
-        soft_start_acceptor_state: Hashable | None = None,
-    ) -> ConstrainedOrderStackBackend:
-        """Bind a dynamic soft acceptor on top of the cached hard support.
-
-        Passing ``None`` samples from the hard support only. For a real soft
-        acceptor, the returned backend has new regular transition-row and beta
-        caches but reuses the source/order-stack graphs and stable masks.
-        """
-
-        if soft_acceptor is None:
-            return self.for_prefix(prefix)
-
-        lower_plan = self.plan.plan
-        soft0 = (
-            soft_acceptor.start_state
-            if soft_start_acceptor_state is None
-            else soft_start_acceptor_state
-        )
-        if isinstance(lower_plan, RegularOrderStackBPPlan):
-            soft_plan = lower_plan.with_soft_acceptor(
-                soft_acceptor,
-                start_acceptor_state=(lower_plan.start_acceptor_state, soft0),
-            )
-        else:
-            soft_plan = lower_plan.with_regular_acceptor(
-                soft_acceptor,
-                start_acceptor_state=soft0,
-            )
-        return ConstrainedOrderStackBackend(soft_plan.for_prefix(prefix))
-
-
-@dataclass(frozen=True)
-class UntilOrderStackBackend:
-    """Prepared reusable first-hit order-stack sampler."""
-
-    backends: tuple[ConstrainedOrderStackBackend, ...]
-    length_weights: tuple[float, ...]
-    min_length: int
-    max_length: int
-
-    def __post_init__(self) -> None:
-        if not self.backends:
-            raise ValueError("at least one feasible length backend is required")
-        if len(self.backends) != len(self.length_weights):
-            raise ValueError("backends and length_weights must have the same length")
-        if any(weight < 0.0 for weight in self.length_weights):
-            raise ValueError("length weights must be non-negative")
-        if sum(self.length_weights) <= 0.0:
-            raise ValueError("at least one length weight must be positive")
-
-    @property
-    def feasible_lengths(self) -> tuple[int, ...]:
-        return tuple(backend.result.length for backend in self.backends)
-
-    @property
-    def diagnostics(self) -> UntilOrderStackDiagnostics:
-        length_diagnostics = tuple(
-            UntilLengthDiagnostics(
-                length=backend.result.length,
-                weight=weight,
-                backend=backend.diagnostics,
-            )
-            for backend, weight in zip(self.backends, self.length_weights)
-        )
-        return UntilOrderStackDiagnostics(
-            backend="order_stack_until",
-            min_length=self.min_length,
-            max_length=self.max_length,
-            feasible_lengths=self.feasible_lengths,
-            length_weights=tuple(
-                (backend.result.length, weight)
-                for backend, weight in zip(self.backends, self.length_weights)
-            ),
-            length_diagnostics=length_diagnostics,
-        )
-
-    def sample(self, *, rng: random.Random | int | None = None) -> tuple[Symbol, ...]:
-        generator = _coerce_backend_rng(rng)
-        return self._choose_backend(generator).sample(rng=generator)
-
-    def sample_many(
-        self,
-        count: int,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> list[tuple[Symbol, ...]]:
-        generator = _coerce_backend_rng(rng)
-        return [self.sample(rng=generator) for _ in range(count)]
-
-    def sample_with_orders(
-        self,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> GeneratedSequence:
-        generator = _coerce_backend_rng(rng)
-        return self._choose_backend(generator).sample_with_orders(rng=generator)
-
-    def sample_many_with_orders(
-        self,
-        count: int,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> list[GeneratedSequence]:
-        generator = _coerce_backend_rng(rng)
-        return [self.sample_with_orders(rng=generator) for _ in range(count)]
-
-    def sample_with_trace(
-        self,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> tuple[tuple[Symbol, ...], tuple[OrderSampleStep, ...]]:
-        generator = _coerce_backend_rng(rng)
-        return self._choose_backend(generator).sample_with_trace(rng=generator)
-
-    def _choose_backend(self, rng: random.Random) -> ConstrainedOrderStackBackend:
-        cumulative = _cumulative_weights(self.length_weights)
-        index = bisect.bisect_left(cumulative, rng.random() * cumulative[-1])
-        if index >= len(self.backends):
-            index = len(self.backends) - 1
-        return self.backends[index]
-
-
-def run_constrained_order_stack(
-    model: OrderStackModel,
-    constraints: ConstraintSet | None = None,
-    *,
-    length: int,
-    prefix: Sequence[Symbol],
-    policy: OrderPolicy | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    allowed_forbidden_symbols: AllowedForbiddenSymbols | None = None,
-    minimize_source_graphs: bool = False,
-) -> OrderStackBPResult | RegularOrderStackBPResult:
-    """Run constrained order-stack BP through the public constraint API.
-
-    Purely positional constraints use the positional order-stack backend.  Any
-    regular component uses the regular backend with positional constraints kept
-    as masks instead of being folded into the DFA state.
-    """
-
-    compiled = compile_constraints(
-        constraints,
-        length=length,
-        alphabet=tuple(model.alphabet if alphabet is None else alphabet),
-        prefer_dense_forbidden=prefer_dense_forbidden,
-    )
-    if compiled.regular_acceptor is None:
-        return run_order_stack_bp(
-            model,
-            length=length,
-            prefix=prefix,
-            constraints=compiled.positional,
-            allowed_forbidden_symbols=allowed_forbidden_symbols,
-            policy=policy,
-            minimize_source_graphs=minimize_source_graphs,
-        )
-    return run_order_stack_masked_dfa_bp(
-        model,
-        compiled.regular_acceptor,
-        length=length,
-        prefix=prefix,
-        constraints=compiled.positional,
-        allowed_forbidden_symbols=allowed_forbidden_symbols,
-        policy=policy,
-        minimize_source_graphs=minimize_source_graphs,
-    )
-
-
-def prepare_constrained_order_stack_plan(
-    model: OrderStackModel,
-    constraints: ConstraintSet | None = None,
-    *,
-    length: int,
-    policy: OrderPolicy | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    allowed_forbidden_symbols: AllowedForbiddenSymbols | None = None,
-    minimize_source_graphs: bool = False,
-) -> ConstrainedOrderStackPlan:
-    """Prepare reusable constrained order-stack BP without binding a prefix."""
-
-    compiled = compile_constraints(
-        constraints,
-        length=length,
-        alphabet=tuple(model.alphabet if alphabet is None else alphabet),
-        prefer_dense_forbidden=prefer_dense_forbidden,
-    )
-    if compiled.regular_acceptor is None:
-        return ConstrainedOrderStackPlan(
-            prepare_order_stack_bp(
-                model,
-                length=length,
-                constraints=compiled.positional,
-                allowed_forbidden_symbols=allowed_forbidden_symbols,
-                policy=policy,
-                minimize_source_graphs=minimize_source_graphs,
-            )
-        )
-    return ConstrainedOrderStackPlan(
-        prepare_order_stack_masked_dfa_bp(
-            model,
-            compiled.regular_acceptor,
-            length=length,
-            constraints=compiled.positional,
-            allowed_forbidden_symbols=allowed_forbidden_symbols,
-            policy=policy,
-            minimize_source_graphs=minimize_source_graphs,
-        )
-    )
-
-
-def prepare_constrained_order_stack_support_plan(
-    model: OrderStackModel,
-    constraints: ConstraintSet | None = None,
-    *,
-    length: int,
-    policy: OrderPolicy | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    allowed_forbidden_symbols: AllowedForbiddenSymbols | None = None,
-    minimize_source_graphs: bool = False,
-) -> ConstrainedOrderStackSupportPlan:
-    """Prepare hard support once for repeated dynamic soft regular weighting."""
-
-    return ConstrainedOrderStackSupportPlan(
-        prepare_constrained_order_stack_plan(
-            model,
-            constraints,
-            length=length,
-            policy=policy,
-            alphabet=alphabet,
-            prefer_dense_forbidden=prefer_dense_forbidden,
-            allowed_forbidden_symbols=allowed_forbidden_symbols,
-            minimize_source_graphs=minimize_source_graphs,
-        )
-    )
-
-
-def prepare_constrained_order_stack(
-    model: OrderStackModel,
-    constraints: ConstraintSet | None = None,
-    *,
-    length: int,
-    prefix: Sequence[Symbol],
-    policy: OrderPolicy | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    allowed_forbidden_symbols: AllowedForbiddenSymbols | None = None,
-    minimize_source_graphs: bool = False,
-) -> ConstrainedOrderStackBackend:
-    """Compile constraints and return a reusable order-stack backend."""
-
-    try:
-        plan = prepare_constrained_order_stack_plan(
-            model,
-            constraints,
-            length=length,
-            policy=policy,
-            alphabet=alphabet,
-            prefer_dense_forbidden=prefer_dense_forbidden,
-            allowed_forbidden_symbols=allowed_forbidden_symbols,
-            minimize_source_graphs=minimize_source_graphs,
-        )
-    except ValueError as error:
-        if "prefix-independent graph compilation" not in str(error):
-            raise
-        return ConstrainedOrderStackBackend(
-            run_constrained_order_stack(
-                model,
-                constraints,
-                length=length,
-                prefix=prefix,
-                policy=policy,
-                alphabet=alphabet,
-                prefer_dense_forbidden=prefer_dense_forbidden,
-                allowed_forbidden_symbols=allowed_forbidden_symbols,
-                minimize_source_graphs=minimize_source_graphs,
-            )
-        )
-    return plan.for_prefix(prefix)
-
-
-def prepare_constrained_order_stack_plan_from_sequences(
-    sequences: Iterable[Sequence[Symbol]],
-    constraints: ConstraintSet | None = None,
-    *,
-    max_order: int,
-    length: int,
-    policy: OrderPolicy | None = None,
-    start_symbol: Symbol | None = None,
-    end_symbol: Symbol | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> ConstrainedOrderStackPlan:
-    """Build an order-stack model from sequences and prepare a prefixless plan."""
-
-    model = OrderStackModel.from_sequences(
-        sequences,
-        max_order=max_order,
-        start_symbol=start_symbol,
-        end_symbol=end_symbol,
-    )
-    return prepare_constrained_order_stack_plan(
-        model,
-        constraints,
-        length=length,
-        policy=policy,
-        alphabet=alphabet,
-        prefer_dense_forbidden=prefer_dense_forbidden,
-        minimize_source_graphs=minimize_source_graphs,
-    )
-
-
-def prepare_constrained_order_stack_support_plan_from_sequences(
-    sequences: Iterable[Sequence[Symbol]],
-    constraints: ConstraintSet | None = None,
-    *,
-    max_order: int,
-    length: int,
-    policy: OrderPolicy | None = None,
-    start_symbol: Symbol | None = None,
-    end_symbol: Symbol | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> ConstrainedOrderStackSupportPlan:
-    """Build an order-stack model once and prepare reusable hard support."""
-
-    return ConstrainedOrderStackSupportPlan(
-        prepare_constrained_order_stack_plan_from_sequences(
-            sequences,
-            constraints,
-            max_order=max_order,
-            length=length,
-            policy=policy,
-            start_symbol=start_symbol,
-            end_symbol=end_symbol,
-            alphabet=alphabet,
-            prefer_dense_forbidden=prefer_dense_forbidden,
-            minimize_source_graphs=minimize_source_graphs,
-        )
-    )
-
-
-def prepare_until_order_stack(
-    model: OrderStackModel,
-    *,
-    prefix: Sequence[Symbol],
-    stop: Symbol | Iterable[Symbol] | Callable[[Symbol], bool],
-    min_length: int = 1,
-    max_length: int = 64,
-    constraints: ConstraintSet | None = None,
-    policy: OrderPolicy | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> UntilOrderStackBackend:
-    """Prepare variable-length first-hit continuation for the order stack.
-
-    The generated suffix has length in ``[min_length, max_length]``. The final
-    emitted symbol satisfies ``stop`` and no earlier emitted symbol does.
-    Lengths are sampled proportionally to the sum of each feasible backend's
-    positive start-order masses, falling back to unit weight if only a success
-    indicator is available.
-    """
-
-    if min_length < 1:
-        raise ValueError("min_length must be at least 1")
-    if max_length < min_length:
-        raise ValueError("max_length must be greater than or equal to min_length")
-    if not prefix:
-        raise ValueError("order-stack BP requires a non-empty prefix")
-
-    stop_predicate, stop_symbols = _coerce_stop_condition(
-        stop,
-        known_symbols=model.alphabet,
-    )
-    prepared: list[ConstrainedOrderStackBackend] = []
-    weights: list[float] = []
-
-    for length in range(min_length, max_length + 1):
-        if not _constraints_compatible_with_length(constraints, length):
-            continue
-        first_hit = _first_hit_constraints(stop_predicate, length)
-        combined = combine_constraints(constraints, first_hit)
-        allowed_forbidden = _allowed_forbidden_stop_symbols(
-            model,
-            stop_predicate,
-            stop_symbols,
-            length,
-        )
-        backend = prepare_constrained_order_stack(
-            model,
-            combined,
-            length=length,
-            prefix=prefix,
-            policy=policy,
-            alphabet=alphabet,
-            prefer_dense_forbidden=prefer_dense_forbidden,
-            allowed_forbidden_symbols=allowed_forbidden,
-            minimize_source_graphs=minimize_source_graphs,
-        )
-        weight = _length_weight(backend)
-        if weight <= 0.0:
-            continue
-        prepared.append(backend)
-        weights.append(weight)
-
-    if not prepared:
-        raise ValueError("No feasible first-hit continuation length satisfies the constraints.")
-
-    return UntilOrderStackBackend(
-        backends=tuple(prepared),
-        length_weights=tuple(weights),
-        min_length=min_length,
-        max_length=max_length,
-    )
-
-
-def prepare_until_end_order_stack(
-    model: OrderStackModel,
-    *,
-    prefix: Sequence[Symbol],
-    end_symbol: Symbol,
-    min_length: int = 1,
-    max_length: int = 64,
-    constraints: ConstraintSet | None = None,
-    policy: OrderPolicy | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> UntilOrderStackBackend:
-    """Prepare first-hit continuation that stops at ``end_symbol``."""
-
-    return prepare_until_order_stack(
-        model,
-        prefix=prefix,
-        stop=end_symbol,
-        min_length=min_length,
-        max_length=max_length,
-        constraints=constraints,
-        policy=policy,
-        alphabet=alphabet,
-        prefer_dense_forbidden=prefer_dense_forbidden,
-        minimize_source_graphs=minimize_source_graphs,
-    )
-
-
-def prepare_constrained_order_stack_from_sequences(
-    sequences: Iterable[Sequence[Symbol]],
-    constraints: ConstraintSet | None = None,
-    *,
-    max_order: int,
-    length: int,
-    prefix: Sequence[Symbol],
-    policy: OrderPolicy | None = None,
-    start_symbol: Symbol | None = None,
-    end_symbol: Symbol | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> ConstrainedOrderStackBackend:
-    """Build an order-stack model from sequences and prepare a backend."""
-
-    model = OrderStackModel.from_sequences(
-        sequences,
-        max_order=max_order,
-        start_symbol=start_symbol,
-        end_symbol=end_symbol,
-    )
-    return prepare_constrained_order_stack(
-        model,
-        constraints,
-        length=length,
-        prefix=prefix,
-        policy=policy,
-        alphabet=alphabet,
-        prefer_dense_forbidden=prefer_dense_forbidden,
-        minimize_source_graphs=minimize_source_graphs,
-    )
-
-
-def _coerce_stop_condition(
-    stop: Symbol | Iterable[Symbol] | Callable[[Symbol], bool],
-    *,
-    known_symbols: Iterable[Symbol] = (),
-) -> tuple[Callable[[Symbol], bool], frozenset[Symbol] | None]:
-    if callable(stop):
-        return lambda symbol: bool(stop(symbol)), None
-    known = frozenset(known_symbols)
-    if isinstance(stop, (str, bytes)) or _is_known_symbol(stop, known):
-        symbols = frozenset({stop})
-    else:
-        try:
-            symbols = frozenset(stop)  # type: ignore[arg-type]
-        except TypeError:
-            symbols = frozenset({stop})  # type: ignore[list-item]
-    if not symbols:
-        raise ValueError("stop symbol set must not be empty")
-    return lambda symbol: symbol in symbols, symbols
-
-
-def _constraints_compatible_with_length(
-    constraints: ConstraintSet | None,
-    length: int,
-) -> bool:
-    if constraints is None:
-        return True
-    for position in constraints.positional:
-        if position < 0:
-            raise IndexError(f"constraint position {position} is outside length {length}")
-        if position >= length:
-            return False
-    if constraints.meter is not None and len(constraints.meter.pattern) != length:
-        return False
-    cumulative = constraints.cumulative_meter
-    if cumulative is not None and cumulative.length is not None and cumulative.length != length:
-        return False
-    return True
-
-
-def _is_known_symbol(value: object, known_symbols: frozenset[Symbol]) -> bool:
-    if not isinstance(value, Hashable):
-        return False
-    try:
-        return value in known_symbols
-    except TypeError:
-        return False
-
-
-def _first_hit_constraints(
-    stop_predicate: Callable[[Symbol], bool],
-    length: int,
-) -> ConstraintSet:
-    positional: dict[int, Callable[[Symbol], bool]] = {
-        position: _negate_predicate(stop_predicate)
-        for position in range(length - 1)
-    }
-    positional[length - 1] = stop_predicate
-    return ConstraintSet(positional=positional)
-
-
-def _negate_predicate(predicate: Callable[[Symbol], bool]) -> Callable[[Symbol], bool]:
-    return lambda symbol: not predicate(symbol)
-
-
-def _allowed_forbidden_stop_symbols(
-    model: OrderStackModel,
-    stop_predicate: Callable[[Symbol], bool],
-    stop_symbols: frozenset[Symbol] | None,
-    length: int,
-) -> dict[int, frozenset[Symbol]]:
-    if stop_symbols is None:
-        allowed = frozenset(
-            symbol
-            for symbol in model.forbidden_symbols
-            if _predicate_accepts(stop_predicate, symbol)
-        )
-    else:
-        allowed = frozenset(symbol for symbol in stop_symbols if symbol in model.forbidden_symbols)
-    return {length - 1: allowed} if allowed else {}
-
-
-def _predicate_accepts(predicate: Callable[[Symbol], bool], symbol: Symbol) -> bool:
-    try:
-        return bool(predicate(symbol))
-    except Exception:
-        return False
-
-
-def _length_weight(backend: ConstrainedOrderStackBackend) -> float:
-    mass = sum(
-        max(0.0, float(start_mass))
-        for _order, start_mass in backend.result.start_order_masses()
-    )
-    if mass > 0.0:
-        return mass
-    return 1.0 if backend.result.success_mass > 0.0 else 0.0
-
-
-def _coerce_backend_rng(rng: random.Random | int | None) -> random.Random:
-    if isinstance(rng, random.Random):
-        return rng
-    return random.Random(rng)
-
-
-def _cumulative_weights(weights: Sequence[float]) -> tuple[float, ...]:
-    total = 0.0
-    cumulative: list[float] = []
-    for weight in weights:
-        total += float(weight)
-        cumulative.append(total)
-    return tuple(cumulative)

vendor/vo_regular_bp/vo_regular_bp/brute_force.py

@@ -1,116 +0,0 @@
-"""Tiny exhaustive checks for exactness tests."""
-
-from __future__ import annotations
-
-from collections import defaultdict
-from itertools import product
-from typing import Hashable, Iterable, Mapping
-
-from .acceptors import DFA, transition_weight as regular_transition_weight
-from .context import Context, ContextGraph, Symbol, _as_context
-
-
-def brute_force_distribution(
-    graph: ContextGraph,
-    acceptor: DFA,
-    *,
-    length: int,
-    alphabet: Iterable[Symbol] | None = None,
-    start_context: Iterable[Symbol] | Context | None = None,
-    start_acceptor_state: Hashable | None = None,
-) -> dict[tuple[Symbol, ...], float]:
-    """Enumerate all accepted length-``n`` strings and unnormalized masses."""
-
-    if length < 0:
-        raise ValueError("length must be non-negative")
-
-    context0 = graph.start_state if start_context is None else _as_context(start_context)
-    acceptor0 = acceptor.start_state if start_acceptor_state is None else start_acceptor_state
-    masses: dict[tuple[Symbol, ...], float] = defaultdict(float)
-
-    if alphabet is not None:
-        for sequence in product(tuple(alphabet), repeat=length):
-            context_state = context0
-            acceptor_state = acceptor0
-            probability = 1.0
-            accepted = True
-            for symbol in sequence:
-                edge = next(
-                    (edge for edge in graph.outgoing(context_state) if edge.symbol == symbol),
-                    None,
-                )
-                if edge is None:
-                    accepted = False
-                    break
-                next_acceptor_state = acceptor.next_state(acceptor_state, symbol)
-                if next_acceptor_state is None:
-                    accepted = False
-                    break
-                dfa_weight = regular_transition_weight(acceptor, acceptor_state, symbol)
-                if dfa_weight <= 0.0:
-                    accepted = False
-                    break
-                probability *= edge.probability * dfa_weight
-                context_state = edge.next_state
-                acceptor_state = next_acceptor_state
-            if accepted and acceptor.is_accepting(acceptor_state) and probability > 0.0:
-                masses[sequence] += probability
-        return dict(masses)
-
-    def visit(
-        time: int,
-        context_state: Context,
-        acceptor_state: Hashable,
-        prefix: tuple[Symbol, ...],
-        probability: float,
-    ) -> None:
-        if time == length:
-            if acceptor.is_accepting(acceptor_state):
-                masses[prefix] += probability
-            return
-
-        for edge in graph.outgoing(context_state):
-            next_acceptor_state = acceptor.next_state(acceptor_state, edge.symbol)
-            if next_acceptor_state is None:
-                continue
-            dfa_weight = regular_transition_weight(acceptor, acceptor_state, edge.symbol)
-            if dfa_weight <= 0.0:
-                continue
-            visit(
-                time + 1,
-                edge.next_state,
-                next_acceptor_state,
-                prefix + (edge.symbol,),
-                probability * edge.probability * dfa_weight,
-            )
-
-    visit(0, context0, acceptor0, (), 1.0)
-    return dict(masses)
-
-
-def brute_force_partition_function(
-    graph: ContextGraph,
-    acceptor: DFA,
-    *,
-    length: int,
-    alphabet: Iterable[Symbol] | None = None,
-    start_context: Iterable[Symbol] | Context | None = None,
-    start_acceptor_state: Hashable | None = None,
-) -> float:
-    return sum(
-        brute_force_distribution(
-            graph,
-            acceptor,
-            length=length,
-            alphabet=alphabet,
-            start_context=start_context,
-            start_acceptor_state=start_acceptor_state,
-        ).values()
-    )
-
-
-def conditional_distribution(masses: Mapping[tuple[Symbol, ...], float]) -> dict[tuple[Symbol, ...], float]:
-    total = float(sum(masses.values()))
-    if total <= 0.0:
-        return {}
-    return {sequence: mass / total for sequence, mass in masses.items()}

vendor/vo_regular_bp/vo_regular_bp/constraint_builders.py

@@ -1,238 +0,0 @@
-"""Convenience builders for common fixed-horizon constraints."""
-
-from __future__ import annotations
-
-from collections.abc import Callable, Iterable, Mapping, Sequence
-
-from .constraints import ConstraintSet, CumulativeMeterConstraint, MeterConstraint
-from .context import Symbol
-from .positional_bp import PositionConstraint, _allows
-
-
-def combine_constraints(*constraints: ConstraintSet | None) -> ConstraintSet:
-    """Merge constraint sets into one specification.
-
-    Positional constraints at the same position are intersected.  The current
-    public ``ConstraintSet`` supports one meter and one cumulative-meter
-    constraint; passing multiple distinct constraints of either kind raises.
-    """
-
-    positional: dict[int, PositionConstraint] = {}
-    forbidden_substrings: list[Sequence[Symbol]] = []
-    regular_acceptors = []
-    meter = None
-    cumulative_meter = None
-
-    for spec in constraints:
-        if spec is None:
-            continue
-        for position, constraint in spec.positional.items():
-            if position in positional:
-                positional[position] = _intersect_position_constraints(
-                    positional[position],
-                    constraint,
-                )
-            else:
-                positional[position] = constraint
-        forbidden_substrings.extend(spec.forbidden_substrings)
-        regular_acceptors.extend(spec.regular_acceptors)
-        if spec.meter is not None:
-            if meter is not None and spec.meter != meter:
-                raise ValueError("combine_constraints supports at most one meter constraint")
-            meter = spec.meter
-        if spec.cumulative_meter is not None:
-            if cumulative_meter is not None and spec.cumulative_meter != cumulative_meter:
-                raise ValueError(
-                    "combine_constraints supports at most one cumulative meter constraint"
-                )
-            cumulative_meter = spec.cumulative_meter
-
-    return ConstraintSet(
-        positional=positional,
-        forbidden_substrings=tuple(forbidden_substrings),
-        regular_acceptors=tuple(regular_acceptors),
-        meter=meter,
-        cumulative_meter=cumulative_meter,
-    )
-
-
-def at_position(position: int, allowed: PositionConstraint | Symbol) -> ConstraintSet:
-    """Constrain one zero-based generated position."""
-
-    return ConstraintSet(positional={int(position): _position_constraint(allowed)})
-
-
-def final_symbol(symbol: Symbol, *, length: int) -> ConstraintSet:
-    """Constrain the final generated symbol to one value."""
-
-    return final_symbols({symbol}, length=length)
-
-
-def final_symbols(allowed: PositionConstraint | Symbol, *, length: int) -> ConstraintSet:
-    """Constrain the final generated position."""
-
-    if length <= 0:
-        raise ValueError("length must be positive for a final-position constraint")
-    return at_position(length - 1, allowed)
-
-
-def final_pitch_class(
-    pitch_class: int,
-    *,
-    length: int,
-    modulo: int = 12,
-    symbol_to_pitch: Mapping[Symbol, int] | Callable[[Symbol], int] | None = None,
-) -> ConstraintSet:
-    """Constrain the final generated symbol by pitch class.
-
-    By default the symbol itself is interpreted as an integer pitch.  Richer
-    symbol types can pass a mapping or callable through ``symbol_to_pitch``.
-    """
-
-    if modulo <= 0:
-        raise ValueError("modulo must be positive")
-    target = int(pitch_class) % int(modulo)
-
-    def allows(symbol: Symbol) -> bool:
-        pitch = _lookup(symbol_to_pitch, symbol)
-        return int(pitch) % int(modulo) == target
-
-    return final_symbols(allows, length=length)
-
-
-def avoid_copied_ngrams(reference: Sequence[Symbol], ngram_length: int) -> ConstraintSet:
-    """Reject generated substrings copied from a reference sequence."""
-
-    if ngram_length <= 0:
-        raise ValueError("ngram_length must be positive")
-    forbidden = tuple(
-        dict.fromkeys(
-            tuple(reference[index : index + ngram_length])
-            for index in range(0, len(reference) - ngram_length + 1)
-        )
-    )
-    return ConstraintSet(forbidden_substrings=forbidden)
-
-
-def meter_pattern(
-    pattern: Sequence[object | Iterable[object] | None],
-    symbol_to_meter: Mapping[Symbol, object] | Callable[[Symbol], object],
-    *,
-    name: str = "meter",
-) -> ConstraintSet:
-    """Constrain per-position meter/classes."""
-
-    return ConstraintSet(
-        meter=MeterConstraint(
-            pattern=pattern,
-            symbol_to_meter=symbol_to_meter,
-            name=name,
-        )
-    )
-
-
-def cumulative_meter(
-    cost: Mapping[Symbol, int] | Callable[[Symbol], int],
-    predicate: Callable[[int, Symbol, int], bool] | None = None,
-    *,
-    length: int | None = None,
-    max_cost: int | None = None,
-    accept_costs: Iterable[int] | Callable[[int], bool] | None = None,
-    end_symbol: Symbol | None = None,
-    name: str = "cumulative_meter",
-) -> ConstraintSet:
-    """Constrain cumulative symbolic cost such as duration or beat position."""
-
-    return ConstraintSet(
-        cumulative_meter=CumulativeMeterConstraint(
-            cost=cost,
-            predicate=predicate,
-            length=length,
-            max_cost=max_cost,
-            accept_costs=accept_costs,
-            end_symbol=end_symbol,
-            name=name,
-        )
-    )
-
-
-def padded_duration_total(
-    total: int,
-    *,
-    length: int,
-    pad_symbol: Symbol,
-    symbol_to_duration: Mapping[Symbol, int] | Callable[[Symbol], int],
-    allow_zero_duration_events: bool = False,
-    name: str = "padded_duration_total",
-) -> ConstraintSet:
-    """Constrain generated duration with trailing PAD symbols.
-
-    The generated sequence has fixed length, but its musical prefix may be
-    shorter. The PAD symbol has duration zero, may appear only once ``total`` is
-    reached, and is absorbing once emitted.
-    """
-
-    if total < 0:
-        raise ValueError("total must be non-negative")
-    if length < 0:
-        raise ValueError("length must be non-negative")
-    target = int(total)
-
-    def duration_of(symbol: Symbol) -> int:
-        if symbol == pad_symbol:
-            return 0
-        return _lookup(symbol_to_duration, symbol)
-
-    def predicate(current_total: int, symbol: Symbol, one_based_position: int) -> bool:
-        del one_based_position
-        if symbol == pad_symbol:
-            return current_total == target
-        duration = duration_of(symbol)
-        if duration == 0 and not allow_zero_duration_events:
-            return False
-        if current_total == target:
-            return False
-        return True
-
-    return cumulative_meter(
-        duration_of,
-        predicate,
-        length=length,
-        max_cost=target,
-        accept_costs={target},
-        end_symbol=pad_symbol,
-        name=name,
-    )
-
-
-def _intersect_position_constraints(
-    left: PositionConstraint,
-    right: PositionConstraint,
-) -> PositionConstraint:
-    def allows(symbol: Symbol) -> bool:
-        return _allows(left, symbol) and _allows(right, symbol)
-
-    return allows
-
-
-def _position_constraint(allowed: PositionConstraint | Symbol) -> PositionConstraint:
-    if callable(allowed):
-        return allowed
-    if isinstance(allowed, (str, bytes)):
-        return {allowed}
-    try:
-        iter(allowed)  # type: ignore[arg-type]
-    except TypeError:
-        return {allowed}  # type: ignore[return-value]
-    return allowed  # type: ignore[return-value]
-
-
-def _lookup(
-    mapping_or_callable: Mapping[Symbol, int] | Callable[[Symbol], int] | None,
-    symbol: Symbol,
-) -> int:
-    if mapping_or_callable is None:
-        return int(symbol)
-    if isinstance(mapping_or_callable, Mapping):
-        return int(mapping_or_callable[symbol])
-    return int(mapping_or_callable(symbol))

vendor/vo_regular_bp/vo_regular_bp/constraints.py

@@ -1,173 +0,0 @@
-"""Public constraint specifications and compiler helpers.
-
-The classes in this module are intentionally project-agnostic: symbols can be
-pitches, durations, tokens, events, or any other hashable objects.  The compiler
-keeps purely positional constraints as time-indexed masks and compiles regular
-constraints into deterministic acceptors.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Callable, Iterable, Mapping, Sequence
-from dataclasses import dataclass, field
-from typing import Hashable
-
-from .acceptors import (
-    DFA,
-    all_of,
-    cumulative_meter_acceptor,
-    dense_forbidden_substring_acceptor,
-    forbidden_substring_acceptor,
-    meter_acceptor,
-)
-from .context import Symbol
-from .positional_bp import PositionConstraint, PositionConstraints
-
-
-MeterClass = Hashable
-
-
-@dataclass(frozen=True)
-class MeterConstraint:
-    """Per-position meter/classes for emitted symbols.
-
-    ``pattern`` entries may be a single class, an iterable of allowed classes,
-    or ``None`` as a wildcard. ``symbol_to_meter`` maps each emitted symbol to
-    its meter class.
-    """
-
-    pattern: Sequence[MeterClass | Iterable[MeterClass] | None]
-    symbol_to_meter: Mapping[Symbol, MeterClass] | Callable[[Symbol], MeterClass]
-    name: str = "meter"
-
-
-@dataclass(frozen=True)
-class CumulativeMeterConstraint:
-    """Cumulative-cost meter predicate.
-
-    The DFA state stores the cumulative cost before each emission. ``predicate``
-    is called as ``predicate(total_cost_before, symbol, one_based_position)``.
-    If ``length`` is omitted, the compiler uses the requested generation length.
-    """
-
-    cost: Mapping[Symbol, int] | Callable[[Symbol], int]
-    predicate: Callable[[int, Symbol, int], bool] | None = None
-    length: int | None = None
-    max_cost: int | None = None
-    accept_costs: Iterable[int] | Callable[[int], bool] | None = None
-    end_symbol: Symbol | None = None
-    name: str = "cumulative_meter"
-
-
-@dataclass(frozen=True)
-class ConstraintSet:
-    """Constraints for fixed-horizon symbolic generation.
-
-    Positional constraints are represented as masks and do not inflate regular
-    DFA state. Regular constraints are compiled into acceptors and intersected.
-    """
-
-    positional: PositionConstraints = field(default_factory=dict)
-    forbidden_substrings: Iterable[Sequence[Symbol]] = field(default_factory=tuple)
-    regular_acceptors: Sequence[DFA] = field(default_factory=tuple)
-    meter: MeterConstraint | None = None
-    cumulative_meter: CumulativeMeterConstraint | None = None
-
-
-@dataclass(frozen=True)
-class CompiledConstraints:
-    """Compiled constraints consumed by BP backends."""
-
-    positional: dict[int, PositionConstraint]
-    regular_acceptor: DFA | None = None
-
-    @property
-    def has_regular(self) -> bool:
-        return self.regular_acceptor is not None
-
-
-def compile_constraints(
-    constraints: ConstraintSet | None,
-    *,
-    length: int,
-    alphabet: Iterable[Symbol],
-    prefer_dense_forbidden: bool = True,
-) -> CompiledConstraints:
-    """Compile public constraint specs into masks plus an optional DFA."""
-
-    if length < 0:
-        raise ValueError("length must be non-negative")
-    spec = constraints or ConstraintSet()
-    alphabet_tuple = tuple(alphabet)
-
-    positional = dict(spec.positional)
-    _validate_positions(positional, length)
-
-    regulars: list[DFA] = list(spec.regular_acceptors)
-    forbidden = tuple(tuple(pattern) for pattern in spec.forbidden_substrings)
-    if forbidden:
-        if prefer_dense_forbidden:
-            regulars.append(
-                dense_forbidden_substring_acceptor(
-                    forbidden,
-                    alphabet=alphabet_tuple,
-                    name="dense_forbidden_substring",
-                )
-            )
-        else:
-            regulars.append(
-                forbidden_substring_acceptor(
-                    forbidden,
-                    alphabet=alphabet_tuple,
-                    name="forbidden_substring",
-                )
-            )
-
-    if spec.meter is not None:
-        if len(spec.meter.pattern) != length:
-            raise ValueError("meter pattern length must match generation length")
-        regulars.append(
-            meter_acceptor(
-                spec.meter.pattern,
-                spec.meter.symbol_to_meter,
-                alphabet=alphabet_tuple,
-                name=spec.meter.name,
-            )
-        )
-
-    if spec.cumulative_meter is not None:
-        cumulative = spec.cumulative_meter
-        if cumulative.length is not None and cumulative.length != length:
-            raise ValueError("cumulative meter length must match generation length")
-        regulars.append(
-            cumulative_meter_acceptor(
-                cumulative.length if cumulative.length is not None else length,
-                cumulative.cost,
-                cumulative.predicate,
-                alphabet=alphabet_tuple,
-                max_cost=cumulative.max_cost,
-                accept_costs=cumulative.accept_costs,
-                end_symbol=cumulative.end_symbol,
-                name=cumulative.name,
-            )
-        )
-
-    regular_acceptor = _combine_regulars(regulars)
-    return CompiledConstraints(
-        positional=positional,
-        regular_acceptor=regular_acceptor,
-    )
-
-
-def _combine_regulars(regulars: Sequence[DFA]) -> DFA | None:
-    if not regulars:
-        return None
-    if len(regulars) == 1:
-        return regulars[0]
-    return all_of(*regulars, name="constraint_set")
-
-
-def _validate_positions(constraints: Mapping[int, PositionConstraint], length: int) -> None:
-    for position in constraints:
-        if position < 0 or position >= length:
-            raise IndexError(f"constraint position {position} is outside length {length}")

vendor/vo_regular_bp/vo_regular_bp/context.py

@@ -1,397 +0,0 @@
-"""Sparse variable-order context graphs."""
-
-from __future__ import annotations
-
-from collections import Counter, defaultdict
-from dataclasses import dataclass
-import math
-from typing import Hashable, Iterable, Mapping, Sequence
-
-Symbol = Hashable
-Context = tuple[Symbol, ...]
-
-
-@dataclass(frozen=True)
-class Edge:
-    """A labeled probabilistic transition in a context graph."""
-
-    symbol: Symbol
-    probability: float
-    next_state: Context
-    order_weights: tuple[tuple[int, float], ...] = ()
-
-
-def _as_context(value: Iterable[Symbol] | Context | None) -> Context:
-    if value is None:
-        return ()
-    if isinstance(value, tuple):
-        return value
-    return tuple(value)
-
-
-class ContextGraph:
-    """Sparse context graph induced by a variable-order/backoff model.
-
-    Nodes are canonical contexts represented as tuples of symbols. Each outgoing
-    edge emits one symbol and moves to the canonical suffix context after that
-    emission.
-    """
-
-    def __init__(
-        self,
-        edges_by_state: Mapping[Iterable[Symbol] | Context, Sequence[Edge]],
-        *,
-        start_state: Iterable[Symbol] | Context = (),
-        max_order: int | None = None,
-        alphabet: Iterable[Symbol] | None = None,
-        validate: bool = True,
-    ) -> None:
-        normalized: dict[Context, tuple[Edge, ...]] = {}
-        states: set[Context] = {_as_context(start_state)}
-        emitted: set[Symbol] = set(alphabet or ())
-
-        for raw_state, raw_edges in edges_by_state.items():
-            state = _as_context(raw_state)
-            edges = tuple(
-                Edge(
-                    edge.symbol,
-                    float(edge.probability),
-                    _as_context(edge.next_state),
-                    tuple((int(order), float(weight)) for order, weight in edge.order_weights),
-                )
-                for edge in raw_edges
-            )
-            normalized[state] = edges
-            states.add(state)
-            for edge in edges:
-                states.add(edge.next_state)
-                emitted.add(edge.symbol)
-
-        self._edges = normalized
-        self.start_state = _as_context(start_state)
-        self.states = frozenset(states)
-        self.alphabet = frozenset(emitted)
-        self._alias_to_state: dict[Context, Context] = {}
-        self._continuation_counts: dict[Context, dict[Symbol, float]] = {}
-        self._state_supports: dict[Context, float] = {}
-        self.max_order = (
-            max_order
-            if max_order is not None
-            else max((len(state) for state in self.states), default=0)
-        )
-
-        if self.max_order < 0:
-            raise ValueError("max_order must be non-negative")
-        if validate:
-            self._validate()
-
-    @classmethod
-    def from_counts(
-        cls,
-        continuation_counts: Mapping[Iterable[Symbol] | Context, Mapping[Symbol, int | float]],
-        *,
-        max_order: int | None = None,
-        start_state: Iterable[Symbol] | Context = (),
-    ) -> "ContextGraph":
-        """Build a graph by normalizing explicit continuation counts."""
-
-        contexts = {_as_context(context) for context in continuation_counts}
-        contexts.add(_as_context(start_state))
-        if max_order is None:
-            max_order = max((len(context) for context in contexts), default=0)
-
-        edges_by_state: dict[Context, list[Edge]] = {}
-        count_metadata: dict[Context, dict[Symbol, float]] = {}
-        state_supports: dict[Context, float] = {}
-        for raw_context, counts in continuation_counts.items():
-            context = _as_context(raw_context)
-            total = float(sum(counts.values()))
-            if total <= 0.0:
-                raise ValueError(f"context {context!r} has no positive continuation mass")
-            state_supports[context] = total
-            count_metadata[context] = {
-                symbol: float(count)
-                for symbol, count in counts.items()
-                if count > 0
-            }
-            edges_by_state[context] = [
-                Edge(symbol, float(count) / total, cls._canon(context, symbol, contexts, max_order))
-                for symbol, count in counts.items()
-                if count > 0
-            ]
-        graph = cls(
-            edges_by_state,
-            start_state=start_state,
-            max_order=max_order,
-            validate=True,
-        )
-        graph._continuation_counts = count_metadata
-        graph._state_supports = state_supports
-        return graph
-
-    @classmethod
-    def from_probabilities(
-        cls,
-        probabilities: Mapping[Iterable[Symbol] | Context, Mapping[Symbol, float]],
-        *,
-        max_order: int | None = None,
-        start_state: Iterable[Symbol] | Context = (),
-    ) -> "ContextGraph":
-        """Build a graph from explicit conditional probabilities."""
-
-        contexts = {_as_context(context) for context in probabilities}
-        contexts.add(_as_context(start_state))
-        if max_order is None:
-            max_order = max((len(context) for context in contexts), default=0)
-
-        edges_by_state: dict[Context, list[Edge]] = {}
-        continuation_counts: dict[Context, dict[Symbol, float]] = {}
-        state_supports: dict[Context, float] = {}
-        for raw_context, probs in probabilities.items():
-            context = _as_context(raw_context)
-            continuation_counts[context] = {
-                symbol: float(prob)
-                for symbol, prob in probs.items()
-                if prob > 0.0
-            }
-            state_supports[context] = float(sum(continuation_counts[context].values()))
-            edges_by_state[context] = [
-                Edge(symbol, float(prob), cls._canon(context, symbol, contexts, max_order))
-                for symbol, prob in probs.items()
-                if prob > 0.0
-            ]
-        graph = cls(
-            edges_by_state,
-            start_state=start_state,
-            max_order=max_order,
-            validate=True,
-        )
-        graph._continuation_counts = continuation_counts
-        graph._state_supports = state_supports
-        return graph
-
-    @classmethod
-    def from_sequences(
-        cls,
-        sequences: Iterable[Sequence[Symbol]],
-        *,
-        max_order: int,
-        start_state: Iterable[Symbol] | Context = (),
-    ) -> "ContextGraph":
-        """Estimate normalized continuation counts for all suffix contexts.
-
-        For every token position, the implementation records the continuation
-        from each suffix context of orders 0..max_order available before that
-        token. This gives a sparse variable-order graph whose transitions are
-        normalized continuation counts at each observed context.
-        """
-
-        if max_order < 0:
-            raise ValueError("max_order must be non-negative")
-
-        return cls.from_weighted_sequences(
-            ((1.0, sequence) for sequence in sequences),
-            max_order=max_order,
-            start_state=start_state,
-        )
-
-    @classmethod
-    def from_weighted_sequences(
-        cls,
-        weighted_sequences: Iterable[tuple[int | float, Sequence[Symbol]]],
-        *,
-        max_order: int,
-        start_state: Iterable[Symbol] | Context = (),
-    ) -> "ContextGraph":
-        """Estimate continuation counts from a weighted sequence multiset."""
-
-        if max_order < 0:
-            raise ValueError("max_order must be non-negative")
-
-        counts: dict[Context, Counter[Symbol]] = defaultdict(Counter)
-
-        for weight, sequence in weighted_sequences:
-            if weight <= 0:
-                continue
-            tokens = tuple(sequence)
-            for index, symbol in enumerate(tokens):
-                order_limit = min(max_order, index)
-                for order in range(order_limit + 1):
-                    context = tokens[index - order : index] if order else ()
-                    counts[context][symbol] += weight
-
-        return cls.from_counts(counts, max_order=max_order, start_state=start_state)
-
-    @classmethod
-    def from_backoff_sequences(
-        cls,
-        sequences: Iterable[Sequence[Symbol]],
-        *,
-        max_order: int,
-        backoff_weight: float = 0.25,
-        start_state: Iterable[Symbol] | Context = (),
-    ) -> "ContextGraph":
-        """Build a sparse graph with explicit lower-order backoff support.
-
-        Each context receives a weighted mixture of normalized continuation
-        distributions from itself and its suffixes. This is useful for exact
-        experiments where a strict longest-context MLE model would make every
-        ``K+1``-gram transition copied from training by construction.
-        """
-
-        if max_order < 0:
-            raise ValueError("max_order must be non-negative")
-        if not 0.0 <= backoff_weight <= 1.0:
-            raise ValueError("backoff_weight must be in [0, 1]")
-
-        counts: dict[Context, Counter[Symbol]] = defaultdict(Counter)
-        for sequence in sequences:
-            tokens = tuple(sequence)
-            for index, symbol in enumerate(tokens):
-                order_limit = min(max_order, index)
-                for order in range(order_limit + 1):
-                    context = tokens[index - order : index] if order else ()
-                    counts[context][symbol] += 1
-
-        contexts = set(counts)
-        contexts.add(_as_context(start_state))
-        edges_by_state: dict[Context, list[Edge]] = {}
-        continuation_counts: dict[Context, dict[Symbol, float]] = {}
-        state_supports: dict[Context, float] = {}
-
-        for context in contexts:
-            scores: Counter[Symbol] = Counter()
-            order_scores: dict[Symbol, Counter[int]] = defaultdict(Counter)
-            context_order = len(context)
-            for order in range(context_order, -1, -1):
-                suffix = context[-order:] if order else ()
-                if suffix not in counts:
-                    continue
-                total = float(sum(counts[suffix].values()))
-                if total <= 0.0:
-                    continue
-                weight = backoff_weight ** (context_order - order)
-                for symbol, count in counts[suffix].items():
-                    contribution = weight * (float(count) / total)
-                    scores[symbol] += contribution
-                    order_scores[symbol][order] += contribution
-
-            total_score = float(sum(scores.values()))
-            if total_score <= 0.0:
-                continue
-            state_supports[context] = total_score
-            continuation_counts[context] = {
-                symbol: float(score)
-                for symbol, score in scores.items()
-                if score > 0.0
-            }
-            edges_by_state[context] = [
-                Edge(
-                    symbol,
-                    score / total_score,
-                    cls._canon(context, symbol, contexts, max_order),
-                    tuple(
-                        sorted(
-                            (
-                                (order, contribution / score)
-                                for order, contribution in order_scores[symbol].items()
-                            ),
-                            reverse=True,
-                        )
-                    ),
-                )
-                for symbol, score in scores.items()
-                if score > 0.0
-            ]
-
-        graph = cls(
-            edges_by_state,
-            start_state=start_state,
-            max_order=max_order,
-            validate=True,
-        )
-        graph._continuation_counts = continuation_counts
-        graph._state_supports = state_supports
-        return graph
-
-    @staticmethod
-    def _canon(
-        context: Context,
-        symbol: Symbol,
-        known_contexts: set[Context] | frozenset[Context],
-        max_order: int,
-    ) -> Context:
-        candidate = context + (symbol,)
-        limit = min(max_order, len(candidate))
-        for order in range(limit, -1, -1):
-            suffix = candidate[-order:] if order else ()
-            if suffix in known_contexts:
-                return suffix
-        return ()
-
-    def canonical_context(self, context: Iterable[Symbol] | Context, symbol: Symbol) -> Context:
-        """Return the longest known suffix after emitting ``symbol``."""
-
-        candidate = _as_context(context) + (symbol,)
-        limit = min(self.max_order, len(candidate))
-        for order in range(limit, -1, -1):
-            suffix = candidate[-order:] if order else ()
-            state = self._resolve_state(suffix)
-            if state in self.states:
-                return state
-        return ()
-
-    def outgoing(self, state: Iterable[Symbol] | Context) -> tuple[Edge, ...]:
-        """Outgoing edges for a context, or an empty tuple for dead contexts."""
-
-        return self._edges.get(self._resolve_state(state), ())
-
-    def _resolve_state(self, state: Iterable[Symbol] | Context) -> Context:
-        context = _as_context(state)
-        return self._alias_to_state.get(context, context)
-
-    def edge_count(self) -> int:
-        return sum(len(edges) for edges in self._edges.values())
-
-    def probability(self, sequence: Sequence[Symbol], *, start_state: Iterable[Symbol] | Context | None = None) -> float:
-        """Unconstrained probability of a sequence under the context graph."""
-
-        state = self.start_state if start_state is None else _as_context(start_state)
-        prob = 1.0
-        for symbol in sequence:
-            edge = next((edge for edge in self.outgoing(state) if edge.symbol == symbol), None)
-            if edge is None:
-                return 0.0
-            prob *= edge.probability
-            state = edge.next_state
-        return prob
-
-    def _validate(self) -> None:
-        if self.start_state not in self.states:
-            raise ValueError("start_state must be present in graph states")
-        if self.max_order < max((len(state) for state in self.states), default=0):
-            raise ValueError("max_order is smaller than at least one context state")
-
-        for state, edges in self._edges.items():
-            seen_symbols: set[Symbol] = set()
-            total = 0.0
-            for edge in edges:
-                if edge.symbol in seen_symbols:
-                    raise ValueError(f"state {state!r} has duplicate edge for symbol {edge.symbol!r}")
-                seen_symbols.add(edge.symbol)
-                if edge.next_state not in self.states:
-                    raise ValueError(f"edge from {state!r} points to unknown state {edge.next_state!r}")
-                if not math.isfinite(edge.probability) or edge.probability < 0.0:
-                    raise ValueError(f"invalid probability {edge.probability!r} on edge {edge!r}")
-                order_total = 0.0
-                for order, weight in edge.order_weights:
-                    if order < 0:
-                        raise ValueError(f"invalid negative order {order!r} on edge {edge!r}")
-                    if not math.isfinite(weight) or weight < 0.0:
-                        raise ValueError(f"invalid order weight {weight!r} on edge {edge!r}")
-                    order_total += weight
-                if edge.order_weights and not math.isclose(order_total, 1.0, rel_tol=1e-9, abs_tol=1e-9):
-                    raise ValueError(f"order weights on edge {edge!r} sum to {order_total}, not 1")
-                total += edge.probability
-            if edges and not math.isclose(total, 1.0, rel_tol=1e-9, abs_tol=1e-9):
-                raise ValueError(f"outgoing probabilities from {state!r} sum to {total}, not 1")

vendor/vo_regular_bp/vo_regular_bp/continuator.py

@@ -1,238 +0,0 @@
-"""Continuator-style facade over the reusable constrained BP backend.
-
-This module does not import Continuator.  It provides a small compatibility
-surface with Continuator vocabulary so an external Continuator project can swap
-its generation backend to ``vo_regular_bp`` with minimal glue code.
-"""
-
-from __future__ import annotations
-
-from collections.abc import Callable, Iterable, Mapping, Sequence
-from typing import TypeVar
-
-from .adapters import (
-    EventCodec,
-    EventOrderStackBackend,
-    EventOrderStackPlan,
-    infer_symbol_to_event as infer_symbol_decoder,
-    prepare_constrained_order_stack_plan_from_events,
-)
-from .constraint_builders import (
-    cumulative_meter,
-    final_pitch_class,
-    meter_pattern,
-    padded_duration_total,
-)
-from .constraints import ConstraintSet
-from .context import Symbol
-from .order_stack_bp import OrderPolicy, SingletonAvoidingBackoffPolicy
-
-
-EventT = TypeVar("EventT")
-
-
-def prepare_continuation_backend(
-    training_events: Iterable[Sequence[EventT]],
-    *,
-    prefix: Sequence[EventT],
-    horizon: int,
-    max_order: int,
-    constraints: ConstraintSet | None = None,
-    event_to_symbol: Callable[[EventT], Symbol] | None = None,
-    symbol_to_event: Mapping[Symbol, EventT] | Callable[[Symbol], EventT] | None = None,
-    infer_decoder: bool = True,
-    strict_decoder: bool = True,
-    policy: OrderPolicy | None = None,
-    start_event: EventT | None = None,
-    end_event: EventT | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> EventOrderStackBackend[EventT]:
-    """Prepare a constrained order-stack backend in Continuator vocabulary.
-
-    ``training_events`` and ``prefix`` are project-level event objects.
-    ``event_to_symbol`` maps those events to hashable symbols consumed by the
-    backend.  If no explicit ``symbol_to_event`` decoder is supplied, the
-    function can infer one from the training material when symbols identify
-    unique events.
-    """
-
-    if horizon < 0:
-        raise ValueError("horizon must be non-negative")
-    material = tuple(tuple(sequence) for sequence in training_events)
-    prefix_tuple = tuple(prefix)
-
-    codec = _build_codec(
-        material,
-        prefix_tuple,
-        event_to_symbol=event_to_symbol,
-        symbol_to_event=symbol_to_event,
-        infer_decoder=infer_decoder,
-        strict_decoder=strict_decoder,
-    )
-    active_policy = policy if policy is not None else SingletonAvoidingBackoffPolicy()
-
-    plan = prepare_constrained_order_stack_plan_from_events(
-        material,
-        constraints,
-        codec=codec,
-        max_order=max_order,
-        length=horizon,
-        policy=active_policy,
-        start_event=start_event,
-        end_event=end_event,
-        alphabet=alphabet,
-        prefer_dense_forbidden=prefer_dense_forbidden,
-        minimize_source_graphs=minimize_source_graphs,
-    )
-    return plan.for_prefix(prefix_tuple)
-
-
-def prepare_continuation_plan(
-    training_events: Iterable[Sequence[EventT]],
-    *,
-    horizon: int,
-    max_order: int,
-    constraints: ConstraintSet | None = None,
-    event_to_symbol: Callable[[EventT], Symbol] | None = None,
-    symbol_to_event: Mapping[Symbol, EventT] | Callable[[Symbol], EventT] | None = None,
-    infer_decoder: bool = True,
-    strict_decoder: bool = True,
-    policy: OrderPolicy | None = None,
-    start_event: EventT | None = None,
-    end_event: EventT | None = None,
-    alphabet: Iterable[Symbol] | None = None,
-    prefer_dense_forbidden: bool = True,
-    minimize_source_graphs: bool = False,
-) -> EventOrderStackPlan[EventT]:
-    """Prepare a reusable Continuator-shaped plan without binding a prefix."""
-
-    if horizon < 0:
-        raise ValueError("horizon must be non-negative")
-    material = tuple(tuple(sequence) for sequence in training_events)
-    codec = _build_codec(
-        material,
-        (),
-        event_to_symbol=event_to_symbol,
-        symbol_to_event=symbol_to_event,
-        infer_decoder=infer_decoder,
-        strict_decoder=strict_decoder,
-    )
-    active_policy = policy if policy is not None else SingletonAvoidingBackoffPolicy()
-
-    return prepare_constrained_order_stack_plan_from_events(
-        material,
-        constraints,
-        codec=codec,
-        max_order=max_order,
-        length=horizon,
-        policy=active_policy,
-        start_event=start_event,
-        end_event=end_event,
-        alphabet=alphabet,
-        prefer_dense_forbidden=prefer_dense_forbidden,
-        minimize_source_graphs=minimize_source_graphs,
-    )
-
-
-def final_pitch_class_constraint(
-    pitch_class: int,
-    *,
-    horizon: int,
-    modulo: int = 12,
-    symbol_to_pitch: Mapping[Symbol, int] | Callable[[Symbol], int] | None = None,
-) -> ConstraintSet:
-    """Constrain the final generated symbol to a pitch class."""
-
-    return final_pitch_class(
-        pitch_class,
-        length=horizon,
-        modulo=modulo,
-        symbol_to_pitch=symbol_to_pitch,
-    )
-
-
-def duration_total_constraint(
-    total: int,
-    *,
-    horizon: int | None = None,
-    symbol_to_duration: Mapping[Symbol, int] | Callable[[Symbol], int],
-    max_total: int | None = None,
-    name: str = "duration_total",
-) -> ConstraintSet:
-    """Constrain the total generated duration/cost."""
-
-    if total < 0:
-        raise ValueError("total must be non-negative")
-    return cumulative_meter(
-        symbol_to_duration,
-        length=horizon,
-        max_cost=total if max_total is None else max_total,
-        accept_costs={total},
-        name=name,
-    )
-
-
-def padded_duration_total_constraint(
-    total: int,
-    *,
-    horizon: int,
-    pad_symbol: Symbol,
-    symbol_to_duration: Mapping[Symbol, int] | Callable[[Symbol], int],
-    allow_zero_duration_events: bool = False,
-    name: str = "padded_duration_total",
-) -> ConstraintSet:
-    """Constrain generated duration with trailing PAD symbols."""
-
-    return padded_duration_total(
-        total,
-        length=horizon,
-        pad_symbol=pad_symbol,
-        symbol_to_duration=symbol_to_duration,
-        allow_zero_duration_events=allow_zero_duration_events,
-        name=name,
-    )
-
-
-def meter_cycle_constraint(
-    cycle: Sequence[object | Iterable[object] | None],
-    *,
-    horizon: int,
-    symbol_to_meter: Mapping[Symbol, object] | Callable[[Symbol], object],
-    offset: int = 0,
-    name: str = "meter_cycle",
-) -> ConstraintSet:
-    """Constrain generated positions by a repeating meter/class cycle."""
-
-    if horizon < 0:
-        raise ValueError("horizon must be non-negative")
-    if not cycle:
-        raise ValueError("cycle must not be empty")
-    pattern = tuple(cycle[(offset + position) % len(cycle)] for position in range(horizon))
-    return meter_pattern(pattern, symbol_to_meter, name=name)
-
-
-def _build_codec(
-    training_events: Sequence[Sequence[EventT]],
-    prefix: Sequence[EventT],
-    *,
-    event_to_symbol: Callable[[EventT], Symbol] | None,
-    symbol_to_event: Mapping[Symbol, EventT] | Callable[[Symbol], EventT] | None,
-    infer_decoder: bool,
-    strict_decoder: bool,
-) -> EventCodec[EventT]:
-    if event_to_symbol is None:
-        codec = EventCodec.identity()
-        if symbol_to_event is not None:
-            return codec.with_decoder(symbol_to_event)  # type: ignore[arg-type, return-value]
-        return codec  # type: ignore[return-value]
-
-    decoder = symbol_to_event
-    if decoder is None and infer_decoder:
-        decoder = infer_symbol_decoder(
-            tuple(training_events) + (tuple(prefix),),
-            event_to_symbol,
-            strict=strict_decoder,
-        )
-    return EventCodec(event_to_symbol=event_to_symbol, symbol_to_event=decoder)

vendor/vo_regular_bp/vo_regular_bp/experimental.py

@@ -1,986 +0,0 @@
-"""Experimental source-model transformations.
-
-The functions in this module are explicit modeling operations. They may change
-the source distribution, unlike exact minimization in :mod:`vo_regular_bp.minimization`.
-"""
-
-from __future__ import annotations
-
-from collections import Counter, defaultdict
-from collections.abc import Callable, Hashable, Mapping
-from dataclasses import dataclass
-import math
-import time
-from typing import Any
-
-from .context import Context, ContextGraph, Edge, Symbol
-
-
-@dataclass(frozen=True)
-class AlergiaMergeMetadata:
-    """Diagnostics for an experimental ALERGIA-style source merge."""
-
-    method: str
-    alpha: float
-    min_support: float
-    recursive: bool
-    original_state_count: int
-    merged_state_count: int
-    original_edge_count: int
-    merged_edge_count: int
-    state_to_class: dict[Context, int]
-    classes: tuple[tuple[Context, ...], ...]
-    conflicting_symbol_destinations: int = 0
-    symbol_projection: str | None = None
-    transition_projection: str | None = None
-    projection_kind: str = "identity"
-
-    @property
-    def state_compression_ratio(self) -> float:
-        return _ratio(self.original_state_count, self.merged_state_count)
-
-    @property
-    def edge_compression_ratio(self) -> float:
-        return _ratio(self.original_edge_count, self.merged_edge_count)
-
-    def as_dict(self) -> dict[str, object]:
-        return {
-            "method": self.method,
-            "alpha": self.alpha,
-            "min_support": self.min_support,
-            "recursive": self.recursive,
-            "original_state_count": self.original_state_count,
-            "merged_state_count": self.merged_state_count,
-            "original_edge_count": self.original_edge_count,
-            "merged_edge_count": self.merged_edge_count,
-            "state_compression_ratio": self.state_compression_ratio,
-            "edge_compression_ratio": self.edge_compression_ratio,
-            "conflicting_symbol_destinations": self.conflicting_symbol_destinations,
-            "symbol_projection": self.symbol_projection,
-            "transition_projection": self.transition_projection,
-            "projection_kind": self.projection_kind,
-        }
-
-
-@dataclass(frozen=True)
-class AlergiaFixedOrderMergeMetadata:
-    """Diagnostics for an experimental ALERGIA merge of one fixed-order graph."""
-
-    method: str
-    order: int
-    alpha: float
-    min_support: float
-    recursive: bool
-    original_state_count: int
-    merged_state_count: int
-    original_edge_count: int
-    merged_edge_count: int
-    state_to_class: dict[Context, int]
-    classes: tuple[tuple[Context, ...], ...]
-    conflicting_symbol_destinations: int = 0
-    conflicting_edge_orders: int = 0
-    symbol_projection: str | None = None
-    transition_projection: str | None = None
-    projection_kind: str = "identity"
-    merge_time_seconds: float = 0.0
-
-    @property
-    def state_compression_ratio(self) -> float:
-        return _ratio(self.original_state_count, self.merged_state_count)
-
-    @property
-    def edge_compression_ratio(self) -> float:
-        return _ratio(self.original_edge_count, self.merged_edge_count)
-
-    def as_dict(self) -> dict[str, object]:
-        return {
-            "method": self.method,
-            "order": self.order,
-            "alpha": self.alpha,
-            "min_support": self.min_support,
-            "recursive": self.recursive,
-            "original_state_count": self.original_state_count,
-            "merged_state_count": self.merged_state_count,
-            "original_edge_count": self.original_edge_count,
-            "merged_edge_count": self.merged_edge_count,
-            "state_compression_ratio": self.state_compression_ratio,
-            "edge_compression_ratio": self.edge_compression_ratio,
-            "conflicting_symbol_destinations": self.conflicting_symbol_destinations,
-            "conflicting_edge_orders": self.conflicting_edge_orders,
-            "symbol_projection": self.symbol_projection,
-            "transition_projection": self.transition_projection,
-            "projection_kind": self.projection_kind,
-            "merge_time_seconds": self.merge_time_seconds,
-        }
-
-
-@dataclass(frozen=True)
-class AlergiaOrderStackMergeMetadata:
-    """Diagnostics for an experimental ALERGIA-merged order-stack model."""
-
-    method: str
-    alpha: float
-    min_support: float
-    recursive: bool
-    orders: dict[int, AlergiaFixedOrderMergeMetadata]
-
-    @property
-    def original_state_count(self) -> int:
-        return sum(metadata.original_state_count for metadata in self.orders.values())
-
-    @property
-    def merged_state_count(self) -> int:
-        return sum(metadata.merged_state_count for metadata in self.orders.values())
-
-    @property
-    def original_edge_count(self) -> int:
-        return sum(metadata.original_edge_count for metadata in self.orders.values())
-
-    @property
-    def merged_edge_count(self) -> int:
-        return sum(metadata.merged_edge_count for metadata in self.orders.values())
-
-    @property
-    def merge_time_seconds(self) -> float:
-        return sum(metadata.merge_time_seconds for metadata in self.orders.values())
-
-    @property
-    def state_compression_ratio(self) -> float:
-        return _ratio(self.original_state_count, self.merged_state_count)
-
-    @property
-    def edge_compression_ratio(self) -> float:
-        return _ratio(self.original_edge_count, self.merged_edge_count)
-
-    def as_dict(self) -> dict[str, object]:
-        return {
-            "method": self.method,
-            "alpha": self.alpha,
-            "min_support": self.min_support,
-            "recursive": self.recursive,
-            "original_state_count": self.original_state_count,
-            "merged_state_count": self.merged_state_count,
-            "original_edge_count": self.original_edge_count,
-            "merged_edge_count": self.merged_edge_count,
-            "state_compression_ratio": self.state_compression_ratio,
-            "edge_compression_ratio": self.edge_compression_ratio,
-            "merge_time_seconds": self.merge_time_seconds,
-            "orders": {
-                order: metadata.as_dict()
-                for order, metadata in sorted(self.orders.items())
-            },
-        }
-
-
-@dataclass(frozen=True)
-class _ProjectionCache:
-    counts: dict[Context, dict[Hashable, float]]
-    dominant_next: dict[Context, dict[Hashable, Context]]
-
-
-_PairMemo = dict[tuple[Context, Context], tuple[bool, tuple[tuple[Context, Context], ...]]]
-
-
-def alergia_merge(
-    graph: ContextGraph,
-    *,
-    alpha: float = 0.01,
-    min_support: int | float = 10,
-    recursive: bool = True,
-    symbol_projection: Callable[[Symbol], Hashable] | None = None,
-    transition_projection: Callable[[Context, Symbol, Edge], Hashable] | None = None,
-) -> ContextGraph:
-    """Return an ALERGIA-like approximate source-state merge.
-
-    This function acts only on the unconstrained source graph. ``symbol_projection``
-    lets a caller compare emitted symbols through a feature map. For
-    context-aware semantics, ``transition_projection`` can instead map
-    ``(state, symbol, edge)`` to a comparison feature, for example an interval
-    from the current context to the emitted symbol. If both projections are
-    supplied, ``transition_projection`` takes precedence. The returned graph
-    still emits concrete symbols and remains a regular :class:`ContextGraph`,
-    so existing BP remains exact with respect to the new merged source model.
-    """
-
-    if alpha <= 0.0 or alpha >= 1.0:
-        raise ValueError("alpha must be in (0, 1)")
-    if min_support < 0:
-        raise ValueError("min_support must be non-negative")
-
-    counts = _state_counts(graph)
-    project = _transition_projector(
-        symbol_projection=symbol_projection,
-        transition_projection=transition_projection,
-    )
-    projection_cache = _precompute_projection_cache(graph, counts, project)
-    supports = {
-        state: float(sum(symbol_counts.values()))
-        for state, symbol_counts in counts.items()
-    }
-    states = tuple(sorted(graph.states, key=_context_sort_key))
-    classes = _ClassRegistry(states)
-    pair_memo: _PairMemo = {}
-
-    for state in states:
-        if supports.get(state, 0.0) < min_support:
-            continue
-        for root in classes.roots():
-            if classes.find(state) == root:
-                break
-            members = classes.members(root)
-            compatible, pairs = _compatible_with_class(
-                graph,
-                counts,
-                supports,
-                state,
-                members,
-                alpha=float(alpha),
-                min_support=float(min_support),
-                recursive=recursive,
-                projection_cache=projection_cache,
-                pair_memo=pair_memo,
-            )
-            if compatible:
-                classes.union(state, members[0])
-                for left, right in pairs:
-                    classes.union(left, right)
-                break
-
-    return _build_merged_graph(
-        graph,
-        counts,
-        classes.union_find,
-        alpha=float(alpha),
-        min_support=float(min_support),
-        recursive=recursive,
-        symbol_projection_name=_projection_name(symbol_projection),
-        transition_projection_name=_projection_name(transition_projection),
-        projection_kind=_projection_kind(symbol_projection, transition_projection),
-    )
-
-
-def alergia_metadata(graph: object) -> (
-    AlergiaMergeMetadata
-    | AlergiaFixedOrderMergeMetadata
-    | AlergiaOrderStackMergeMetadata
-    | None
-):
-    """Return ALERGIA metadata if ``graph`` was produced by an experimental merge."""
-
-    return getattr(graph, "alergia_metadata", None)
-
-
-def alergia_merge_fixed_order_graph(
-    graph: object,
-    *,
-    alpha: float = 0.01,
-    min_support: int | float = 10,
-    recursive: bool = True,
-    symbol_projection: Callable[[Symbol], Hashable] | None = None,
-    transition_projection: Callable[[Context, Symbol, Edge], Hashable] | None = None,
-    continuation_counts: Mapping[Context, Mapping[Symbol, int | float]] | None = None,
-) -> object:
-    """Return an experimental ALERGIA-merged fixed-order order-stack graph.
-
-    The merge is applied to one unconstrained fixed-order source graph. The
-    result is still a ``FixedOrderContextGraph`` and can be used by the existing
-    order-stack BP backends without a special sampling path.
-    """
-
-    started = time.perf_counter()
-    context_graph = _fixed_order_to_context_graph(
-        graph,
-        continuation_counts=continuation_counts,
-    )
-    merged_context_graph = alergia_merge(
-        context_graph,
-        alpha=alpha,
-        min_support=min_support,
-        recursive=recursive,
-        symbol_projection=symbol_projection,
-        transition_projection=transition_projection,
-    )
-    merged_graph, conflicting_edge_orders = _context_graph_to_fixed_order_graph(
-        merged_context_graph,
-        graph,
-    )
-    source_metadata = alergia_metadata(merged_context_graph)
-    if not isinstance(source_metadata, AlergiaMergeMetadata):
-        raise RuntimeError("missing ContextGraph ALERGIA metadata")
-    merged_graph.alergia_metadata = AlergiaFixedOrderMergeMetadata(
-        method="alergia_fixed_order",
-        order=int(graph.order),
-        alpha=float(alpha),
-        min_support=float(min_support),
-        recursive=recursive,
-        original_state_count=source_metadata.original_state_count,
-        merged_state_count=source_metadata.merged_state_count,
-        original_edge_count=source_metadata.original_edge_count,
-        merged_edge_count=source_metadata.merged_edge_count,
-        state_to_class=source_metadata.state_to_class,
-        classes=source_metadata.classes,
-        conflicting_symbol_destinations=source_metadata.conflicting_symbol_destinations,
-        conflicting_edge_orders=conflicting_edge_orders,
-        symbol_projection=source_metadata.symbol_projection,
-        transition_projection=source_metadata.transition_projection,
-        projection_kind=source_metadata.projection_kind,
-        merge_time_seconds=time.perf_counter() - started,
-    )
-    return merged_graph
-
-
-def alergia_merge_order_stack_model(
-    model: object,
-    *,
-    alpha: float = 0.01,
-    min_support: int | float = 10,
-    recursive: bool = True,
-    symbol_projection: Callable[[Symbol], Hashable] | None = None,
-    transition_projection: Callable[[Context, Symbol, Edge], Hashable] | None = None,
-) -> "AlergiaOrderStackModel":
-    """Return an experimental order-stack model with ALERGIA-merged sources.
-
-    Each fixed-order graph is merged independently. The returned model exposes
-    the same ``compile_graph(order)`` interface used by the existing order-stack
-    BP preparation functions, so sampling APIs do not need a special path.
-    """
-
-    graphs = {}
-    metadata_by_order = {}
-    for order in range(1, int(model.max_order) + 1):
-        graph = model.compile_graph(order)
-        merged_graph = alergia_merge_fixed_order_graph(
-            graph,
-            alpha=alpha,
-            min_support=min_support,
-            recursive=recursive,
-            symbol_projection=symbol_projection,
-            transition_projection=transition_projection,
-            continuation_counts=_fixed_order_counts_from_model(model, graph),
-        )
-        graphs[order] = merged_graph
-        metadata = alergia_metadata(merged_graph)
-        if not isinstance(metadata, AlergiaFixedOrderMergeMetadata):
-            raise RuntimeError("missing fixed-order ALERGIA metadata")
-        metadata_by_order[order] = metadata
-
-    return AlergiaOrderStackModel(
-        base_model=model,
-        graphs=graphs,
-        metadata=AlergiaOrderStackMergeMetadata(
-            method="alergia_order_stack",
-            alpha=float(alpha),
-            min_support=float(min_support),
-            recursive=recursive,
-            orders=metadata_by_order,
-        ),
-    )
-
-
-class AlergiaOrderStackModel:
-    """Experimental order-stack model backed by ALERGIA-merged fixed-order graphs."""
-
-    def __init__(
-        self,
-        *,
-        base_model: object,
-        graphs: dict[int, object],
-        metadata: AlergiaOrderStackMergeMetadata,
-    ) -> None:
-        self.base_model = base_model
-        self.max_order = int(base_model.max_order)
-        self.forbidden_symbols = frozenset(getattr(base_model, "forbidden_symbols", ()))
-        self.alphabet = frozenset(getattr(base_model, "alphabet", ()))
-        self._graphs = dict(graphs)
-        self.alergia_metadata = metadata
-
-    def compile_graph(self, order: int) -> object:
-        if order < 1 or order > self.max_order:
-            raise ValueError(f"order must be between 1 and {self.max_order}")
-        return self._graphs[int(order)]
-
-    def compile_graphs_for_plan(self, *, length: int) -> dict[int, object]:
-        return dict(self._graphs)
-
-
-def _compatible_with_class(
-    graph: ContextGraph,
-    counts: dict[Context, dict[Symbol, float]],
-    supports: dict[Context, float],
-    state: Context,
-    members: list[Context],
-    *,
-    alpha: float,
-    min_support: float,
-    recursive: bool,
-    projection_cache: _ProjectionCache,
-    pair_memo: _PairMemo,
-) -> tuple[bool, list[tuple[Context, Context]]]:
-    pairs: list[tuple[Context, Context]] = []
-    for member in members:
-        compatible, member_pairs = _compatible_pair(
-            graph,
-            counts,
-            supports,
-            state,
-            member,
-            alpha=alpha,
-            min_support=min_support,
-            recursive=recursive,
-            projection_cache=projection_cache,
-            pair_memo=pair_memo,
-            seen=set(),
-        )
-        if not compatible:
-            return False, []
-        pairs.extend(member_pairs)
-    return True, pairs
-
-
-def _compatible_pair(
-    graph: ContextGraph,
-    counts: dict[Context, dict[Symbol, float]],
-    supports: dict[Context, float],
-    left: Context,
-    right: Context,
-    *,
-    alpha: float,
-    min_support: float,
-    recursive: bool,
-    projection_cache: _ProjectionCache,
-    pair_memo: _PairMemo,
-    seen: set[tuple[Context, Context]],
-) -> tuple[bool, list[tuple[Context, Context]]]:
-    if left == right:
-        return True, []
-    pair = (left, right) if _context_sort_key(left) <= _context_sort_key(right) else (right, left)
-    if pair in seen:
-        return True, []
-    cache_positive = not seen
-    cached = pair_memo.get(pair)
-    if cached is not None:
-        compatible, pairs = cached
-        if not compatible or cache_positive:
-            return compatible, list(pairs)
-    seen.add(pair)
-
-    left_counts = counts.get(left, {})
-    right_counts = counts.get(right, {})
-    left_support = supports.get(left, 0.0)
-    right_support = supports.get(right, 0.0)
-    if left_support == 0.0 and right_support == 0.0 and not left_counts and not right_counts:
-        return _cache_pair_result(pair_memo, pair, True, [pair], cache_positive)
-    if left_support <= 0.0 or right_support <= 0.0:
-        return _cache_pair_result(pair_memo, pair, False, [], cache_positive)
-    if left_support < min_support or right_support < min_support:
-        return _cache_pair_result(pair_memo, pair, False, [], cache_positive)
-
-    left_projected = projection_cache.counts.get(left, {})
-    right_projected = projection_cache.counts.get(right, {})
-    bound = _hoeffding_bound(left_support, right_support, alpha)
-    for projected_symbol in set(left_projected) | set(right_projected):
-        left_probability = left_projected.get(projected_symbol, 0.0) / left_support
-        right_probability = right_projected.get(projected_symbol, 0.0) / right_support
-        if abs(left_probability - right_probability) > bound:
-            return _cache_pair_result(pair_memo, pair, False, [], cache_positive)
-
-    pairs = [pair]
-    if recursive:
-        left_next = projection_cache.dominant_next.get(left, {})
-        right_next = projection_cache.dominant_next.get(right, {})
-        for projected_symbol in set(left_next) & set(right_next):
-            compatible, child_pairs = _compatible_pair(
-                graph,
-                counts,
-                supports,
-                left_next[projected_symbol],
-                right_next[projected_symbol],
-                alpha=alpha,
-                min_support=min_support,
-                recursive=True,
-                projection_cache=projection_cache,
-                pair_memo=pair_memo,
-                seen=seen,
-            )
-            if not compatible:
-                return _cache_pair_result(pair_memo, pair, False, [], cache_positive)
-            pairs.extend(child_pairs)
-    return _cache_pair_result(pair_memo, pair, True, pairs, cache_positive)
-
-
-def _build_merged_graph(
-    graph: ContextGraph,
-    counts: dict[Context, dict[Symbol, float]],
-    union: "_UnionFind",
-    *,
-    alpha: float,
-    min_support: float,
-    recursive: bool,
-    symbol_projection_name: str | None,
-    transition_projection_name: str | None,
-    projection_kind: str,
-) -> ContextGraph:
-    states = tuple(sorted(graph.states, key=_context_sort_key))
-    root_to_members: dict[Context, list[Context]] = defaultdict(list)
-    for state in states:
-        root_to_members[union.find(state)].append(state)
-
-    roots = tuple(
-        sorted(
-            root_to_members,
-            key=lambda root: _context_sort_key(root_to_members[root][0]),
-        )
-    )
-    root_to_class = {root: class_id for class_id, root in enumerate(roots)}
-    representative_by_root = {
-        root: tuple(sorted(members, key=_context_sort_key))[0]
-        for root, members in root_to_members.items()
-    }
-
-    edges_by_state: dict[Context, list[Edge]] = {}
-    merged_counts: dict[Context, dict[Symbol, float]] = {}
-    merged_supports: dict[Context, float] = {}
-    conflicting_symbol_destinations = 0
-
-    for root in roots:
-        representative = representative_by_root[root]
-        symbol_counts: Counter[Symbol] = Counter()
-        destination_counts: dict[Symbol, Counter[Context]] = defaultdict(Counter)
-        order_scores: dict[Symbol, Counter[int]] = defaultdict(Counter)
-
-        for member in root_to_members[root]:
-            member_counts = counts.get(member, {})
-            for edge in graph.outgoing(member):
-                count = member_counts.get(edge.symbol, 0.0)
-                if count <= 0.0:
-                    continue
-                symbol_counts[edge.symbol] += count
-                destination_counts[edge.symbol][union.find(edge.next_state)] += count
-                for order, weight in edge.order_weights:
-                    order_scores[edge.symbol][int(order)] += count * float(weight)
-
-        total = float(sum(symbol_counts.values()))
-        merged_supports[representative] = total
-        merged_counts[representative] = {
-            symbol: float(count)
-            for symbol, count in symbol_counts.items()
-            if count > 0.0
-        }
-        if total <= 0.0:
-            edges_by_state[representative] = []
-            continue
-
-        state_edges: list[Edge] = []
-        for symbol, count in sorted(symbol_counts.items(), key=lambda item: _hashable_key(item[0])):
-            destinations = destination_counts[symbol]
-            if len(destinations) > 1:
-                conflicting_symbol_destinations += 1
-            destination_root = max(
-                destinations,
-                key=lambda candidate: (destinations[candidate], repr(candidate)),
-            )
-            order_weights = ()
-            if order_scores[symbol]:
-                order_total = float(sum(order_scores[symbol].values()))
-                if order_total > 0.0:
-                    order_weights = tuple(
-                        sorted(
-                            (
-                                (order, score / order_total)
-                                for order, score in order_scores[symbol].items()
-                            ),
-                            reverse=True,
-                        )
-                    )
-            state_edges.append(
-                Edge(
-                    symbol=symbol,
-                    probability=float(count) / total,
-                    next_state=representative_by_root[destination_root],
-                    order_weights=order_weights,
-                )
-            )
-        edges_by_state[representative] = state_edges
-
-    start_state = representative_by_root[union.find(graph._resolve_state(graph.start_state))]
-    merged = ContextGraph(
-        edges_by_state,
-        start_state=start_state,
-        max_order=graph.max_order,
-        alphabet=graph.alphabet,
-        validate=True,
-    )
-
-    alias_to_state = {
-        state: representative_by_root[union.find(state)]
-        for state in states
-    }
-    for alias, resolved in getattr(graph, "_alias_to_state", {}).items():
-        alias_to_state[alias] = representative_by_root[union.find(resolved)]
-    merged._alias_to_state = alias_to_state
-    merged._continuation_counts = merged_counts
-    merged._state_supports = merged_supports
-
-    state_to_class = {
-        state: root_to_class[union.find(state)]
-        for state in states
-    }
-    classes = tuple(
-        tuple(sorted(root_to_members[root], key=_context_sort_key))
-        for root in roots
-    )
-    merged.alergia_metadata = AlergiaMergeMetadata(
-        method="alergia",
-        alpha=alpha,
-        min_support=min_support,
-        recursive=recursive,
-        original_state_count=len(graph.states),
-        merged_state_count=len(merged.states),
-        original_edge_count=graph.edge_count(),
-        merged_edge_count=merged.edge_count(),
-        state_to_class=state_to_class,
-        classes=classes,
-        conflicting_symbol_destinations=conflicting_symbol_destinations,
-        symbol_projection=symbol_projection_name,
-        transition_projection=transition_projection_name,
-        projection_kind=projection_kind,
-    )
-    return merged
-
-
-def _fixed_order_to_context_graph(
-    graph: object,
-    *,
-    continuation_counts: Mapping[Context, Mapping[Symbol, int | float]] | None,
-) -> ContextGraph:
-    contexts = tuple(graph.contexts)
-    edges_by_state: dict[Context, list[Edge]] = {}
-    count_metadata: dict[Context, dict[Symbol, float]] = {}
-    state_supports: dict[Context, float] = {}
-
-    for state_id, context in enumerate(contexts):
-        raw_counts = continuation_counts.get(context) if continuation_counts is not None else None
-        support = (
-            float(sum(count for count in raw_counts.values() if count > 0))
-            if raw_counts is not None
-            else 1.0
-        )
-        if support <= 0.0 and graph.outgoing[state_id]:
-            support = 1.0
-        state_counts: dict[Symbol, float] = {}
-        state_edges: list[Edge] = []
-        for edge in graph.outgoing[state_id]:
-            count = (
-                float(raw_counts.get(edge.symbol, edge.probability * support))
-                if raw_counts is not None
-                else edge.probability * support
-            )
-            if count > 0.0:
-                state_counts[edge.symbol] = count
-            state_edges.append(
-                Edge(
-                    symbol=edge.symbol,
-                    probability=edge.probability,
-                    next_state=contexts[edge.dst],
-                    order_weights=((int(edge.order), 1.0),),
-                )
-            )
-        edges_by_state[context] = state_edges
-        count_metadata[context] = state_counts
-        state_supports[context] = float(sum(state_counts.values()))
-
-    start_state = contexts[0] if contexts else ()
-    context_graph = ContextGraph(
-        edges_by_state,
-        start_state=start_state,
-        max_order=int(graph.order),
-        validate=True,
-    )
-    alias_to_state = {
-        alias: contexts[state_id]
-        for alias, state_id in graph.context_to_id.items()
-        if state_id < len(contexts)
-    }
-    for state_id, aliases in enumerate(getattr(graph, "state_aliases", ())):
-        if state_id >= len(contexts):
-            continue
-        for alias in aliases:
-            alias_to_state[alias] = contexts[state_id]
-    context_graph._alias_to_state = alias_to_state
-    context_graph._continuation_counts = count_metadata
-    context_graph._state_supports = state_supports
-    return context_graph
-
-
-def _context_graph_to_fixed_order_graph(
-    merged: ContextGraph,
-    source_graph: object,
-) -> tuple[object, int]:
-    from .order_stack_bp import FixedOrderContextGraph, StackEdge
-
-    metadata = alergia_metadata(merged)
-    if not isinstance(metadata, AlergiaMergeMetadata):
-        raise RuntimeError("missing ContextGraph ALERGIA metadata")
-
-    fixed = FixedOrderContextGraph(int(source_graph.order))
-    representatives = [class_members[0] for class_members in metadata.classes]
-    fixed.contexts = representatives
-    fixed.context_to_id = {
-        context: state_id
-        for state_id, context in enumerate(representatives)
-    }
-    fixed.outgoing = [[] for _context in representatives]
-
-    aliases: list[set[Context]] = [set(class_members) for class_members in metadata.classes]
-    for alias, resolved in getattr(merged, "_alias_to_state", {}).items():
-        state_id = fixed.context_to_id.get(resolved)
-        if state_id is None:
-            continue
-        fixed.context_to_id[alias] = state_id
-        aliases[state_id].add(alias)
-    fixed.state_aliases = tuple(
-        tuple(sorted(group, key=_context_sort_key))
-        for group in aliases
-    )
-
-    conflicting_edge_orders = 0
-    for src, context in enumerate(fixed.contexts):
-        for edge in merged.outgoing(context):
-            order, has_conflict = _dominant_order(edge.order_weights, int(source_graph.order))
-            if has_conflict:
-                conflicting_edge_orders += 1
-            fixed.outgoing[src].append(
-                StackEdge(
-                    src=src,
-                    dst=fixed.context_to_id[edge.next_state],
-                    symbol=edge.symbol,
-                    probability=edge.probability,
-                    order=order,
-                )
-            )
-    return fixed, conflicting_edge_orders
-
-
-def _fixed_order_counts_from_model(
-    model: object,
-    graph: object,
-) -> dict[Context, dict[Symbol, float]] | None:
-    model_counts = getattr(model, "counts", None)
-    longest_available_suffix = getattr(model, "longest_available_suffix", None)
-    if not isinstance(model_counts, Mapping) or longest_available_suffix is None:
-        return None
-
-    result: dict[Context, dict[Symbol, float]] = {}
-    for state_id, context in enumerate(graph.contexts):
-        suffix = longest_available_suffix(context, max_order=int(graph.order))
-        suffix_counts = model_counts.get(suffix, {}) if suffix is not None else {}
-        state_counts = {
-            edge.symbol: float(suffix_counts.get(edge.symbol, 0.0))
-            for edge in graph.outgoing[state_id]
-            if suffix_counts.get(edge.symbol, 0.0) > 0.0
-        }
-        if not state_counts and graph.outgoing[state_id]:
-            state_counts = {
-                edge.symbol: float(edge.probability)
-                for edge in graph.outgoing[state_id]
-                if edge.probability > 0.0
-            }
-        result[context] = state_counts
-    return result
-
-
-def _dominant_order(
-    order_weights: tuple[tuple[int, float], ...],
-    default_order: int,
-) -> tuple[int, bool]:
-    if not order_weights:
-        return default_order, False
-    order, _weight = max(order_weights, key=lambda item: (item[1], item[0]))
-    return int(order), len(order_weights) > 1
-
-
-def _state_counts(graph: ContextGraph) -> dict[Context, dict[Symbol, float]]:
-    metadata = getattr(graph, "_continuation_counts", {})
-    result: dict[Context, dict[Symbol, float]] = {}
-    supports = getattr(graph, "_state_supports", {})
-    for state in graph.states:
-        if state in metadata:
-            result[state] = dict(metadata[state])
-            continue
-        support = float(supports.get(state, 0.0))
-        edges = graph.outgoing(state)
-        if support <= 0.0 and edges:
-            support = 1.0
-        result[state] = {
-            edge.symbol: edge.probability * support
-            for edge in edges
-            if edge.probability > 0.0
-        }
-    return result
-
-
-def _precompute_projection_cache(
-    graph: ContextGraph,
-    counts: dict[Context, dict[Symbol, float]],
-    transition_projection: Callable[[Context, Symbol, Edge], Hashable],
-) -> _ProjectionCache:
-    projected_counts: dict[Context, dict[Hashable, float]] = {}
-    dominant_next: dict[Context, dict[Hashable, Context]] = {}
-    for state in graph.states:
-        state_projected: Counter[Hashable] = Counter()
-        destination_counts: dict[Hashable, Counter[Context]] = defaultdict(Counter)
-        state_counts = counts.get(state, {})
-        for edge in graph.outgoing(state):
-            count = state_counts.get(edge.symbol, 0.0)
-            if count <= 0.0:
-                continue
-            projected_symbol = transition_projection(state, edge.symbol, edge)
-            state_projected[projected_symbol] += count
-            destination_counts[projected_symbol][edge.next_state] += count
-        projected_counts[state] = dict(state_projected)
-        dominant_next[state] = {
-            projected_symbol: max(
-                destinations,
-                key=lambda candidate: (destinations[candidate], repr(candidate)),
-            )
-            for projected_symbol, destinations in destination_counts.items()
-        }
-    return _ProjectionCache(counts=projected_counts, dominant_next=dominant_next)
-
-
-def _cache_pair_result(
-    pair_memo: _PairMemo,
-    pair: tuple[Context, Context],
-    compatible: bool,
-    pairs: list[tuple[Context, Context]],
-    cache_positive: bool,
-) -> tuple[bool, list[tuple[Context, Context]]]:
-    frozen_pairs = tuple(pairs)
-    if not compatible or cache_positive:
-        pair_memo[pair] = (compatible, frozen_pairs)
-    return compatible, list(frozen_pairs)
-
-
-def _hoeffding_bound(left_support: float, right_support: float, alpha: float) -> float:
-    scale = math.log(2.0 / alpha) / 2.0
-    return math.sqrt(scale / left_support) + math.sqrt(scale / right_support)
-
-
-class _ClassRegistry:
-    def __init__(self, states: tuple[Context, ...]) -> None:
-        self.union_find = _UnionFind(states)
-        self._state_index = {state: index for index, state in enumerate(states)}
-        self._active_roots = list(states)
-        self._members_by_root = {state: [state] for state in states}
-
-    def find(self, state: Context) -> Context:
-        return self.union_find.find(state)
-
-    def roots(self) -> tuple[Context, ...]:
-        return tuple(self._active_roots)
-
-    def members(self, root: Context) -> list[Context]:
-        return self._members_by_root[self.find(root)]
-
-    def union(self, left: Context, right: Context) -> Context:
-        left_root = self.union_find.find(left)
-        right_root = self.union_find.find(right)
-        if left_root == right_root:
-            return left_root
-
-        root = self.union_find.union(left_root, right_root)
-        removed_root = right_root if root == left_root else left_root
-        self._members_by_root[root] = self._merge_members(
-            self._members_by_root[root],
-            self._members_by_root.pop(removed_root),
-        )
-        self._active_roots.remove(removed_root)
-        return root
-
-    def _merge_members(self, left: list[Context], right: list[Context]) -> list[Context]:
-        merged: list[Context] = []
-        left_index = 0
-        right_index = 0
-        while left_index < len(left) and right_index < len(right):
-            if self._state_index[left[left_index]] <= self._state_index[right[right_index]]:
-                merged.append(left[left_index])
-                left_index += 1
-            else:
-                merged.append(right[right_index])
-                right_index += 1
-        merged.extend(left[left_index:])
-        merged.extend(right[right_index:])
-        return merged
-
-
-def _context_sort_key(context: Context) -> tuple[int, str]:
-    return (len(context), repr(context))
-
-
-def _hashable_key(value: Any) -> Hashable:
-    try:
-        hash(value)
-    except TypeError:
-        return (type(value).__name__, repr(value))
-    return (type(value).__name__, value)
-
-
-def _identity_projection(_state: Context, symbol: Symbol, _edge: Edge) -> Hashable:
-    return symbol
-
-
-def _transition_projector(
-    *,
-    symbol_projection: Callable[[Symbol], Hashable] | None,
-    transition_projection: Callable[[Context, Symbol, Edge], Hashable] | None,
-) -> Callable[[Context, Symbol, Edge], Hashable]:
-    if transition_projection is not None:
-        return transition_projection
-    if symbol_projection is not None:
-        return lambda _state, symbol, _edge: symbol_projection(symbol)
-    return _identity_projection
-
-
-def _projection_name(projection: Callable[..., Hashable] | None) -> str | None:
-    if projection is None:
-        return None
-    return getattr(projection, "__name__", repr(projection))
-
-
-def _projection_kind(
-    symbol_projection: Callable[[Symbol], Hashable] | None,
-    transition_projection: Callable[[Context, Symbol, Edge], Hashable] | None,
-) -> str:
-    if transition_projection is not None:
-        return "transition"
-    if symbol_projection is not None:
-        return "symbol"
-    return "identity"
-
-
-def _ratio(numerator: int, denominator: int) -> float:
-    if denominator == 0:
-        return float("inf") if numerator else 1.0
-    return float(numerator) / float(denominator)
-
-
-class _UnionFind:
-    def __init__(self, states: tuple[Context, ...]) -> None:
-        self.parent = {state: state for state in states}
-
-    def find(self, state: Context) -> Context:
-        parent = self.parent[state]
-        if parent != state:
-            parent = self.find(parent)
-            self.parent[state] = parent
-        return parent
-
-    def union(self, left: Context, right: Context) -> Context:
-        left_root = self.find(left)
-        right_root = self.find(right)
-        if left_root == right_root:
-            return left_root
-        if _context_sort_key(right_root) < _context_sort_key(left_root):
-            left_root, right_root = right_root, left_root
-        self.parent[right_root] = left_root
-        return left_root

vendor/vo_regular_bp/vo_regular_bp/metrics.py

@@ -1,24 +0,0 @@
-"""Small metrics for exactness experiments."""
-
-from __future__ import annotations
-
-from collections import Counter
-from typing import Hashable, Iterable, Mapping
-
-Symbol = Hashable
-
-
-def total_variation(
-    p: Mapping[tuple[Symbol, ...], float],
-    q: Mapping[tuple[Symbol, ...], float],
-) -> float:
-    support = set(p) | set(q)
-    return 0.5 * sum(abs(float(p.get(item, 0.0)) - float(q.get(item, 0.0))) for item in support)
-
-
-def empirical_distribution(samples: Iterable[tuple[Symbol, ...]]) -> dict[tuple[Symbol, ...], float]:
-    counts = Counter(samples)
-    total = sum(counts.values())
-    if total == 0:
-        return {}
-    return {sample: count / total for sample, count in counts.items()}

vendor/vo_regular_bp/vo_regular_bp/minimization.py

@@ -1,326 +0,0 @@
-"""Exact quotient/minimization helpers for stochastic context graphs."""
-
-from __future__ import annotations
-
-from collections.abc import Hashable, Iterable, Sequence
-from dataclasses import dataclass
-from typing import Any
-
-from .context import Context, ContextGraph, Edge
-
-
-@dataclass(frozen=True)
-class ExactQuotientStats:
-    """State/edge counts for an exact graph quotient."""
-
-    states: int
-    classes: int
-    edges: int
-    quotient_edges: int
-    max_class_size: int
-    refinement_rounds: int
-
-    @property
-    def state_reduction(self) -> float:
-        return _ratio(self.states, self.classes)
-
-    @property
-    def edge_reduction(self) -> float:
-        return _ratio(self.edges, self.quotient_edges)
-
-    def as_dict(self) -> dict[str, int | float]:
-        return {
-            "states": self.states,
-            "classes": self.classes,
-            "edges": self.edges,
-            "quotient_edges": self.quotient_edges,
-            "max_class_size": self.max_class_size,
-            "refinement_rounds": self.refinement_rounds,
-            "state_reduction": self.state_reduction,
-            "edge_reduction": self.edge_reduction,
-        }
-
-
-def exact_context_graph_quotient_stats(graph: ContextGraph) -> ExactQuotientStats:
-    """Return exact quotient counts for a :class:`ContextGraph`."""
-
-    partition = _context_graph_partition(graph)
-    return partition.stats
-
-
-def minimize_context_graph(graph: ContextGraph) -> ContextGraph:
-    """Return a read-only exact quotient of ``graph``.
-
-    The quotient keeps one representative context per equivalence class and
-    records aliases so callers may still ask for outgoing edges using original
-    context labels.
-    """
-
-    partition = _context_graph_partition(graph)
-    if partition.stats.classes == partition.stats.states:
-        return graph
-
-    representative_by_class = partition.representatives
-    representative_for_state = {
-        state: representative_by_class[partition.class_by_state[state]]
-        for state in partition.states
-    }
-    edges_by_state: dict[Context, list[Edge]] = {}
-    for class_id, representative in enumerate(representative_by_class):
-        edges_by_state[representative] = [
-            Edge(
-                edge.symbol,
-                edge.probability,
-                representative_for_state[edge.next_state],
-                edge.order_weights,
-            )
-            for edge in graph.outgoing(representative)
-        ]
-
-    result = ContextGraph(
-        edges_by_state,
-        start_state=representative_for_state[graph.start_state],
-        max_order=graph.max_order,
-        alphabet=graph.alphabet,
-        validate=True,
-    )
-    result._alias_to_state = representative_for_state  # type: ignore[attr-defined]
-    result._quotient_stats = partition.stats  # type: ignore[attr-defined]
-    result._quotient_classes = tuple(  # type: ignore[attr-defined]
-        tuple(
-            state
-            for state in partition.states
-            if partition.class_by_state[state] == class_id
-        )
-        for class_id in range(len(representative_by_class))
-    )
-    return result
-
-
-def exact_fixed_order_graph_quotient_stats(graph: object) -> ExactQuotientStats:
-    """Return exact quotient counts for a fixed-order context graph."""
-
-    partition = _fixed_order_partition(graph)
-    return partition.stats
-
-
-def minimize_fixed_order_graph(graph: object) -> object:
-    """Return a minimized fixed-order graph, preserving original context lookup."""
-
-    partition = _fixed_order_partition(graph)
-    if partition.stats.classes == partition.stats.states:
-        return graph
-
-    from .order_stack_bp import FixedOrderContextGraph, StackEdge
-
-    minimized = FixedOrderContextGraph(graph.order)
-    representative_by_class = partition.representatives
-    minimized.contexts = [graph.contexts[index] for index in representative_by_class]
-    minimized.outgoing = [[] for _ in representative_by_class]
-
-    for original_context, original_state in graph.context_to_id.items():
-        minimized.context_to_id[original_context] = partition.class_by_state[original_state]
-
-    aliases: list[list[Context]] = [[] for _ in representative_by_class]
-    for state, context in enumerate(graph.contexts):
-        aliases[partition.class_by_state[state]].append(context)
-    minimized.state_aliases = tuple(tuple(group) for group in aliases)
-    minimized.quotient_stats = partition.stats
-
-    for class_id, representative_state in enumerate(representative_by_class):
-        minimized.outgoing[class_id] = [
-            StackEdge(
-                src=class_id,
-                dst=partition.class_by_state[edge.dst],
-                symbol=edge.symbol,
-                probability=edge.probability,
-                order=edge.order,
-            )
-            for edge in graph.outgoing[representative_state]
-        ]
-
-    return minimized
-
-
-def minimize_fixed_order_graphs(graphs: dict[int, object]) -> dict[int, object]:
-    """Minimize every materialized fixed-order graph in ``graphs``."""
-
-    return {
-        order: minimize_fixed_order_graph(graph)
-        for order, graph in graphs.items()
-    }
-
-
-@dataclass(frozen=True)
-class _ContextGraphPartition:
-    states: tuple[Context, ...]
-    class_by_state: dict[Context, int]
-    representatives: tuple[Context, ...]
-    stats: ExactQuotientStats
-
-
-@dataclass(frozen=True)
-class _FixedOrderPartition:
-    class_by_state: tuple[int, ...]
-    representatives: tuple[int, ...]
-    stats: ExactQuotientStats
-
-
-def _context_graph_partition(graph: ContextGraph) -> _ContextGraphPartition:
-    states = tuple(sorted(graph.states, key=_context_sort_key))
-    state_index = {state: index for index, state in enumerate(states)}
-    classes = [0 for _state in states]
-    rounds = 0
-
-    while True:
-        signatures = [
-            tuple(
-                sorted(
-                    (
-                        _hashable_key(edge.symbol),
-                        _float_key(edge.probability),
-                        _order_weights_key(edge.order_weights),
-                        classes[state_index[edge.next_state]],
-                    )
-                    for edge in graph.outgoing(state)
-                )
-            )
-            for state in states
-        ]
-        new_classes, counts = _classes_from_signatures(signatures)
-        rounds += 1
-        if new_classes == classes:
-            representatives = _representative_states(new_classes)
-            quotient_edges = sum(
-                len(graph.outgoing(states[state]))
-                for state in representatives
-            )
-            return _ContextGraphPartition(
-                states=states,
-                class_by_state={
-                    state: class_id
-                    for state, class_id in zip(states, new_classes)
-                },
-                representatives=tuple(states[state] for state in representatives),
-                stats=ExactQuotientStats(
-                    states=len(states),
-                    classes=len(counts),
-                    edges=graph.edge_count(),
-                    quotient_edges=quotient_edges,
-                    max_class_size=max(counts or [0]),
-                    refinement_rounds=rounds,
-                ),
-            )
-        classes = new_classes
-
-
-def _fixed_order_partition(graph: object) -> _FixedOrderPartition:
-    _require_materialized_fixed_order_graph(graph)
-    state_count = len(graph.contexts)
-    classes = [0 for _state in graph.contexts]
-    rounds = 0
-
-    while True:
-        signatures = [
-            tuple(
-                sorted(
-                    (
-                        _hashable_key(edge.symbol),
-                        _float_key(edge.probability),
-                        int(edge.order),
-                        classes[edge.dst],
-                    )
-                    for edge in edges
-                )
-            )
-            for edges in graph.outgoing
-        ]
-        new_classes, counts = _classes_from_signatures(signatures)
-        rounds += 1
-        if new_classes == classes:
-            representatives = _representative_states(new_classes)
-            quotient_edges = sum(
-                len(graph.outgoing[state])
-                for state in representatives
-            )
-            return _FixedOrderPartition(
-                class_by_state=tuple(new_classes),
-                representatives=tuple(representatives),
-                stats=ExactQuotientStats(
-                    states=state_count,
-                    classes=len(counts),
-                    edges=graph.edge_count,
-                    quotient_edges=quotient_edges,
-                    max_class_size=max(counts or [0]),
-                    refinement_rounds=rounds,
-                ),
-            )
-        classes = new_classes
-
-
-def _classes_from_signatures(
-    signatures: Sequence[Hashable],
-) -> tuple[list[int], list[int]]:
-    class_ids: dict[Hashable, int] = {}
-    classes: list[int] = []
-    counts: list[int] = []
-    for signature in signatures:
-        class_id = class_ids.get(signature)
-        if class_id is None:
-            class_id = len(class_ids)
-            class_ids[signature] = class_id
-            counts.append(0)
-        classes.append(class_id)
-        counts[class_id] += 1
-    return classes, counts
-
-
-def _representative_states(classes: Sequence[int]) -> list[int]:
-    representatives: dict[int, int] = {}
-    for state, class_id in enumerate(classes):
-        representatives.setdefault(class_id, state)
-    return [representatives[class_id] for class_id in range(len(representatives))]
-
-
-def _require_materialized_fixed_order_graph(graph: object) -> None:
-    from .order_stack_bp import FixedOrderContextGraph
-
-    if not isinstance(graph, FixedOrderContextGraph):
-        raise TypeError(
-            "exact source-graph minimization currently supports materialized "
-            "FixedOrderContextGraph instances"
-        )
-    required = ("order", "contexts", "context_to_id", "outgoing", "edge_count")
-    if not all(hasattr(graph, name) for name in required):
-        raise TypeError("expected a materialized fixed-order context graph")
-    if len(graph.outgoing) != len(graph.contexts):
-        raise ValueError(
-            "exact minimization requires materialized graph outgoing rows; "
-            "lazy graph views are not supported"
-        )
-
-
-def _context_sort_key(context: Context) -> tuple[int, str]:
-    return (len(context), repr(context))
-
-
-def _hashable_key(value: Any) -> Hashable:
-    try:
-        hash(value)
-    except TypeError:
-        return (type(value).__name__, repr(value))
-    return (type(value).__name__, value)
-
-
-def _float_key(value: float) -> tuple[int, int]:
-    return float(value).as_integer_ratio()
-
-
-def _order_weights_key(weights: Iterable[tuple[int, float]]) -> tuple[tuple[int, tuple[int, int]], ...]:
-    return tuple((int(order), _float_key(weight)) for order, weight in weights)
-
-
-def _ratio(numerator: int, denominator: int) -> float:
-    if denominator == 0:
-        return float("inf") if numerator else 1.0
-    return float(numerator) / float(denominator)

vendor/vo_regular_bp/vo_regular_bp/orbit_diagnostics.py

@@ -1,574 +0,0 @@
-"""Diagnostics for transformation-orbit structure in regular BP products."""
-
-from __future__ import annotations
-
-from collections import Counter
-from collections.abc import Hashable, Iterable, Mapping
-from dataclasses import dataclass
-from numbers import Integral
-from typing import Any
-
-from .context import Symbol
-from .order_stack_bp import RegularOrderStackBPResult
-from .positional_bp import PositionConstraint
-
-
-@dataclass(frozen=True)
-class ForbiddenPatternOrbitStats:
-    """Orbit counts for forbidden patterns and their DFA prefix states."""
-
-    pattern_count: int
-    pattern_orbit_count: int
-    prefix_state_count: int
-    prefix_state_orbit_count: int
-    alphabet_size: int | None = None
-    alphabet_orbit_count: int | None = None
-
-    @property
-    def pattern_reduction_factor(self) -> float:
-        return _ratio(self.pattern_count, self.pattern_orbit_count)
-
-    @property
-    def prefix_state_reduction_factor(self) -> float:
-        return _ratio(self.prefix_state_count, self.prefix_state_orbit_count)
-
-    @property
-    def alphabet_reduction_factor(self) -> float | None:
-        if self.alphabet_size is None or self.alphabet_orbit_count is None:
-            return None
-        return _ratio(self.alphabet_size, self.alphabet_orbit_count)
-
-    def as_dict(self) -> dict[str, int | float | None]:
-        return {
-            "forbidden_patterns": self.pattern_count,
-            "forbidden_pattern_orbits": self.pattern_orbit_count,
-            "forbidden_pattern_reduction": self.pattern_reduction_factor,
-            "dfa_prefix_states": self.prefix_state_count,
-            "dfa_prefix_state_orbits": self.prefix_state_orbit_count,
-            "dfa_prefix_state_reduction": self.prefix_state_reduction_factor,
-            "alphabet_size": self.alphabet_size,
-            "alphabet_orbits": self.alphabet_orbit_count,
-            "alphabet_reduction": self.alphabet_reduction_factor,
-        }
-
-
-@dataclass(frozen=True)
-class RegularProductOrbitStats:
-    """Orbit counts for the currently expanded regular BP product."""
-
-    time_indexed_product_states: int
-    time_indexed_product_state_orbits: int
-    product_states: int
-    product_state_orbits: int
-    product_edges: int
-    time_indexed_product_edge_orbits: int
-    product_edge_orbits: int
-    expanded_rows: int
-    transition_row_shape_orbits: int
-    max_time_state_orbit_size: int
-    max_product_state_orbit_size: int
-    max_time_edge_orbit_size: int
-    max_edge_orbit_size: int
-
-    @property
-    def time_indexed_product_state_reduction_factor(self) -> float:
-        return _ratio(
-            self.time_indexed_product_states,
-            self.time_indexed_product_state_orbits,
-        )
-
-    @property
-    def product_state_reduction_factor(self) -> float:
-        return _ratio(self.product_states, self.product_state_orbits)
-
-    @property
-    def time_indexed_product_edge_reduction_factor(self) -> float:
-        return _ratio(self.product_edges, self.time_indexed_product_edge_orbits)
-
-    @property
-    def product_edge_reduction_factor(self) -> float:
-        return _ratio(self.product_edges, self.product_edge_orbits)
-
-    @property
-    def transition_row_shape_reduction_factor(self) -> float:
-        return _ratio(self.expanded_rows, self.transition_row_shape_orbits)
-
-    def as_dict(self) -> dict[str, int | float]:
-        return {
-            "time_indexed_product_states": self.time_indexed_product_states,
-            "time_indexed_product_state_orbits": self.time_indexed_product_state_orbits,
-            "time_indexed_product_state_reduction": (
-                self.time_indexed_product_state_reduction_factor
-            ),
-            "product_states": self.product_states,
-            "product_state_orbits": self.product_state_orbits,
-            "product_state_reduction": self.product_state_reduction_factor,
-            "product_edges": self.product_edges,
-            "time_indexed_product_edge_orbits": self.time_indexed_product_edge_orbits,
-            "time_indexed_product_edge_reduction": (
-                self.time_indexed_product_edge_reduction_factor
-            ),
-            "product_edge_orbits": self.product_edge_orbits,
-            "product_edge_reduction": self.product_edge_reduction_factor,
-            "expanded_rows": self.expanded_rows,
-            "transition_row_shape_orbits": self.transition_row_shape_orbits,
-            "transition_row_shape_reduction": self.transition_row_shape_reduction_factor,
-            "max_time_state_orbit_size": self.max_time_state_orbit_size,
-            "max_product_state_orbit_size": self.max_product_state_orbit_size,
-            "max_time_edge_orbit_size": self.max_time_edge_orbit_size,
-            "max_edge_orbit_size": self.max_edge_orbit_size,
-        }
-
-
-@dataclass(frozen=True)
-class RegularRowSignatureStats:
-    """Exact shifted-row reuse counts for already-expanded regular BP rows."""
-
-    product_rows: int
-    product_row_signatures: int
-    reusable_product_rows: int
-    max_product_row_signature_size: int
-    time_masked_rows: int
-    time_masked_row_signatures: int
-    reusable_time_masked_rows: int
-    max_time_masked_row_signature_size: int
-    time_masked_edges: int
-
-    @property
-    def product_row_reduction_factor(self) -> float:
-        return _ratio(self.product_rows, self.product_row_signatures)
-
-    @property
-    def reusable_product_row_fraction(self) -> float:
-        return _ratio(self.reusable_product_rows, self.product_rows)
-
-    @property
-    def time_masked_row_reduction_factor(self) -> float:
-        return _ratio(self.time_masked_rows, self.time_masked_row_signatures)
-
-    @property
-    def reusable_time_masked_row_fraction(self) -> float:
-        return _ratio(self.reusable_time_masked_rows, self.time_masked_rows)
-
-    def as_dict(self) -> dict[str, int | float]:
-        return {
-            "exact_product_rows": self.product_rows,
-            "exact_product_row_signatures": self.product_row_signatures,
-            "exact_product_row_reduction": self.product_row_reduction_factor,
-            "exact_reusable_product_rows": self.reusable_product_rows,
-            "exact_reusable_product_row_fraction": self.reusable_product_row_fraction,
-            "exact_max_product_row_signature_size": self.max_product_row_signature_size,
-            "exact_time_masked_rows": self.time_masked_rows,
-            "exact_time_masked_row_signatures": self.time_masked_row_signatures,
-            "exact_time_masked_row_reduction": self.time_masked_row_reduction_factor,
-            "exact_reusable_time_masked_rows": self.reusable_time_masked_rows,
-            "exact_reusable_time_masked_row_fraction": (
-                self.reusable_time_masked_row_fraction
-            ),
-            "exact_max_time_masked_row_signature_size": (
-                self.max_time_masked_row_signature_size
-            ),
-            "exact_time_masked_edges": self.time_masked_edges,
-        }
-
-
-def canonical_integer_shift_key(
-    value: Any,
-    *,
-    fixed_symbols: Iterable[Symbol] = (),
-    reference: int | None = None,
-) -> Hashable:
-    """Canonicalize nested symbols modulo a common integer shift.
-
-    Integer-like symbols are represented relative to the first integer symbol
-    unless ``reference`` is supplied. Non-integer symbols, such as sentinels,
-    are kept fixed. The function is intentionally diagnostic: it exposes how
-    much repeated structure is present under transposition without changing any
-    BP semantics.
-    """
-
-    fixed = frozenset(fixed_symbols)
-    origin = reference
-    if origin is None:
-        first = _first_shiftable_symbol(value, fixed)
-        origin = 0 if first is None else first
-    return _canonicalize(value, fixed, int(origin))
-
-
-def forbidden_pattern_orbit_stats(
-    patterns: Iterable[Iterable[Symbol]],
-    *,
-    alphabet: Iterable[Symbol] | None = None,
-    fixed_symbols: Iterable[Symbol] = (),
-) -> ForbiddenPatternOrbitStats:
-    """Measure integer-shift orbit structure in forbidden n-grams."""
-
-    fixed = frozenset(fixed_symbols)
-    pattern_tuple = tuple(dict.fromkeys(tuple(pattern) for pattern in patterns))
-    prefixes: set[tuple[Symbol, ...]] = {()}
-    for pattern in pattern_tuple:
-        for prefix_len in range(1, len(pattern)):
-            prefixes.add(pattern[:prefix_len])
-
-    pattern_orbits = {
-        canonical_integer_shift_key(pattern, fixed_symbols=fixed)
-        for pattern in pattern_tuple
-    }
-    prefix_orbits = {
-        canonical_integer_shift_key(prefix, fixed_symbols=fixed)
-        for prefix in prefixes
-    }
-
-    alphabet_size = None
-    alphabet_orbit_count = None
-    if alphabet is not None:
-        alphabet_tuple = tuple(dict.fromkeys(alphabet))
-        alphabet_size = len(alphabet_tuple)
-        alphabet_orbit_count = len(
-            {
-                canonical_integer_shift_key(symbol, fixed_symbols=fixed)
-                for symbol in alphabet_tuple
-            }
-        )
-
-    return ForbiddenPatternOrbitStats(
-        pattern_count=len(pattern_tuple),
-        pattern_orbit_count=len(pattern_orbits),
-        prefix_state_count=len(prefixes),
-        prefix_state_orbit_count=len(prefix_orbits),
-        alphabet_size=alphabet_size,
-        alphabet_orbit_count=alphabet_orbit_count,
-    )
-
-
-def regular_product_orbit_stats(
-    result: RegularOrderStackBPResult,
-    *,
-    fixed_symbols: Iterable[Symbol] = (),
-    include_transition_row_shapes: bool = False,
-) -> RegularProductOrbitStats:
-    """Measure transposition-orbit repetition in an already-run regular BP."""
-
-    fixed = frozenset(fixed_symbols)
-    result.start_order_masses()
-
-    time_state_counts: Counter[Hashable] = Counter()
-    state_counts: Counter[Hashable] = Counter()
-    time_edge_counts: Counter[Hashable] = Counter()
-    edge_counts: Counter[Hashable] = Counter()
-    row_shapes: set[Hashable] = set()
-    state_seen: set[Hashable] = set()
-    state_key_cache: dict[tuple[int, int, Hashable], Hashable] = {}
-    acceptor_payload_cache: dict[Hashable, Hashable] = {}
-    edge_key_cache: dict[tuple[int, int, Hashable, int, Symbol, Hashable], Hashable] = {}
-    scanned_edge_count = 0
-    expanded_rows = 0
-
-    for order, cache in result.backwards.items():
-        graph = cache.graph
-        for time, graph_state, acceptor_state in cache.memo:
-            context = graph.contexts[graph_state]
-            acceptor_payload = acceptor_payload_cache.get(acceptor_state)
-            if acceptor_payload is None:
-                acceptor_payload = _acceptor_state_payload(result.acceptor, acceptor_state)
-                acceptor_payload_cache[acceptor_state] = acceptor_payload
-
-            raw_state_key = (order, graph_state, acceptor_state)
-            state_key = state_key_cache.get(raw_state_key)
-            if state_key is None:
-                state_payload = (context, acceptor_payload)
-                state_key = (
-                    order,
-                    canonical_integer_shift_key(state_payload, fixed_symbols=fixed),
-                )
-                state_key_cache[raw_state_key] = state_key
-            time_state_key = (time, *state_key)
-            time_state_counts[time_state_key] += 1
-            if raw_state_key not in state_seen:
-                state_seen.add(raw_state_key)
-                state_counts[state_key] += 1
-
-            if time == result.length:
-                continue
-
-            expanded_rows += 1
-            row_edges: list[Hashable] = []
-            reference = None
-            if include_transition_row_shapes:
-                reference = _first_shiftable_symbol((context, acceptor_payload), fixed)
-            for edge, next_acceptor_state, _transition_weight in cache.accepted_transitions(
-                graph_state,
-                acceptor_state,
-            ):
-                if not _constraint_allows(result.constraints.get(time), edge.symbol):
-                    continue
-                edge_cache_key = (
-                    order,
-                    graph_state,
-                    acceptor_state,
-                    edge.dst,
-                    edge.symbol,
-                    next_acceptor_state,
-                )
-                edge_key = edge_key_cache.get(edge_cache_key)
-                if edge_key is None:
-                    next_payload = acceptor_payload_cache.get(next_acceptor_state)
-                    if next_payload is None:
-                        next_payload = _acceptor_state_payload(
-                            result.acceptor,
-                            next_acceptor_state,
-                        )
-                        acceptor_payload_cache[next_acceptor_state] = next_payload
-                    dst_context = graph.contexts[edge.dst]
-                    edge_payload = (
-                        context,
-                        acceptor_payload,
-                        edge.symbol,
-                        dst_context,
-                        next_payload,
-                        edge.probability,
-                    )
-                    edge_key = (
-                        order,
-                        canonical_integer_shift_key(edge_payload, fixed_symbols=fixed),
-                    )
-                    edge_key_cache[edge_cache_key] = edge_key
-                time_edge_key = (time, *edge_key)
-                edge_counts[edge_key] += 1
-                time_edge_counts[time_edge_key] += 1
-                scanned_edge_count += 1
-
-                if include_transition_row_shapes:
-                    next_payload = acceptor_payload_cache[next_acceptor_state]
-                    dst_context = graph.contexts[edge.dst]
-                    row_reference = reference
-                    if row_reference is None:
-                        row_reference = _first_shiftable_symbol(
-                            (edge.symbol, dst_context, next_payload),
-                            fixed,
-                        )
-                    row_edges.append(
-                        canonical_integer_shift_key(
-                            (edge.symbol, dst_context, next_payload, edge.probability),
-                            fixed_symbols=fixed,
-                            reference=row_reference,
-                        )
-                    )
-
-            if include_transition_row_shapes:
-                row_key = (
-                    order,
-                    canonical_integer_shift_key(
-                        (context, acceptor_payload),
-                        fixed_symbols=fixed,
-                        reference=reference,
-                    ),
-                    tuple(sorted(row_edges, key=repr)),
-                )
-                row_shapes.add(row_key)
-
-    if scanned_edge_count != result.product_edge_count:
-        raise RuntimeError(
-            "orbit diagnostic edge scan did not match BP product-edge count: "
-            f"{scanned_edge_count} != {result.product_edge_count}"
-        )
-
-    return RegularProductOrbitStats(
-        time_indexed_product_states=result.time_indexed_product_state_count,
-        time_indexed_product_state_orbits=len(time_state_counts),
-        product_states=result.product_state_count,
-        product_state_orbits=len(state_counts),
-        product_edges=result.product_edge_count,
-        time_indexed_product_edge_orbits=len(time_edge_counts),
-        product_edge_orbits=len(edge_counts),
-        expanded_rows=expanded_rows,
-        transition_row_shape_orbits=len(row_shapes),
-        max_time_state_orbit_size=_max_count(time_state_counts),
-        max_product_state_orbit_size=_max_count(state_counts),
-        max_time_edge_orbit_size=_max_count(time_edge_counts),
-        max_edge_orbit_size=_max_count(edge_counts),
-    )
-
-
-def regular_row_signature_stats(
-    result: RegularOrderStackBPResult,
-    *,
-    fixed_symbols: Iterable[Symbol] = (),
-) -> RegularRowSignatureStats:
-    """Count exact row-template reuse under integer-shift canonicalization.
-
-    Unlike the looser orbit edge counters, a row signature contains the
-    canonical source payload, every accepted symbol, successor context, successor
-    DFA payload, and transition probability. Finite-transform boundary effects
-    and nonuniform continuation probabilities therefore split into distinct
-    signatures instead of being merged.
-    """
-
-    fixed = frozenset(fixed_symbols)
-    result.start_order_masses()
-
-    acceptor_payload_cache: dict[Hashable, Hashable] = {}
-    product_row_counts: Counter[Hashable] = Counter()
-    time_masked_row_counts: Counter[Hashable] = Counter()
-    product_seen: set[tuple[int, int, Hashable]] = set()
-    time_masked_edges = 0
-
-    for order, cache in result.backwards.items():
-        graph = cache.graph
-        for time, graph_state, acceptor_state in cache.memo:
-            if time == result.length:
-                continue
-
-            product_key = (order, graph_state, acceptor_state)
-            if product_key not in product_seen:
-                product_seen.add(product_key)
-                signature, _edge_count = _regular_row_signature(
-                    result,
-                    order=order,
-                    graph_state=graph_state,
-                    acceptor_state=acceptor_state,
-                    constraint=None,
-                    acceptor_payload_cache=acceptor_payload_cache,
-                    fixed_symbols=fixed,
-                )
-                product_row_counts[signature] += 1
-
-            time_signature, edge_count = _regular_row_signature(
-                result,
-                order=order,
-                graph_state=graph_state,
-                acceptor_state=acceptor_state,
-                constraint=result.constraints.get(time),
-                acceptor_payload_cache=acceptor_payload_cache,
-                fixed_symbols=fixed,
-            )
-            time_masked_row_counts[time_signature] += 1
-            time_masked_edges += edge_count
-
-    if time_masked_edges != result.product_edge_count:
-        raise RuntimeError(
-            "row-signature edge scan did not match BP product-edge count: "
-            f"{time_masked_edges} != {result.product_edge_count}"
-        )
-
-    return RegularRowSignatureStats(
-        product_rows=len(product_seen),
-        product_row_signatures=len(product_row_counts),
-        reusable_product_rows=_reusable_count(product_row_counts),
-        max_product_row_signature_size=_max_count(product_row_counts),
-        time_masked_rows=sum(time_masked_row_counts.values()),
-        time_masked_row_signatures=len(time_masked_row_counts),
-        reusable_time_masked_rows=_reusable_count(time_masked_row_counts),
-        max_time_masked_row_signature_size=_max_count(time_masked_row_counts),
-        time_masked_edges=time_masked_edges,
-    )
-
-
-def _acceptor_state_payload(acceptor: Any, state: Hashable) -> Hashable:
-    prefixes = getattr(acceptor, "prefixes", None)
-    if prefixes is not None and isinstance(state, int) and 0 <= state < len(prefixes):
-        return tuple(prefixes[state])
-    return ("dfa_state", state)
-
-
-def _constraint_allows(constraint: PositionConstraint | None, symbol: Symbol) -> bool:
-    if constraint is None:
-        return True
-    if callable(constraint):
-        return bool(constraint(symbol))
-    return symbol in constraint
-
-
-def _regular_row_signature(
-    result: RegularOrderStackBPResult,
-    *,
-    order: int,
-    graph_state: int,
-    acceptor_state: Hashable,
-    constraint: PositionConstraint | None,
-    acceptor_payload_cache: dict[Hashable, Hashable],
-    fixed_symbols: frozenset[Symbol],
-) -> tuple[Hashable, int]:
-    cache = result.backwards[order]
-    graph = cache.graph
-    context = graph.contexts[graph_state]
-    acceptor_payload = acceptor_payload_cache.get(acceptor_state)
-    if acceptor_payload is None:
-        acceptor_payload = _acceptor_state_payload(result.acceptor, acceptor_state)
-        acceptor_payload_cache[acceptor_state] = acceptor_payload
-
-    source_payload = (context, acceptor_payload)
-    reference = _first_shiftable_symbol(source_payload, fixed_symbols)
-    source_key = canonical_integer_shift_key(
-        source_payload,
-        fixed_symbols=fixed_symbols,
-        reference=reference,
-    )
-
-    transition_keys: list[Hashable] = []
-    edge_count = 0
-    for edge, next_acceptor_state, _transition_weight in cache.accepted_transitions(
-        graph_state,
-        acceptor_state,
-    ):
-        if not _constraint_allows(constraint, edge.symbol):
-            continue
-        next_payload = acceptor_payload_cache.get(next_acceptor_state)
-        if next_payload is None:
-            next_payload = _acceptor_state_payload(result.acceptor, next_acceptor_state)
-            acceptor_payload_cache[next_acceptor_state] = next_payload
-        dst_context = graph.contexts[edge.dst]
-        transition_keys.append(
-            canonical_integer_shift_key(
-                (
-                    edge.symbol,
-                    dst_context,
-                    next_payload,
-                    edge.probability,
-                ),
-                fixed_symbols=fixed_symbols,
-                reference=reference,
-            )
-        )
-        edge_count += 1
-
-    return (order, source_key, tuple(sorted(transition_keys, key=repr))), edge_count
-
-
-def _canonicalize(value: Any, fixed_symbols: frozenset[Symbol], reference: int) -> Hashable:
-    if _is_shiftable_integer(value, fixed_symbols):
-        return ("rel_int", int(value) - reference)
-    if isinstance(value, tuple):
-        return tuple(_canonicalize(item, fixed_symbols, reference) for item in value)
-    if isinstance(value, list):
-        return tuple(_canonicalize(item, fixed_symbols, reference) for item in value)
-    return ("fixed", value)
-
-
-def _first_shiftable_symbol(value: Any, fixed_symbols: frozenset[Symbol]) -> int | None:
-    if _is_shiftable_integer(value, fixed_symbols):
-        return int(value)
-    if isinstance(value, (tuple, list)):
-        for item in value:
-            found = _first_shiftable_symbol(item, fixed_symbols)
-            if found is not None:
-                return found
-    return None
-
-
-def _is_shiftable_integer(value: Any, fixed_symbols: frozenset[Symbol]) -> bool:
-    return isinstance(value, Integral) and not isinstance(value, bool) and value not in fixed_symbols
-
-
-def _max_count(counter: Mapping[Hashable, int]) -> int:
-    return max(counter.values(), default=0)
-
-
-def _reusable_count(counter: Mapping[Hashable, int]) -> int:
-    return sum(count for count in counter.values() if count > 1)
-
-
-def _ratio(numerator: int, denominator: int) -> float:
-    if denominator == 0:
-        return 0.0
-    return float(numerator) / float(denominator)

vendor/vo_regular_bp/vo_regular_bp/padding.py

@@ -1,35 +0,0 @@
-"""Utilities for fixed-horizon padded generation."""
-
-from __future__ import annotations
-
-from collections.abc import Iterable, Sequence
-from typing import TypeVar
-
-
-SymbolT = TypeVar("SymbolT")
-
-
-def append_padding(
-    sequences: Iterable[Sequence[SymbolT]],
-    *,
-    pad_symbol: SymbolT,
-    pad_count: int,
-) -> tuple[tuple[SymbolT, ...], ...]:
-    """Append repeated PAD symbols to each training sequence.
-
-    This is the standard fixed-horizon encoding of variable-length generation:
-    train on phrase/bar sequences followed by enough PAD symbols for the chosen
-    model order, then constrain PAD to be zero-cost and absorbing.
-
-    Probabilities are those of the padded model: the first PAD transition is a
-    modeled stop/ending event, while trailing PAD self-loops should become
-    deterministic when enough padding is appended.
-
-    Use at least ``max_order + 1`` PAD symbols when building an order-stack
-    model so every fixed-order PAD context has a PAD self-loop.
-    """
-
-    if pad_count < 1:
-        raise ValueError("pad_count must be at least 1")
-    padding = (pad_symbol,) * int(pad_count)
-    return tuple(tuple(sequence) + padding for sequence in sequences)

vendor/vo_regular_bp/vo_regular_bp/positional_bp.py

@@ -1,393 +0,0 @@
-"""Exact BP for fixed-horizon positional constraints.
-
-This module is the no-DFA specialization of product BP.  Positional constraints
-are represented as time-indexed symbol masks, so the recursion only ranges over
-context states:
-
-    beta[t, s] = sum_y P(y | s) beta[t + 1, canon(s, y))
-
-No approximation or pruning is introduced.
-"""
-
-from __future__ import annotations
-
-from collections import Counter, defaultdict
-from collections.abc import Callable, Iterable, Mapping, Sequence
-from dataclasses import dataclass, field
-import random
-from typing import Protocol
-
-from .context import Context, ContextGraph, Edge, Symbol, _as_context
-from .product_bp import _sample_order
-
-PositionConstraint = Callable[[Symbol], bool] | Iterable[Symbol]
-PositionConstraints = Mapping[int, PositionConstraint]
-AllowedForbiddenSymbols = Mapping[int, Iterable[Symbol]]
-
-
-class ContextModel(Protocol):
-    start_state: Context
-
-    def outgoing(self, context: Iterable[Symbol] | Context) -> tuple[Edge, ...]:
-        ...
-
-
-class LazyBackoffContextModel:
-    """Backoff continuation model with outgoing edges compiled on demand.
-
-    The stochastic semantics match ``ContextGraph.from_backoff_sequences``:
-    each context uses a normalized weighted mixture of its own continuation
-    distribution and lower-order suffix distributions.  The difference is only
-    operational: edges are materialized when reached by BP.
-    """
-
-    def __init__(
-        self,
-        counts: Mapping[Context, Counter[Symbol]],
-        *,
-        max_order: int,
-        backoff_weight: float,
-        start_state: Iterable[Symbol] | Context = (),
-        track_order_weights: bool = False,
-    ) -> None:
-        if max_order < 0:
-            raise ValueError("max_order must be non-negative")
-        if not 0.0 <= backoff_weight <= 1.0:
-            raise ValueError("backoff_weight must be in [0, 1]")
-
-        self.counts = dict(counts)
-        self.max_order = int(max_order)
-        self.backoff_weight = float(backoff_weight)
-        self.start_state = _as_context(start_state)
-        self.track_order_weights = bool(track_order_weights)
-        contexts = set(self.counts)
-        contexts.add(self.start_state)
-        self.contexts = frozenset(contexts)
-        self.alphabet = frozenset(symbol for counter in self.counts.values() for symbol in counter)
-        self.distributions = {
-            context: tuple((symbol, float(count) / total) for symbol, count in counter.items())
-            for context, counter in self.counts.items()
-            for total in (float(sum(counter.values())),)
-            if total > 0.0
-        }
-        self._outgoing_cache: dict[Context, tuple[Edge, ...]] = {}
-
-    @classmethod
-    def from_sequences(
-        cls,
-        sequences: Iterable[Sequence[Symbol]],
-        *,
-        max_order: int,
-        backoff_weight: float,
-        start_state: Iterable[Symbol] | Context = (),
-        track_order_weights: bool = False,
-    ) -> "LazyBackoffContextModel":
-        if max_order < 0:
-            raise ValueError("max_order must be non-negative")
-
-        counts: dict[Context, Counter[Symbol]] = defaultdict(Counter)
-        for sequence in sequences:
-            tokens = tuple(sequence)
-            for index, symbol in enumerate(tokens):
-                order_limit = min(max_order, index)
-                for order in range(order_limit + 1):
-                    context = tokens[index - order : index] if order else ()
-                    counts[context][symbol] += 1
-        return cls(
-            counts,
-            max_order=max_order,
-            backoff_weight=backoff_weight,
-            start_state=start_state,
-            track_order_weights=track_order_weights,
-        )
-
-    def outgoing(self, context: Iterable[Symbol] | Context) -> tuple[Edge, ...]:
-        state = _as_context(context)
-        cached = self._outgoing_cache.get(state)
-        if cached is not None:
-            return cached
-
-        if self.track_order_weights:
-            edges = self._outgoing_with_order_weights(state)
-            self._outgoing_cache[state] = edges
-            return edges
-
-        scores: dict[Symbol, float] = {}
-        context_order = len(state)
-        for order in range(context_order, -1, -1):
-            suffix = state[-order:] if order else ()
-            distribution = self.distributions.get(suffix)
-            if not distribution:
-                continue
-            weight = self.backoff_weight ** (context_order - order)
-            for symbol, probability in distribution:
-                contribution = weight * probability
-                scores[symbol] = scores.get(symbol, 0.0) + contribution
-
-        total_score = float(sum(scores.values()))
-        if total_score <= 0.0:
-            edges: tuple[Edge, ...] = ()
-        else:
-            edges = tuple(
-                Edge(
-                    symbol,
-                    score / total_score,
-                    ContextGraph._canon(state, symbol, self.contexts, self.max_order),
-                )
-                for symbol, score in scores.items()
-                if score > 0.0
-            )
-        self._outgoing_cache[state] = edges
-        return edges
-
-    def _outgoing_with_order_weights(self, state: Context) -> tuple[Edge, ...]:
-        scores: dict[Symbol, float] = {}
-        order_scores: dict[Symbol, Counter[int]] = defaultdict(Counter)
-        context_order = len(state)
-        for order in range(context_order, -1, -1):
-            suffix = state[-order:] if order else ()
-            distribution = self.distributions.get(suffix)
-            if not distribution:
-                continue
-            weight = self.backoff_weight ** (context_order - order)
-            for symbol, probability in distribution:
-                contribution = weight * probability
-                scores[symbol] = scores.get(symbol, 0.0) + contribution
-                order_scores[symbol][order] += contribution
-
-        total_score = float(sum(scores.values()))
-        if total_score <= 0.0:
-            return ()
-        return tuple(
-            Edge(
-                symbol,
-                score / total_score,
-                ContextGraph._canon(state, symbol, self.contexts, self.max_order),
-                _normalized_order_weights(order_scores[symbol], score),
-            )
-            for symbol, score in scores.items()
-            if score > 0.0
-        )
-
-    def prefix_context(self, prefix: Sequence[Symbol]) -> Context:
-        for order in range(min(self.max_order, len(prefix)), -1, -1):
-            suffix = tuple(prefix[-order:]) if order else ()
-            if suffix in self.contexts:
-                return suffix
-        return self.start_state
-
-    @property
-    def materialized_edge_count(self) -> int:
-        return sum(len(edges) for edges in self._outgoing_cache.values())
-
-
-@dataclass
-class PositionalBPResult:
-    model: ContextModel
-    length: int
-    start_context: Context
-    constraints: PositionConstraints
-    betas: list[dict[Context, float]]
-    allowed_edge_counts: list[dict[Context, int]]
-    _transition_weight_cache: dict[tuple[int, Context], tuple[tuple[Edge, float], ...]] = field(
-        default_factory=dict,
-        init=False,
-        repr=False,
-    )
-
-    @property
-    def partition_function(self) -> float:
-        return self.betas[0].get(self.start_context, 0.0)
-
-    @property
-    def unique_state_count(self) -> int:
-        return len(set().union(*self.betas)) if self.betas else 0
-
-    @property
-    def time_indexed_state_count(self) -> int:
-        return sum(len(layer) for layer in self.betas)
-
-    @property
-    def edge_count(self) -> int:
-        return sum(sum(layer.values()) for layer in self.allowed_edge_counts)
-
-    def transition_weights(self, time: int, context: Iterable[Symbol] | Context) -> tuple[tuple[Edge, float], ...]:
-        if time < 0 or time >= self.length:
-            raise ValueError("time is outside the BP horizon")
-        state = _as_context(context)
-        cache_key = (time, state)
-        cached = self._transition_weight_cache.get(cache_key)
-        if cached is not None:
-            return cached
-
-        beta_next = self.betas[time + 1]
-        constraint = self.constraints.get(time)
-        weighted = []
-        for edge in self.model.outgoing(state):
-            if not _allows(constraint, edge.symbol):
-                continue
-            weight = edge.probability * beta_next.get(edge.next_state, 0.0)
-            if weight > 0.0:
-                weighted.append((edge, weight))
-        result = tuple(weighted)
-        self._transition_weight_cache[cache_key] = result
-        return result
-
-    def _sample_edge(self, time: int, context: Context, generator: random.Random) -> Edge:
-        weighted = self.transition_weights(time, context)
-        total = sum(weight for _, weight in weighted)
-        if total <= 0.0:
-            raise RuntimeError("BP table contains no positive continuation for a reachable state")
-        threshold = generator.random() * total
-        cumulative = 0.0
-        chosen = weighted[-1][0]
-        for edge, weight in weighted:
-            cumulative += weight
-            if threshold <= cumulative:
-                return edge
-        return chosen
-
-    def sample(self, *, rng: random.Random | int | None = None) -> tuple[Symbol, ...]:
-        generator = _coerce_rng(rng)
-        if self.partition_function <= 0.0:
-            raise ValueError("cannot sample because the constrained partition function is zero")
-
-        context = self.start_context
-        output: list[Symbol] = []
-        for time in range(self.length):
-            edge = self._sample_edge(time, context, generator)
-            output.append(edge.symbol)
-            context = edge.next_state
-        return tuple(output)
-
-    def sample_many(self, count: int, *, rng: random.Random | int | None = None) -> list[tuple[Symbol, ...]]:
-        generator = _coerce_rng(rng)
-        return [self.sample(rng=generator) for _ in range(count)]
-
-    def sample_with_orders(
-        self,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> tuple[tuple[Symbol, ...], tuple[int, ...]]:
-        generator = _coerce_rng(rng)
-        if self.partition_function <= 0.0:
-            raise ValueError("cannot sample because the constrained partition function is zero")
-
-        context = self.start_context
-        output: list[Symbol] = []
-        orders: list[int] = []
-        for time in range(self.length):
-            edge = self._sample_edge(time, context, generator)
-            output.append(edge.symbol)
-            orders.append(_sample_order(edge.order_weights or ((len(context), 1.0),), generator))
-            context = edge.next_state
-        return tuple(output), tuple(orders)
-
-    def conditional_probability(self, sequence: Sequence[Symbol]) -> float:
-        if len(sequence) != self.length:
-            raise ValueError("sequence length must match the BP horizon")
-        if self.partition_function <= 0.0:
-            return 0.0
-
-        context = self.start_context
-        probability = 1.0
-        for time, symbol in enumerate(sequence):
-            beta_now = self.betas[time].get(context, 0.0)
-            if beta_now <= 0.0:
-                return 0.0
-            edge = next(
-                (edge for edge, _ in self.transition_weights(time, context) if edge.symbol == symbol),
-                None,
-            )
-            if edge is None:
-                return 0.0
-            weight = edge.probability * self.betas[time + 1].get(edge.next_state, 0.0)
-            if weight <= 0.0:
-                return 0.0
-            probability *= weight / beta_now
-            context = edge.next_state
-        return probability
-
-
-def run_positional_bp(
-    model: ContextModel,
-    *,
-    length: int,
-    start_context: Iterable[Symbol] | Context | None = None,
-    constraints: PositionConstraints | None = None,
-) -> PositionalBPResult:
-    """Run exact backward DP with time-indexed positional constraints."""
-
-    if length < 0:
-        raise ValueError("length must be non-negative")
-    context0 = model.start_state if start_context is None else _as_context(start_context)
-    position_constraints = dict(constraints or {})
-    _validate_constraint_positions(position_constraints, length)
-
-    betas: list[dict[Context, float]] = [dict() for _ in range(length + 1)]
-    allowed_edge_counts: list[dict[Context, int]] = [dict() for _ in range(length)]
-
-    def beta(time_index: int, context: Context) -> float:
-        cached = betas[time_index].get(context)
-        if cached is not None:
-            return cached
-        if time_index == length:
-            betas[time_index][context] = 1.0
-            return 1.0
-
-        constraint = position_constraints.get(time_index)
-        total = 0.0
-        allowed_count = 0
-        for edge in model.outgoing(context):
-            if not _allows(constraint, edge.symbol):
-                continue
-            allowed_count += 1
-            total += edge.probability * beta(time_index + 1, edge.next_state)
-
-        allowed_edge_counts[time_index][context] = allowed_count
-        betas[time_index][context] = total
-        return total
-
-    beta(0, context0)
-    return PositionalBPResult(
-        model=model,
-        length=length,
-        start_context=context0,
-        constraints=position_constraints,
-        betas=betas,
-        allowed_edge_counts=allowed_edge_counts,
-    )
-
-
-def _normalized_order_weights(
-    order_scores: Counter[int],
-    symbol_score: float,
-) -> tuple[tuple[int, float], ...]:
-    if symbol_score <= 0.0:
-        return ()
-    return tuple(
-        sorted(
-            ((order, contribution / symbol_score) for order, contribution in order_scores.items()),
-            reverse=True,
-        )
-    )
-
-
-def _allows(constraint: PositionConstraint | None, symbol: Symbol) -> bool:
-    if constraint is None:
-        return True
-    if callable(constraint):
-        return bool(constraint(symbol))
-    return symbol in constraint
-
-
-def _validate_constraint_positions(constraints: Mapping[int, PositionConstraint], length: int) -> None:
-    for position in constraints:
-        if position < 0 or position >= length:
-            raise IndexError(f"constraint position {position} is outside length {length}")
-
-
-def _coerce_rng(rng: random.Random | int | None) -> random.Random:
-    if isinstance(rng, random.Random):
-        return rng
-    return random.Random(rng)

vendor/vo_regular_bp/vo_regular_bp/product_bp.py

@@ -1,304 +0,0 @@
-"""Backward dynamic programming on reachable context-acceptor products."""
-
-from __future__ import annotations
-
-from dataclasses import dataclass, field
-import random
-from typing import Hashable, Iterable, Sequence
-
-from .acceptors import DFA, transition_weight as regular_transition_weight
-from .context import Context, ContextGraph, Symbol, _as_context
-
-ProductState = tuple[Context, Hashable]
-
-
-@dataclass(frozen=True)
-class ProductEdge:
-    symbol: Symbol
-    probability: float
-    transition_weight: float
-    next_state: ProductState
-    order_weights: tuple[tuple[int, float], ...] = ()
-
-
-@dataclass
-class ProductBPResult:
-    graph: ContextGraph
-    acceptor: DFA
-    length: int
-    start_state: ProductState
-    layers: list[set[ProductState]]
-    edges: list[dict[ProductState, tuple[ProductEdge, ...]]]
-    betas: list[dict[ProductState, float]]
-    _transition_weight_cache: dict[tuple[int, ProductState], tuple[tuple[ProductEdge, float], ...]] = field(
-        default_factory=dict,
-        init=False,
-        repr=False,
-    )
-
-    @property
-    def partition_function(self) -> float:
-        return self.betas[0].get(self.start_state, 0.0)
-
-    @property
-    def unique_product_state_count(self) -> int:
-        return len(set().union(*self.layers)) if self.layers else 0
-
-    @property
-    def time_indexed_product_state_count(self) -> int:
-        return sum(len(layer) for layer in self.layers)
-
-    @property
-    def product_edge_count(self) -> int:
-        return sum(len(edges) for layer_edges in self.edges for edges in layer_edges.values())
-
-    @property
-    def reachable_acceptor_state_count(self) -> int:
-        return len({state for layer in self.layers for _, state in layer})
-
-    def transition_weights(self, time: int, state: ProductState) -> tuple[tuple[ProductEdge, float], ...]:
-        """Return outgoing product edges and their conditional sampling weights."""
-
-        if time < 0 or time >= self.length:
-            raise ValueError("time is outside the BP horizon")
-        cache_key = (time, state)
-        if cache_key in self._transition_weight_cache:
-            return self._transition_weight_cache[cache_key]
-
-        beta_next = self.betas[time + 1]
-        weighted = []
-        for edge in self.edges[time].get(state, ()):
-            weight = (
-                edge.probability
-                * edge.transition_weight
-                * beta_next.get(edge.next_state, 0.0)
-            )
-            if weight > 0.0:
-                weighted.append((edge, weight))
-        result = tuple(weighted)
-        self._transition_weight_cache[cache_key] = result
-        return result
-
-    def _sample_edge(self, time: int, state: ProductState, generator: random.Random) -> ProductEdge:
-        weighted = self.transition_weights(time, state)
-        total = sum(weight for _, weight in weighted)
-        if total <= 0.0:
-            raise RuntimeError("BP table contains no positive continuation for a reachable state")
-        threshold = generator.random() * total
-        cumulative = 0.0
-        chosen = weighted[-1][0]
-        for edge, weight in weighted:
-            cumulative += weight
-            if threshold <= cumulative:
-                return edge
-        return chosen
-
-    def sample(self, *, rng: random.Random | int | None = None) -> tuple[Symbol, ...]:
-        """Draw one exact sample from the constrained distribution."""
-
-        generator = _coerce_rng(rng)
-        if self.partition_function <= 0.0:
-            raise ValueError("cannot sample because the constrained partition function is zero")
-
-        state = self.start_state
-        output: list[Symbol] = []
-        for time in range(self.length):
-            chosen = self._sample_edge(time, state, generator)
-            output.append(chosen.symbol)
-            state = chosen.next_state
-        return tuple(output)
-
-    def sample_many(self, count: int, *, rng: random.Random | int | None = None) -> list[tuple[Symbol, ...]]:
-        generator = _coerce_rng(rng)
-        return [self.sample(rng=generator) for _ in range(count)]
-
-    def sample_with_orders(
-        self,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> tuple[tuple[Symbol, ...], tuple[int, ...]]:
-        """Draw one exact sample and a latent order used at each event.
-
-        For explicit backoff edges, the order is sampled from the posterior
-        contribution of each suffix order to the selected symbol. For ordinary
-        MLE edges without order metadata, the current context length is used.
-        """
-
-        generator = _coerce_rng(rng)
-        if self.partition_function <= 0.0:
-            raise ValueError("cannot sample because the constrained partition function is zero")
-
-        state = self.start_state
-        output: list[Symbol] = []
-        orders: list[int] = []
-        for time in range(self.length):
-            chosen = self._sample_edge(time, state, generator)
-            output.append(chosen.symbol)
-            orders.append(_sample_order(chosen.order_weights, generator))
-            state = chosen.next_state
-        return tuple(output), tuple(orders)
-
-    def sample_many_with_orders(
-        self,
-        count: int,
-        *,
-        rng: random.Random | int | None = None,
-    ) -> list[tuple[tuple[Symbol, ...], tuple[int, ...]]]:
-        generator = _coerce_rng(rng)
-        return [self.sample_with_orders(rng=generator) for _ in range(count)]
-
-    def conditional_probability(self, sequence: Sequence[Symbol]) -> float:
-        """Probability of ``sequence`` under the constrained BP distribution."""
-
-        if len(sequence) != self.length:
-            raise ValueError("sequence length must match the BP horizon")
-        if self.partition_function <= 0.0:
-            return 0.0
-
-        state = self.start_state
-        probability = 1.0
-        for time, symbol in enumerate(sequence):
-            beta_now = self.betas[time].get(state, 0.0)
-            if beta_now <= 0.0:
-                return 0.0
-            edge = next(
-                (
-                    edge
-                    for edge in self.edges[time].get(state, ())
-                    if edge.symbol == symbol
-                ),
-                None,
-            )
-            if edge is None:
-                return 0.0
-            weight = (
-                edge.probability
-                * edge.transition_weight
-                * self.betas[time + 1].get(edge.next_state, 0.0)
-            )
-            if weight <= 0.0:
-                return 0.0
-            probability *= weight / beta_now
-            state = edge.next_state
-        return probability
-
-
-def run_bp(
-    graph: ContextGraph,
-    acceptor: DFA,
-    *,
-    length: int,
-    start_context: Iterable[Symbol] | Context | None = None,
-    start_acceptor_state: Hashable | None = None,
-) -> ProductBPResult:
-    """Run exact backward DP on the reachable product for a fixed horizon."""
-
-    if length < 0:
-        raise ValueError("length must be non-negative")
-
-    context0 = graph.start_state if start_context is None else _as_context(start_context)
-    acceptor0 = acceptor.start_state if start_acceptor_state is None else start_acceptor_state
-    start = (context0, acceptor0)
-
-    layers: list[set[ProductState]] = [set([start])]
-    edges_by_time: list[dict[ProductState, tuple[ProductEdge, ...]]] = []
-
-    for _time in range(length):
-        current_layer = layers[-1]
-        next_layer: set[ProductState] = set()
-        current_edges: dict[ProductState, tuple[ProductEdge, ...]] = {}
-
-        for context_state, acceptor_state in current_layer:
-            product_edges = []
-            for edge in graph.outgoing(context_state):
-                next_acceptor_state = acceptor.next_state(acceptor_state, edge.symbol)
-                if next_acceptor_state is None:
-                    continue
-                dfa_weight = regular_transition_weight(acceptor, acceptor_state, edge.symbol)
-                if dfa_weight <= 0.0:
-                    continue
-                next_product_state = (edge.next_state, next_acceptor_state)
-                product_edges.append(
-                    ProductEdge(
-                        symbol=edge.symbol,
-                        probability=edge.probability,
-                        transition_weight=dfa_weight,
-                        next_state=next_product_state,
-                        order_weights=edge.order_weights or ((len(context_state), 1.0),),
-                    )
-                )
-                next_layer.add(next_product_state)
-            current_edges[(context_state, acceptor_state)] = tuple(product_edges)
-
-        edges_by_time.append(current_edges)
-        layers.append(next_layer)
-
-    betas: list[dict[ProductState, float]] = [dict() for _ in range(length + 1)]
-    betas[length] = {
-        state: 1.0 if acceptor.is_accepting(state[1]) else 0.0
-        for state in layers[length]
-    }
-
-    for time in range(length - 1, -1, -1):
-        beta_next = betas[time + 1]
-        beta_now: dict[ProductState, float] = {}
-        for state in layers[time]:
-            beta_now[state] = sum(
-                edge.probability
-                * edge.transition_weight
-                * beta_next.get(edge.next_state, 0.0)
-                for edge in edges_by_time[time].get(state, ())
-            )
-        betas[time] = beta_now
-
-    return ProductBPResult(
-        graph=graph,
-        acceptor=acceptor,
-        length=length,
-        start_state=start,
-        layers=layers,
-        edges=edges_by_time,
-        betas=betas,
-    )
-
-
-def sample_exact(
-    graph: ContextGraph,
-    acceptor: DFA,
-    *,
-    length: int,
-    rng: random.Random | int | None = None,
-    start_context: Iterable[Symbol] | Context | None = None,
-    start_acceptor_state: Hashable | None = None,
-) -> tuple[Symbol, ...]:
-    """Convenience wrapper: run BP and draw one exact sample."""
-
-    result = run_bp(
-        graph,
-        acceptor,
-        length=length,
-        start_context=start_context,
-        start_acceptor_state=start_acceptor_state,
-    )
-    return result.sample(rng=rng)
-
-
-def _coerce_rng(rng: random.Random | int | None) -> random.Random:
-    if rng is None:
-        return random.Random()
-    if isinstance(rng, int):
-        return random.Random(rng)
-    return rng
-
-
-def _sample_order(order_weights: tuple[tuple[int, float], ...], rng: random.Random) -> int:
-    if not order_weights:
-        return 0
-    threshold = rng.random() * sum(weight for _, weight in order_weights)
-    cumulative = 0.0
-    chosen = order_weights[-1][0]
-    for order, weight in order_weights:
-        cumulative += weight
-        if threshold <= cumulative:
-            return order
-    return chosen