Production condensation (test_mode: false) caused SIGSEGV in TID within
~80s of startup despite #260 ring improvements + tombstone guard. Reverted
immediately. Root cause investigation required before re-enabling.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Enable production condensation with recently_freed tombstone guardβ
- condenser.rs: add recently_freed tombstone set (FREED_RECENCY_NS=5s).
unregister() stamps freed addresses; scan_and_compress() prunes stale
tombstones and skips any address within the recency window. Closes the
secondary address-reuse race: free(X) processed β malloc(X) re-registers
X β scan now skips X for 5s before treating it as a fresh allocation.
- membrane.rs: flip PIPELINE from test_mode:true β test_mode:false.
72h VPS stability check passed (no SIGBUS/SIGSEGV). Three-layer guard:
(1) 64K ring + deferred scan (#260) minimises unprocessed-free window;
(2) 60s burst gate blocks scan during startup coldload (original crash
window); (3) recently_freed tombstone closes the reuse race.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix Cargo.toml default feature: preload-only build for LD_PRELOAD safetyβ
Default was ["python"] which includes PyO3's extension-module feature.
PyO3 extension-module leaves Python symbols unresolved (expected to come
from the embedding process at runtime). When LD_PRELOAD loads the .so into
non-Python processes (bash ExecStartPre, grep, ss, etc.), the dynamic
linker hits `undefined symbol: PyExc_TypeError` and exits 127, killing
the service before it starts.
Fix: default = [] (no features). `cargo build --release` now produces a
preload-safe .so with no Python symbol dependencies. Python bindings are
built explicitly with `--features python` when needed.
Also fixes cleanup.conf on VPS (system file, not tracked): ExecStartPre
now uses `env -u LD_PRELOAD` to strip LD_PRELOAD for the bash cleanup
script, as a belt-and-suspenders guard against future .so load issues.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Derived from 40 days / 1M+ observations in condensate_observer.py:
- neurograph_rpc burst events: 8238 >100MB/10s, max 7042MB in 10.5s (672MB/s)
- Median restart interval: 2min; coldload to steady state: 31s median
- Observer p95 growth rate: 21.5 MB/s
Changes:
1. RING_SIZE 8192β65536: burst peak = 171k events/s, old ring filled in 47ms,
new ring buys 380ms drain headroom.
2. Defer scan() to post-batch: was firing inline inside 1024-event loop while
pending EVENT_FREEs for same addresses sat 3 slots ahead. Root cause of the
scan race. Now deferred to after the full batch is drained.
3. Burst gate: suppress scan() for 60s when rolling 10s alloc rate exceeds
50 MB/s (2.3Γ observer p95). Handles coldload storms and burst alloc events.
4. Ring overflow counter: RING_OVERFLOW_COUNT surfaced in summary output.
5. Large-alloc passthrough: allocs >10MB skip condenser registration and Lenia
field but still flow through graph/predictor for pattern learning. Covers
ONNX model (β¦
Fix condenser SIGSEGV β set test_mode=true in LD_PRELOAD pipelineβ
When running as LD_PRELOAD, the global PIPELINE condenser was transitioning
to ACTIVE mode and calling scan_and_compress(), which directly read from and
wrote back to live Python/uvicorn heap addresses via unsafe ptr operations.
Freed allocations were read and their reused memory overwritten, corrupting
glibc chunk metadata and crashing in libc freelist traversal ~35-50s after
TID startup.
Fix: PipelineConfig { test_mode: true } on the global PIPELINE. Condenser
learns allocation patterns (graph, predictor) but never dereferences observed
addresses in LD_PRELOAD context.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
real_malloc's zero-pointer fallback called dlsym(RTLD_NEXT, "malloc"),
but dlsym calls malloc internally β infinite recursion β stack overflow
β SIGSEGV. Killed TID and Node.js on every fresh start.
Fix: static 1MB bump-allocator in .bss (EARLY_BUF / EARLY_POS) that
serves the handful of allocs dlsym needs during bootstrap. Once
cache_real_functions() returns, REAL_MALLOC is set and all subsequent
allocs go through the real allocator. Early-buf pointers are detected
in real_free() via is_early_ptr() and silently dropped (they live in
.bss, never need freeing).
Closes #258.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Born dormant: grace period replaces skip list for system-wide safetyβ
The membrane crashed Node.js and xrdp during initialization because
dlsym(RTLD_NEXT) was called on every malloc before the process was
ready. Skip lists are whack-a-mole.
New approach: every process starts DORMANT (pure passthrough, ~1ns
atomic load per malloc). After a 10-second grace period, the first
malloc transitions to OBSERVING and starts the drain thread. The
process finishes initializing unmolested.
Changes:
- Cache real malloc/free pointers once at init (no per-call dlsym)
- Three-phase engagement: DORMANT β OBSERVING β ACTIVE
- Grace period before any recording (10s default)
- Drain thread and atexit only start when actually observing
- Removed skip list β all processes get the same treatment
The substrate will decide when OBSERVING β ACTIVE. For now,
OBSERVING does full pipeline processing. ACTIVE will add condensation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skip Node.js processes in LD_PRELOAD hooks β fix gateway SEGVβ
Node.js/V8 crashes with SIGSEGV when starting under the membrane.
Rather than debug V8's allocator, skip Node.js entirely β it's the
gateway process, not the condensation target.
Process detection via /proc/self/exe at init time. If binary name
matches skip list (node, npm, npx), set SKIP_PROCESS flag. Hooked
malloc/free become pure passthrough β single atomic load, zero overhead.
No drain thread started, no atexit registered. The process runs as if
the membrane isn't there.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lazy neighbor cleanup: remove O(N) scan from every freeβ
Eager neighbor list cleanup on remove_region() was O(N Γ avg_neighbors)
per free β killed throughput from 1.47M/s to 48K/s. Stale neighbor
references are harmless (step() already checks regions.contains_key).
Batch cleanup deferred to prune_stale_neighbors() during sleep.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Exorcise field ghosts: remove_region on free, dead allocations leave the fieldβ
63,000 cold regions in the Lenia field were freed allocations that never
got removed. The field was paying to cool corpses β stepping temperature
on memory that no longer exists.
Fix: add LeniaField::remove_region(), wire into Pipeline::process_free().
Freed allocations are removed from the field, their energy reclaimed,
and their neighbor links cleaned up. The field only contains live memory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix LD_PRELOAD deadlock: static .bss ring buffer, zero heap allocationβ
LazyLock heap-allocated the ring buffer via malloc β which is hooked β
causing infinite recursion deadlock on startup.
Fix: pack events into AtomicU64 (8 bytes each), store in a static array
that lives in .bss (zero-initialized by the OS loader, no malloc needed).
8K slots Γ 8 bytes = 64KB ring, entirely in static memory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lock-free ring buffer for LD_PRELOAD hooks β eliminate malloc overheadβ
The malloc/free hooks were doing 2 mutex try_locks + pipeline processing
synchronously on every allocation. This caused 17-19x slowdown on rapid
small allocations (TID's inference pattern).
Fix: lock-free ring buffer (64K slots, 2MB). malloc/free push events with
a single atomic increment (~10ns). Background drain thread reads events
in batches and feeds the pipeline. No locks on the hot path.
If the drain thread falls behind, events are dropped (better to lose an
event than stall malloc). The ring is sized for 64K in-flight events β
far more than any realistic burst.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Condensate v2: Full Rust conversion β 12 modules, 105 tests, zero Python inflationβ
Every allocation heats the Lenia field. Every 100 events the field
steps forward β growth function, decay, mass conservation. Cold
regions identified by field temperature, not hard idle timers.
Tested on live workload:
216 HOT / 502 WARM / 1,098 COLD regions
Continuous thermal gradient (not discrete tiers)
18 Lenia steps, energy at 1.2% of budget
1,998 predictions fired, 1,598 acted (80%)
The organism:
Membrane = sensory input (raw malloc/free)
Graph = substrate (learns topology)
Predictor = spreading activation (spike propagation)
Condenser = motor output (compress/promote)
Lenia = autonomic nervous system (continuous thermal field)
Pipeline = the River (connects everything)
28 tests, 0 failures. 7 modules. One living system.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lenia thermal field: continuous dynamics for memory temperatureβ
Gaussian splatting meets Lenia dynamics:
- Each memory region is a Gaussian in memory space
- Temperature = opacity (access heat)
- Growth function: Gaussian bump on neighborhood activation
- Mass conservation: total energy bounded by RAM budget
- Continuous gradient: no hard HOT/WARM/COLD tiers
Splat analogy test: 3 hot regions / 7 cold regions correctly
identified through field dynamics. Energy at 26.4% of budget
with mass conservation holding.
6 Lenia tests + 22 existing = 28 total, 0 failures.
Not yet wired into pipeline β next step is replacing the
condenser's hard tiers with Lenia field dynamics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
malloc/free hooks now feed BOTH membrane state (stats) and the
living pipeline (learn β predict β act). Periodic compression
scan every 10,000 allocs.
Tested on live Python + numpy workload:
19,396 events processed
500 nodes, 11,239 edges, 3 clusters discovered
6 graph rebuilds (substrate self-organizing)
40 managed regions tracked
On exit: both membrane stats AND pipeline report print.
Full loop: LD_PRELOAD β membrane β graph β predictor β condenser.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pipeline: the living loop β membrane β graph β predictor β condenserβ
The River flows. Allocation events enter the membrane, feed the
graph (the substrate), trigger predictions (spreading activation),
and the condenser acts (motor output). No orchestrator. The
substrate drives itself.
- Graph rebuilds automatically from accumulated events
- Predictor fires spikes when patterns repeat
- Condenser pre-promotes predicted regions before access
- Addresses bucketed by size class for better pattern learning
- Event buffer keeps 20% for continuity across rebuilds
22 tests pass across all 5 modules.
All 4 layers connected into one living system.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rust condenser: HOT/WARM/COLD tier management with LZ4 compressionβ
All four Condensate layers now in Rust:
Membrane β intercepts malloc/free system-wide (LD_PRELOAD)
Graph β learns causal access topology
Predictor β predicts next access from learned patterns
Condenser β compresses idle regions, pre-promotes on prediction
17 tests pass. Raw input, no classification β Law 7 for memory.
The substrate learns allocation patterns from pure observation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hooks malloc/free via LD_PRELOAD to track memory allocations for
ANY process, not just Python. Records allocation patterns, size
distribution, hot/cold classification.
Tested on live Python + numpy workload:
15,237 malloc calls intercepted
5 huge allocations (38.1 MB numpy arrays) correctly identified
Size distribution matches actual workload
Feature-gated: --no-default-features builds without PyO3.
All 12 tests pass.
Usage: LD_PRELOAD=libcondensate_core.so ./any_program
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rust core: AccessGraph + RustPredictor with PyO3 bindingsβ
