GraphBit's Techniques for CPU Efficiency: A Comprehensive Overview

Below is a concise, one-stop summary of GraphBit’s CPU efficiency techniques that are directly evidenced in the repo, with short code excerpts and file paths.

Lock-free per-node concurrency (no global semaphore)

Atomic counters and notify queues per node type; avoids hot global locks and reduces contention. File: core/src/types.rs

/// Eliminates global semaphore bottleneck struct NodeTypeConcurrency { max_concurrent: usize, current_count: Arcstd::sync::atomic::AtomicUsize, wait_queue: Arctokio::sync::Notify, }

CAS + async yield to prevent CPU spin

Compare-and-swap acquisition; yield when saturated to avoid busy waiting. File: core/src/embeddings.rs

if current < max_concurrency { match current_requests.compare_exchange(current, current + 1, AcqRel, Acquire) { Ok() => break, Err() => continue } } else { tokio::task::yield_now().await; // avoid busy waiting }

Fast path: selective permit acquisition

Skip permits for simple/non-agent nodes to cut per-task overhead. File: core/src/workflow.rs

let _permits = if matches!(node.node_type, NodeType::Agent { .. }) { Some(concurrency_manager.acquire_permits(&task_info).await?) } else { None };

Atomic cleanup with targeted wakeups

On drop: decrement atomically and wake exactly one waiter. File: core/src/types.rs

impl Drop for ConcurrencyPermits { fn drop(&mut self) { self.current_count.fetch_sub(1, AcqRel); self.wait_queue.notify_one(); } }

Configurable concurrency profiles (throughput / latency / memory)

Allows tuning to saturate cores or minimize noise. File: core/src/workflow.rs

pub fn new_high_throughput() -> Self { /* high throughput / } pub fn new_low_latency() -> Self { / fail_fast / } pub fn new_memory_optimized() -> Self { / conservative limits */ }

jemalloc on Unix (disabled where harmful)

Faster allocator on Unix; off on Windows/Python. File: core/src/lib.rs

#[cfg(all(not(feature = "python"), unix))] #[global_allocator] static GLOBAL: jemallocator::Jemalloc = jemallocator::Jemalloc;

HTTP connection pooling and keep-alive

Cuts per-request syscalls and CPU overhead for LLM calls. File: core/src/llm/openai.rs

let client = Client::builder() .timeout(Duration::from_secs(60)) .pool_max_idle_per_host(10) .pool_idle_timeout(Duration::from_secs(30)) .tcp_keepalive(Duration::from_secs(60)) .build()?;

Provider-level “first-call” cache (avoid redundant work)

Caches model verification to skip repeated initialization. File: core/src/llm/ollama.rs

let verified = self.model_verified.read().await; if *verified { return Ok(()); } // verify once, then set *verified = true

Graph-level caching for dependency lookups

Caches adjacency queries to avoid repeated traversal. File: core/src/graph.rs

if let Some(deps) = self.dependencies_cache.get(node_id) { return deps.clone(); }

Pre-allocation to reduce reallocs and cache misses

with_capacity for hot maps/vectors. File: core/src/graph.rs

Self { node_map: HashMap::with_capacity(16), nodes: HashMap::with_capacity(16), edges: Vec::with_capacity(16), }

CPU usage measured in benchmarks (validation)

CPU% computed from user+system time vs wall time. File: benchmarks/frameworks/common.py

cpu_time_used = (end_cpu.user - start_cpu.user) + (end_cpu.system - start_cpu.system) wall_time = end_time - self.start_time cpu_usage_percent = (cpu_time_used / wall_time) * 100 if wall_time > 0 else 0

CPU affinity helpers (optional tuning)

Pin process to specific cores for consistent benchmarks. File: benchmarks/frameworks/common.py

def set_process_affinity(cores): if cores and psutil is not None: psutil.Process().cpu_affinity(cores)