--- license: apache-2.0 base_model: - google/gemma-4-E4B-it pipeline_tag: text-generation --- # J-Space Deliberation Engine ![7bf90805-af84-4287-9cab-5241a3741387](https://cdn-uploads.huggingface.co/production/uploads/644afe169279988e0cbcd2d9/3Lw9hiiDAOhLaRU3NsrQR.png) > **Structured Latent Deliberation for Gemma 4 E4B using Jacobian Lens** Use our Web Interface: 👉 **https://github.com/guell11/Lebron-Local-UI** --- ## Overview **J-Space Deliberation Engine** is a latent reasoning architecture coupled to **Gemma 4 E4B-it**. Inspired by Anthropic's discovery of the **J-space** (Global Workspace Theory), this project extends the concept beyond interpretability. While previous work used the **Jacobian Lens** only to observe the model's internal representations, **J-Space actively trains and structures the latent space**. Instead of allowing residual vectors to overlap chaotically, the engine introduces a **5-slot deliberation workspace** inside the residual stream. Each slot represents an isolated reasoning stage before token generation, encouraging organized internal reasoning. --- # How It Works ## Structured Latent Deliberation + Native Chain-of-Thought The J-Space module injects sparse conceptual representations into the residual stream while Gemma continues performing standard autoregressive generation. The architecture combines: - Continuous latent representations divided into isolated slots - Recurrent deliberative memory - Standard autoregressive token generation - Jacobian Lens projections and control --- # Five Non-Overlapping Latent Workspace Slots The internal reasoning process is organized into five strictly separated latent states. | Slot | Function | Description | |-------|----------|-------------| | **Objective** | Defines the goal | Maps the task objective without interference | | **Hypothesis** | Stores candidate solutions | Keeps possible solution paths isolated | | **Evidence** | Stores relevant signals | Filters and preserves contextual evidence | | **Critic** | Evaluates consistency | Detects contradictions and logical errors | | **Commit** | Produces final decision | Consolidates the latent state for text generation | --- # Runtime Quantization The Gemma 4 backbone can be executed efficiently using: - BitsAndBytes 4-bit NF4 - PyTorch - Accelerate while keeping all J-Space modules in full precision. --- # Installation ```bash pip install torch transformers accelerate bitsandbytes huggingface_hub ``` --- # Running Inference This example loads **Gemma 4 E4B-it** quantized in **NF4** and attaches the structured J-Space artifacts directly from Hugging Face. ```python import torch from transformers import ( AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig ) from huggingface_hub import hf_hub_download from lebron_jspace.reasoner import JReasonerModule REPO_ID = "guell00/J-Space-Deliberation" BASE_MODEL = "google/gemma-4-E4B-it" REVISION = "fee6332c1abaafb77f6f9624236c63aa2f1d0187" print("1. Configuring 4-bit NF4 quantization...") quant_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True ) print("2. Loading base model and tokenizer...") tokenizer = AutoTokenizer.from_pretrained( BASE_MODEL, revision=REVISION ) model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, revision=REVISION, quantization_config=quant_config, device_map="auto" ) print("3. Downloading J-Space artifacts...") adapter_path = hf_hub_download( repo_id=REPO_ID, filename="jreasoner_adapter.pt" ) config_path = hf_hub_download( repo_id=REPO_ID, filename="jreasoner_config.json" ) dict_path = hf_hub_download( repo_id=REPO_ID, filename="jspace_dictionary_v3.pt" ) print("4. Attaching J-Space Deliberation Engine...") jspace_engine = JReasonerModule.load_from_checkpoint( model=model, adapter_path=adapter_path, config_path=config_path, dictionary_path=dict_path ) print("Model ready for inference!") ``` --- # Generation Example ```python prompt = """ user Explain logically: If every A is B, and every B is C, what can we conclude about A and C? model """ inputs = tokenizer( prompt, return_tensors="pt" ).to("cuda") with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=200 ) print("\nModel Response:") print( tokenizer.decode( outputs[0], skip_special_tokens=True ) ) ``` --- # Repository Files | File | Description | |------|-------------| | `jreasoner_adapter.pt` | Recurrent module weights responsible for organizing the latent workspace | | `jreasoner_config.json` | Configuration for the five latent slots and gating mechanisms | | `jspace_dictionary_v3.pt` | Sparse concept dictionary | | `jacobian_lens.pt` | Jacobian Lens projection matrix | | `LICENSE` | Apache 2.0 License | --- # Architecture ```text ┌─────────────────────┐ │ User Prompt │ └──────────┬──────────┘ │ ▼ ┌─────────────────────┐ │ Gemma 4 E4B-it │ └──────────┬──────────┘ │ Residual Stream Injection │ ▼ ┌────────────────────────────────┐ │ J-Space Workspace │ │ │ │ Objective │ │ Hypothesis │ │ Evidence │ │ Critic │ │ Commit │ └──────────────┬─────────────────┘ │ ▼ Jacobian Lens Projection │ ▼ Final Text Generation ``` --- # What is J-Space? J-Space Deliberation Engine is an experimental latent reasoning layer that introduces a structured continuous workspace into Gemma 4, allowing the model to internally organize its reasoning process before generating tokens. --- # Original Contribution **Author** **Miguel Penha Reis** Publication date: **July 26, 2026** --- ## Research Summary J-Space combines several ideas into a unified latent reasoning architecture: - Non-overlapping latent workspace slots - Active latent-space training using Jacobian Lens - Structured residual-stream injection - Multi-step recurrent causal reasoning - NF4 quantization while preserving full-precision reasoning modules Unlike previous Jacobian Lens work, which focused primarily on interpreting latent representations, this implementation uses those representations as an active reasoning workspace. The central proposal is a latent deliberation engine that structures internal representations into five causal compartments before autoregressive generation. --- # Related Work | Area | Difference from J-Space | |------|--------------------------| | **Anthropic – J-space / Global Workspace** | Maps the latent workspace for interpretability. J-Space actively trains and structures it. | | **Chain-of-Thought** | Uses intermediate output tokens. J-Space operates directly in latent representations. | | **Recurrent Memory Networks** | Maintain recurrent hidden states without an explicit structured workspace. | | **Activation Steering** | Modifies isolated activations. J-Space builds a persistent internal reasoning structure. | --- # License Apache License 2.0 --- # Citation ```bibtex @software{reis2026jspace, author = {Miguel Penha Reis}, title = {J-Space Deliberation Engine}, year = {2026}, url = {https://huggingface.co/guell00/J-Space-Deliberation}, license = {Apache-2.0} } ``` --- # Author **Miguel Penha Reis** Creator of the **J-Space Deliberation Engine**, an experimental latent reasoning architecture for large language models built upon **Gemma 4 E4B-it**. --- ## History - **Project created:** July 26, 2026 - **Architecture:** J-Space Deliberation Engine - **Base model:** Gemma 4 E4B-it - **Method:** Structured Latent Deliberation + Jacobian Lens --- ## Scientific Note This repository presents an **independent experimental research project**. Claims of originality refer specifically to the implementation provided here and to the proposed method of **actively organizing latent representations into non-overlapping reasoning slots** prior to token generation. The project should be understood as an experimental exploration of structured latent reasoning rather than a claim about the capabilities or internal mechanisms of language models in general.