| --- |
| license: apache-2.0 |
| base_model: |
| - google/gemma-4-E4B-it |
| pipeline_tag: text-generation |
| --- |
| # J-Space Deliberation Engine |
|
|
|  |
|
|
| > **Structured Latent Deliberation for Gemma 4 E4B using Jacobian Lens** |
|
|
| Use our Web Interface: |
|
|
| 👉 **https://github.com/guell11/Lebron-Local-UI** |
|
|
| --- |
|
|
| ## Overview |
|
|
| **J-Space Deliberation Engine** is a latent reasoning architecture coupled to **Gemma 4 E4B-it**. |
|
|
| Inspired by Anthropic's discovery of the **J-space** (Global Workspace Theory), this project extends the concept beyond interpretability. |
|
|
| While previous work used the **Jacobian Lens** only to observe the model's internal representations, **J-Space actively trains and structures the latent space**. |
|
|
| Instead of allowing residual vectors to overlap chaotically, the engine introduces a **5-slot deliberation workspace** inside the residual stream. |
|
|
| Each slot represents an isolated reasoning stage before token generation, encouraging organized internal reasoning. |
|
|
| --- |
|
|
| # How It Works |
|
|
| ## Structured Latent Deliberation + Native Chain-of-Thought |
|
|
| The J-Space module injects sparse conceptual representations into the residual stream while Gemma continues performing standard autoregressive generation. |
|
|
| The architecture combines: |
|
|
| - Continuous latent representations divided into isolated slots |
| - Recurrent deliberative memory |
| - Standard autoregressive token generation |
| - Jacobian Lens projections and control |
|
|
| --- |
|
|
| # Five Non-Overlapping Latent Workspace Slots |
|
|
| The internal reasoning process is organized into five strictly separated latent states. |
|
|
| | Slot | Function | Description | |
| |-------|----------|-------------| |
| | **Objective** | Defines the goal | Maps the task objective without interference | |
| | **Hypothesis** | Stores candidate solutions | Keeps possible solution paths isolated | |
| | **Evidence** | Stores relevant signals | Filters and preserves contextual evidence | |
| | **Critic** | Evaluates consistency | Detects contradictions and logical errors | |
| | **Commit** | Produces final decision | Consolidates the latent state for text generation | |
|
|
| --- |
|
|
| # Runtime Quantization |
|
|
| The Gemma 4 backbone can be executed efficiently using: |
|
|
| - BitsAndBytes 4-bit NF4 |
| - PyTorch |
| - Accelerate |
|
|
| while keeping all J-Space modules in full precision. |
|
|
| --- |
|
|
| # Installation |
|
|
| ```bash |
| pip install torch transformers accelerate bitsandbytes huggingface_hub |
| ``` |
|
|
| --- |
|
|
| # Running Inference |
|
|
| This example loads **Gemma 4 E4B-it** quantized in **NF4** and attaches the structured J-Space artifacts directly from Hugging Face. |
|
|
| ```python |
| import torch |
| |
| from transformers import ( |
| AutoModelForCausalLM, |
| AutoTokenizer, |
| BitsAndBytesConfig |
| ) |
| |
| from huggingface_hub import hf_hub_download |
| |
| from lebron_jspace.reasoner import JReasonerModule |
| |
| |
| REPO_ID = "guell00/J-Space-Deliberation" |
| |
| BASE_MODEL = "google/gemma-4-E4B-it" |
| |
| REVISION = "fee6332c1abaafb77f6f9624236c63aa2f1d0187" |
| |
| |
| print("1. Configuring 4-bit NF4 quantization...") |
| |
| quant_config = BitsAndBytesConfig( |
| load_in_4bit=True, |
| bnb_4bit_quant_type="nf4", |
| bnb_4bit_compute_dtype=torch.bfloat16, |
| bnb_4bit_use_double_quant=True |
| ) |
| |
| print("2. Loading base model and tokenizer...") |
| |
| tokenizer = AutoTokenizer.from_pretrained( |
| BASE_MODEL, |
| revision=REVISION |
| ) |
| |
| model = AutoModelForCausalLM.from_pretrained( |
| BASE_MODEL, |
| revision=REVISION, |
| quantization_config=quant_config, |
| device_map="auto" |
| ) |
| |
| print("3. Downloading J-Space artifacts...") |
| |
| adapter_path = hf_hub_download( |
| repo_id=REPO_ID, |
| filename="jreasoner_adapter.pt" |
| ) |
| |
| config_path = hf_hub_download( |
| repo_id=REPO_ID, |
| filename="jreasoner_config.json" |
| ) |
| |
| dict_path = hf_hub_download( |
| repo_id=REPO_ID, |
| filename="jspace_dictionary_v3.pt" |
| ) |
| |
| print("4. Attaching J-Space Deliberation Engine...") |
| |
| jspace_engine = JReasonerModule.load_from_checkpoint( |
| model=model, |
| adapter_path=adapter_path, |
| config_path=config_path, |
| dictionary_path=dict_path |
| ) |
| |
| print("Model ready for inference!") |
| ``` |
|
|
| --- |
|
|
| # Generation Example |
|
|
| ```python |
| prompt = """ |
| user |
| Explain logically: |
| |
| If every A is B, |
| and every B is C, |
| |
| what can we conclude about A and C? |
| |
| model |
| """ |
| |
| inputs = tokenizer( |
| prompt, |
| return_tensors="pt" |
| ).to("cuda") |
| |
| with torch.no_grad(): |
| |
| outputs = model.generate( |
| **inputs, |
| max_new_tokens=200 |
| ) |
| |
| print("\nModel Response:") |
| |
| print( |
| tokenizer.decode( |
| outputs[0], |
| skip_special_tokens=True |
| ) |
| ) |
| ``` |
|
|
| --- |
|
|
| # Repository Files |
|
|
| | File | Description | |
| |------|-------------| |
| | `jreasoner_adapter.pt` | Recurrent module weights responsible for organizing the latent workspace | |
| | `jreasoner_config.json` | Configuration for the five latent slots and gating mechanisms | |
| | `jspace_dictionary_v3.pt` | Sparse concept dictionary | |
| | `jacobian_lens.pt` | Jacobian Lens projection matrix | |
| | `LICENSE` | Apache 2.0 License | |
|
|
| --- |
|
|
| # Architecture |
|
|
| ```text |
| ┌─────────────────────┐ |
| │ User Prompt │ |
| └──────────┬──────────┘ |
| │ |
| ▼ |
| ┌─────────────────────┐ |
| │ Gemma 4 E4B-it │ |
| └──────────┬──────────┘ |
| │ |
| Residual Stream Injection |
| │ |
| ▼ |
| ┌────────────────────────────────┐ |
| │ J-Space Workspace │ |
| │ │ |
| │ Objective │ |
| │ Hypothesis │ |
| │ Evidence │ |
| │ Critic │ |
| │ Commit │ |
| └──────────────┬─────────────────┘ |
| │ |
| ▼ |
| Jacobian Lens Projection |
| │ |
| ▼ |
| Final Text Generation |
| ``` |
|
|
| --- |
|
|
| # What is J-Space? |
|
|
| J-Space Deliberation Engine is an experimental latent reasoning layer that introduces a structured continuous workspace into Gemma 4, allowing the model to internally organize its reasoning process before generating tokens. |
|
|
| --- |
|
|
| # Original Contribution |
|
|
| **Author** |
|
|
| **Miguel Penha Reis** |
|
|
| Publication date: |
|
|
| **July 26, 2026** |
|
|
| --- |
|
|
| ## Research Summary |
|
|
| J-Space combines several ideas into a unified latent reasoning architecture: |
|
|
| - Non-overlapping latent workspace slots |
| - Active latent-space training using Jacobian Lens |
| - Structured residual-stream injection |
| - Multi-step recurrent causal reasoning |
| - NF4 quantization while preserving full-precision reasoning modules |
|
|
| Unlike previous Jacobian Lens work, which focused primarily on interpreting latent representations, this implementation uses those representations as an active reasoning workspace. |
|
|
| The central proposal is a latent deliberation engine that structures internal representations into five causal compartments before autoregressive generation. |
|
|
| --- |
|
|
| # Related Work |
|
|
| | Area | Difference from J-Space | |
| |------|--------------------------| |
| | **Anthropic – J-space / Global Workspace** | Maps the latent workspace for interpretability. J-Space actively trains and structures it. | |
| | **Chain-of-Thought** | Uses intermediate output tokens. J-Space operates directly in latent representations. | |
| | **Recurrent Memory Networks** | Maintain recurrent hidden states without an explicit structured workspace. | |
| | **Activation Steering** | Modifies isolated activations. J-Space builds a persistent internal reasoning structure. | |
|
|
| --- |
|
|
| # License |
|
|
| Apache License 2.0 |
|
|
| --- |
|
|
| # Citation |
|
|
| ```bibtex |
| @software{reis2026jspace, |
| author = {Miguel Penha Reis}, |
| title = {J-Space Deliberation Engine}, |
| year = {2026}, |
| url = {https://huggingface.co/guell00/J-Space-Deliberation}, |
| license = {Apache-2.0} |
| } |
| ``` |
|
|
| --- |
|
|
| # Author |
|
|
| **Miguel Penha Reis** |
|
|
| Creator of the **J-Space Deliberation Engine**, an experimental latent reasoning architecture for large language models built upon **Gemma 4 E4B-it**. |
|
|
| --- |
|
|
| ## History |
|
|
| - **Project created:** July 26, 2026 |
| - **Architecture:** J-Space Deliberation Engine |
| - **Base model:** Gemma 4 E4B-it |
| - **Method:** Structured Latent Deliberation + Jacobian Lens |
|
|
| --- |
|
|
| ## Scientific Note |
|
|
| This repository presents an **independent experimental research project**. |
|
|
| Claims of originality refer specifically to the implementation provided here and to the proposed method of **actively organizing latent representations into non-overlapping reasoning slots** prior to token generation. |
|
|
| The project should be understood as an experimental exploration of structured latent reasoning rather than a claim about the capabilities or internal mechanisms of language models in general. |
