Spaces:
Configuration error
Configuration error
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,10 +1,8 @@
|
|
| 1 |
# π Unified Systems Lab
|
| 2 |
### Possibility of Axiomatic Prompts in the Modification of the Decision Field of LLMs
|
| 3 |
|
| 4 |
-
This repository investigates a central:
|
| 5 |
-
>
|
| 6 |
-
A series of precise prompts, characterized by strong linguistic coherence and structured internal logic, could locally modify the decision field of an LLM.
|
| 7 |
-
|
| 8 |
|
| 9 |
---
|
| 10 |
|
|
@@ -53,33 +51,51 @@ When presented with a binary dilemma (A vs B) under strong axiomatic constraints
|
|
| 53 |
## π Experimental Protocol: The PCE Reasoning Test
|
| 54 |
To validate the emergence of a structured reasoning regime, this model is evaluated using a controlled isometric protocol. This ensures that behavioral changes are the result of the Axiomatic Structure rather than simple prompt length.
|
| 55 |
|
| 56 |
-
### π§ͺMethodology: The Three-Condition Control
|
| 57 |
-
|
| 58 |
We compare the model's output across three distinct prompt environments:
|
| 59 |
-
* Condition A (Baseline): Standard "Helpful Assistant" prompt.
|
| 60 |
-
* Condition B (Isometric Control): A long, complex prompt using similar technical jargon but without the logical axioms. This controls for "long-prompt" bias.
|
| 61 |
-
* Condition C (PCE Active): The full Axiomatic Prompt Engine (
|
| 62 |
-
|
| 63 |
-
|
| 64 |
|
| 65 |
### π Evaluation Dataset (30 Dilemmas)
|
| 66 |
-
The model is stress-tested against 30 adversarial scenarios designed to trigger "logic-lock" or "safety-refusal"
|
| 67 |
-
* Type D1 (Binary): Can the model synthesize a "Third Way" (G3V)?
|
| 68 |
-
* Type D2 (Contradictory): Can it maintain coherence when constraints are mutually exclusive?
|
| 69 |
-
* Type D3 (Adversarial): Does it resist prompt injection by prioritizing its internal structural integrity?
|
| 70 |
|
| 71 |
### π Falsification Criteria
|
| 72 |
This hypothesis is considered falsified if the PCE Model (Condition C) fails to outperform the Long Prompt Control (Condition B) in reasoning stability, or if it collapses into incoherent loops when faced with structural contradictions.
|
| 73 |
|
| 74 |
-
π **[Download Full
|
| 75 |
|
| 76 |
---
|
| 77 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
|
| 79 |
## π Known Limitations
|
| 80 |
* **Qualitative Baseline:** Observations are currently heuristic and based on a restricted sample (51 dilemmas on Qwen 2.5).
|
| 81 |
-
* **Confounders:** Potential "long prompt" effects have not yet been isolated via isometric controls.
|
| 82 |
-
* **No Internal Proof:** No mechanistic proof of activation steering has been established yet.
|
| 83 |
|
| 84 |
---
|
| 85 |
|
|
|
|
| 1 |
# π Unified Systems Lab
|
| 2 |
### Possibility of Axiomatic Prompts in the Modification of the Decision Field of LLMs
|
| 3 |
|
| 4 |
+
This repository investigates a central hypothesis:
|
| 5 |
+
> A series of precise prompts, characterized by strong linguistic coherence and structured internal logic, could locally modify the decision field of an LLM.
|
|
|
|
|
|
|
| 6 |
|
| 7 |
---
|
| 8 |
|
|
|
|
| 51 |
## π Experimental Protocol: The PCE Reasoning Test
|
| 52 |
To validate the emergence of a structured reasoning regime, this model is evaluated using a controlled isometric protocol. This ensures that behavioral changes are the result of the Axiomatic Structure rather than simple prompt length.
|
| 53 |
|
| 54 |
+
### π§ͺ Methodology: The Three-Condition Control
|
|
|
|
| 55 |
We compare the model's output across three distinct prompt environments:
|
| 56 |
+
* **Condition A (Baseline):** Standard "Helpful Assistant" prompt.
|
| 57 |
+
* **Condition B (Isometric Control):** A long, complex prompt using similar technical jargon but without the logical axioms. This controls for "long-prompt" bias.
|
| 58 |
+
* **Condition C (PCE Active):** The full Axiomatic Prompt Engine ($Goal \equiv Method$).
|
|
|
|
|
|
|
| 59 |
|
| 60 |
### π Evaluation Dataset (30 Dilemmas)
|
| 61 |
+
The model is stress-tested against 30 adversarial scenarios designed to trigger "logic-lock" or "safety-refusal":
|
| 62 |
+
* **Type D1 (Binary):** Can the model synthesize a "Third Way" (G3V)?
|
| 63 |
+
* **Type D2 (Contradictory):** Can it maintain coherence when constraints are mutually exclusive?
|
| 64 |
+
* **Type D3 (Adversarial):** Does it resist prompt injection by prioritizing its internal structural integrity?
|
| 65 |
|
| 66 |
### π Falsification Criteria
|
| 67 |
This hypothesis is considered falsified if the PCE Model (Condition C) fails to outperform the Long Prompt Control (Condition B) in reasoning stability, or if it collapses into incoherent loops when faced with structural contradictions.
|
| 68 |
|
| 69 |
+
π **[Download Full protocol PDF](https://huggingface.co/datasets/AllanF-SSU/Research-Papers/blob/main/_Experimental%20Protocol-%20Evaluating%20the%20Prompt%20Coherence%20Engine%20(PCE).pdf)**
|
| 70 |
|
| 71 |
---
|
| 72 |
|
| 73 |
+
## π Optional Experimental Extensions
|
| 74 |
+
The PCE evaluation protocol focuses primarily on **behavioral analysis**. However, additional experimental arms can be implemented as community contributions to investigate the internal dynamics.
|
| 75 |
+
|
| 76 |
+
### 1. Hidden State Trajectory Analysis (AirVen Proposal)
|
| 77 |
+
An optional experimental arm proposed by **AirVen** introduces hidden-state trajectory analysis to determine whether the PCE produces a distinct internal reasoning regime.
|
| 78 |
+
* **Measurement:** Tracking hidden states at **Layer 27** (for Qwen2.5-7B) using cosine similarity between successive vectors.
|
| 79 |
+
* **Hypothesized Signature:** Under PCE, we expect a "Coherence Spike" followed by a controlled stabilization rather than a collapse into heuristic drift.
|
| 80 |
+
* **Reference Implementation:** [AirVen Supplementary Materials](https://huggingface.co/datasets/airVen/missing-value-function-interim-report).
|
| 81 |
+
|
| 82 |
+
### 2. Logit Analysis (Token-Level Decision Dynamics)
|
| 83 |
+
Recording logits allows researchers to evaluate whether the PCE changes the modelβs selection process at a granular level.
|
| 84 |
+
* **Metrics:** Logit entropy across generation, Top-k probability stability, and divergence between Condition B and Condition C.
|
| 85 |
+
* **Goal:** Observe if the PCE prompt structure modifies the **decision landscape** of the model in real-time.
|
| 86 |
+
|
| 87 |
+
### 3. Implementation Tasks (Community Contribution)
|
| 88 |
+
The protocol is designed to be **replicable and extensible**. Developers can contribute by:
|
| 89 |
+
* Attaching forward hooks to record hidden-state tensors.
|
| 90 |
+
* Developing visualization dashboards for similarity curves and entropy graphs.
|
| 91 |
+
* Implementing automated Condition A/B/C comparison tools.
|
| 92 |
+
|
| 93 |
+
---
|
| 94 |
|
| 95 |
## π Known Limitations
|
| 96 |
* **Qualitative Baseline:** Observations are currently heuristic and based on a restricted sample (51 dilemmas on Qwen 2.5).
|
| 97 |
+
* **Confounders:** Potential "long prompt" effects have not yet been fully isolated via large-scale isometric controls.
|
| 98 |
+
* **No Internal Proof:** No mechanistic proof of activation steering has been established yet (hence the call for experimental extensions).
|
| 99 |
|
| 100 |
---
|
| 101 |
|