| --- |
| license: cc-by-nc-nd-4.0 |
| tags: |
| - non-markovian |
| - relevance |
| - coherence |
| - goal-directed-generation |
| - bidirectional-lstm |
| - transformer |
| - language-model-evaluation |
| - ai-safety |
| - empirical-proof |
| language: |
| - en |
| --- |
| |
| # The Markovian Ceiling: Coherence ≠ Relevance |
|
|
| Empirical proof that relevance (goal-directed generation) requires non-Markovian conditioning. |
|
|
| ## The Finding |
|
|
| A clean 0% vs 100% split: |
|
|
| - **Unidirectional LSTM (Markovian):** 0.0% relevance — collapses to one repeated sentence |
| - **Bidirectional Seq2Seq LSTM:** 100.0% relevance — correctly composes unseen goals |
| - **Multidirectional Transformer:** 100.0% relevance — same |
|
|
| ## What This Proves |
|
|
| Coherence and relevance are fundamentally different dimensions of text quality. A Markovian model achieves fluency but cannot steer toward a goal. Non-Markovian conditioning (bidirectional or multidirectional context) is the minimum architectural requirement for relevance. |
|
|
| ## Paper |
|
|
| > Chetan Sharma. *Relevance Requires Non‑Markovian Conditioning: Empirical Proof that Goal‑Directed Generation Demands Bidirectional Context.* Zenodo, 15 May 2026. |
|
|
| - **DOI:** [10.5281/zenodo.20201889](https://doi.org/10.5281/zenodo.20201889) |
| - **Blog:** [antimarkovian-eureka.blogspot.com](https://antimarkovian-eureka.blogspot.com) |
|
|
| ## Citation |
|
|
|
|
| ```bibtex |
| |
| |
| @misc{sharma2026relevance, |
| author = {Chetan Sharma}, |
| title = {Relevance Requires Non‑Markovian Conditioning: Empirical Proof that Goal‑Directed Generation Demands Bidirectional Context}, |
| year = {2026}, |
| month = may, |
| doi = {10.5281/zenodo.20201889}, |
| publisher = {Zenodo}, |
| url = {https://zenodo.org/records/20201889} |
| } |
| |
| Author |
| Chetan Sharma — Independent Researcher, Kolkata, India. |
| |
| Blog: antimarkovian-eureka.blogspot.com |
| |
| Zenodo: 10.5281/zenodo.20201889 |
| |
| License |
| This model card and the associated paper are licensed under cc-by-nc-nd-4.0 (Creative Commons Attribution Non Commercial No Derivatives 4.0 International) |
| ``` |
|
|
| ``` |
| 5. Commit. |
| |
| ### Part B: Update `Transcrypto/yesterday-json` README |
| |
| Add this section at the bottom (above License): |
| |
| markdown |
| ## Related Research |
| |
| **The Markovian Ceiling (May 2026):** A clean empirical proof that coherence and relevance are architecturally distinct. Unidirectional Markovian models achieve 0% relevance on novel compositional goals, while bidirectional and multidirectional models achieve 100%. This finding provides the theoretical foundation for the yesterday.json architecture — episodic memory alone is insufficient; non-Markovian conditioning is the minimum requirement for goal-directed AI. |
| |
| - **Paper:** [Relevance Requires Non‑Markovian Conditioning](https://doi.org/10.5281/zenodo.20201889) |
| - **Blog:** [antimarkovian-eureka.blogspot.com](https://antimarkovian-eureka.blogspot.com) |
| ``` |
| ``` |
| Part C: Result |
| You'll have: |
| |
| Asset Link |
| Blog antimarkovian-eureka.blogspot.com |
| Zenodo DOI 10.5281/zenodo.20201889 |
| HF Model Repo Transcrypto/markovian-ceiling-relevance |
| HF (updated) Transcrypto/yesterday-json (with cross-link) |
| All four cross-linked, all timestamped, all public. |
| ``` |
