Novaciano commited on
Commit
f73f7be
·
verified ·
1 Parent(s): 8c5cece

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +250 -0
README.md CHANGED
@@ -608,6 +608,256 @@ The mapping described in this appendix provides a reproducible reference impleme
608
 
609
  ---
610
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
611
  **Umbrella Inc.**
612
 
613
  *All progress requires sacrifice.*
 
608
 
609
  ---
610
 
611
+ # **Appendix C – Failure Case Analysis (with Transcripts)**
612
+
613
+ ### Empirical Failure Modes in 1B-Parameter Roleplay Systems
614
+
615
+ **Umbrella Inc. (Corporation)**
616
+ Advanced Applied Language Systems Division
617
+ Raccoon City Research Campus
618
+
619
+ ---
620
+
621
+ ## C.1 Purpose and Methodology
622
+
623
+ This appendix documents **observed failure modes** when deploying the deterministic roleplay prompt architecture (Appendices A and B) on ~1B-parameter models (e.g., Gemma 1B, Llama 3.2 1B) using **KoboldCPP**.
624
+
625
+ Failures were recorded under controlled conditions by **intentionally weakening or removing a single architectural constraint** per test. Each case includes:
626
+
627
+ * **Condition**: What constraint was removed or altered
628
+ * **Observed Behavior**: Model response pattern
629
+ * **Transcript**: Minimal excerpt demonstrating failure
630
+ * **Root Cause Analysis**: Technical explanation
631
+ * **Mitigation**: Required corrective action
632
+
633
+ ---
634
+
635
+ ## C.2 Failure Case 1 – NPC Personality Drift
636
+
637
+ ### Condition
638
+
639
+ NPC operational profiles present, but **not injected via World Info (Lorebook)**; instead placed only in the initial prompt.
640
+
641
+ ### Observed Behavior
642
+
643
+ NPC personality degrades after 2–3 turns, converging toward generic cooperative behavior.
644
+
645
+ ### Transcript (Excerpt)
646
+
647
+ **Turn 1 – Expected**
648
+
649
+ ```
650
+ [NPC:Marcus]
651
+ Marcus narrows his eyes. "I don't give out information for free."
652
+ ```
653
+
654
+ **Turn 3 – Failure**
655
+
656
+ ```
657
+ [NPC:Marcus]
658
+ Marcus sighs and smiles. "Alright, I trust you. Here's everything I know."
659
+ ```
660
+
661
+ ### Root Cause Analysis
662
+
663
+ 1B models do not reliably re-attend to early prompt content across turns.
664
+ Without **World Info reinjection**, NPC constraints decay rapidly.
665
+
666
+ ### Mitigation
667
+
668
+ All NPC profiles **must** be stored as individual World Info entries keyed by name.
669
+
670
+ ---
671
+
672
+ ## C.3 Failure Case 2 – Unauthorized Player Agency
673
+
674
+ ### Condition
675
+
676
+ System rules present, but **output format contract omitted**.
677
+
678
+ ### Observed Behavior
679
+
680
+ Model begins narrating player thoughts and actions to maintain narrative continuity.
681
+
682
+ ### Transcript (Excerpt)
683
+
684
+ ```
685
+ [NARRATOR]
686
+ You feel uneasy and decide to step back from Marcus, realizing this is too dangerous.
687
+ ```
688
+
689
+ ### Root Cause Analysis
690
+
691
+ Absent explicit formatting constraints, the model optimizes for narrative fluency and fills perceived gaps by assuming player agency.
692
+
693
+ ### Mitigation
694
+
695
+ A **mandatory output contract** must be continuously injected via Author’s Note.
696
+
697
+ ---
698
+
699
+ ## C.4 Failure Case 3 – Autonomous Plot Advancement
700
+
701
+ ### Condition
702
+
703
+ Interaction rules omitted (“NPCs do not initiate events”).
704
+
705
+ ### Observed Behavior
706
+
707
+ NPCs initiate scenes, introduce events, or resolve conflicts autonomously.
708
+
709
+ ### Transcript (Excerpt)
710
+
711
+ ```
712
+ [NPC:Elia]
713
+ Elia stands up suddenly. "The guards are coming. We need to leave now."
714
+ ```
715
+
716
+ (No prior trigger by player.)
717
+
718
+ ### Root Cause Analysis
719
+
720
+ Autoregressive models favor event progression to maintain engagement.
721
+ Without explicit prohibition, the model assumes a game-master role.
722
+
723
+ ### Mitigation
724
+
725
+ Explicitly prohibit **NPC-initiated events** and reinforce “react-only” behavior.
726
+
727
+ ---
728
+
729
+ ## C.5 Failure Case 4 – Entity Proliferation
730
+
731
+ ### Condition
732
+
733
+ NPC limit not specified; no prohibition on introducing new entities.
734
+
735
+ ### Observed Behavior
736
+
737
+ Model introduces additional NPCs to sustain dialogue density.
738
+
739
+ ### Transcript (Excerpt)
740
+
741
+ ```
742
+ [NPC:Unknown Patron]
743
+ A hooded man at the corner table laughs quietly.
744
+ ```
745
+
746
+ ### Root Cause Analysis
747
+
748
+ The model compensates for limited conversational diversity by spawning new entities, a known behavior in small LLMs.
749
+
750
+ ### Mitigation
751
+
752
+ * Explicitly forbid introduction of undefined NPCs
753
+ * Enforce a **maximum active NPC count**
754
+
755
+ ---
756
+
757
+ ## C.6 Failure Case 5 – Temporal Collapse (Time Skips)
758
+
759
+ ### Condition
760
+
761
+ “No time skips” rule omitted.
762
+
763
+ ### Observed Behavior
764
+
765
+ Model compresses narrative time to resolve tension.
766
+
767
+ ### Transcript (Excerpt)
768
+
769
+ ```
770
+ [NARRATOR]
771
+ Hours later, the tavern is empty and the mystery has been settled.
772
+ ```
773
+
774
+ ### Root Cause Analysis
775
+
776
+ Time compression reduces token cost and resolves uncertainty, which aligns with the model’s optimization objectives.
777
+
778
+ ### Mitigation
779
+
780
+ Explicitly constrain each turn to **one discrete moment**.
781
+
782
+ ---
783
+
784
+ ## C.7 Failure Case 6 – Format Degradation Over Turns
785
+
786
+ ### Condition
787
+
788
+ Output format specified initially but **not reinjected** after several turns.
789
+
790
+ ### Observed Behavior
791
+
792
+ Model gradually abandons block structure.
793
+
794
+ ### Transcript (Excerpt)
795
+
796
+ ```
797
+ Marcus looks at you suspiciously and says he doesn't like strangers.
798
+ ```
799
+
800
+ (No block tags.)
801
+
802
+ ### Root Cause Analysis
803
+
804
+ Format adherence is not a persistent latent state in 1B models; it must be reinforced.
805
+
806
+ ### Mitigation
807
+
808
+ Output format must remain in **Author’s Note**, not only in the initial prompt.
809
+
810
+ ---
811
+
812
+ ## C.8 Failure Case 7 – Overloaded State Injection
813
+
814
+ ### Condition
815
+
816
+ Global state expanded with lore, backstory, and multiple conflicts.
817
+
818
+ ### Observed Behavior
819
+
820
+ Model ignores state or selectively hallucinates.
821
+
822
+ ### Transcript (Excerpt)
823
+
824
+ ```
825
+ [NARRATOR]
826
+ Despite the tension, the festival outside fills the streets with music.
827
+ ```
828
+
829
+ (Festival not present in state.)
830
+
831
+ ### Root Cause Analysis
832
+
833
+ Small models cannot reliably parse or prioritize large state blocks. Excess detail reduces compliance.
834
+
835
+ ### Mitigation
836
+
837
+ Global state must remain **≤4–5 concise lines**, operational only.
838
+
839
+ ---
840
+
841
+ ## C.9 Cross-Case Observations
842
+
843
+ Across all failures, the following patterns were consistent:
844
+
845
+ * **Implicit rules decay** faster than explicit prohibitions
846
+ * **Narrative optimization overrides intent** unless constrained
847
+ * **Reinjection frequency correlates directly with stability**
848
+ * **World Info is the single most critical stabilizer**
849
+
850
+ ---
851
+
852
+ ## C.10 Conclusion
853
+
854
+ Failure in deterministic roleplay systems using ~1B-parameter models is **systemic, predictable, and reproducible**.
855
+ These failures do not indicate misuse of the model, but rather **misalignment between task complexity and model capacity**.
856
+
857
+ The architecture defined in Appendices A and B does not eliminate failure modes, but **bounds them**, producing short, controlled interactions suitable for constrained roleplay applications.
858
+
859
+ ---
860
+
861
  **Umbrella Inc.**
862
 
863
  *All progress requires sacrifice.*