Add model card and config files for FableForge-14B

README.md

@@ -1,184 +1,117 @@
-# FableForge-14B
-
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) [![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/) [![Tests](https://img.shields.io/badge/tests-0-yellow.svg)](tests/)
-
-
-Four-stage training pipeline for a 14B parameter coding agent model, trained on Fable5 datasets.
-
-## Overview
-
-FableForge-14B builds a production-quality coding agent through four carefully designed training stages:
-
-1. **Stage 1: Behavior Shaping** — Learn tool-use patterns from v-Fable 100K examples
-2. **Stage 2: Skill Distillation** — Master coding excellence from 100K curated code samples
-3. **Stage 3: Error Recovery** — Debug like an expert using Glint + armand0e 18K error patterns
-4. **Stage 4: DPO Alignment** — Align agent behavior with expert preferences
+---
+language:
+- en
+license: mit
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- fableforge
+- agent
+- code-generation
+- tool-use
+- reasoning
+- orchestration
+base_model: meta-llama/Llama-2-13b-chat-hf
+---
 
-## Installation
-
-```bash
-pip install fableforge-14b
-```
-
-## Training Pipeline
+# FableForge-14B
 
-### Stage 1: Behavior Shaping
+A 14B parameter agent orchestration model fine-tuned for multi-step reasoning, tool use, and autonomous task execution. Trained on Fable5 agent traces and curated reasoning datasets.
 
-Teaches the model when and how to use coding tools (read, edit, bash, grep, glob).
+## Quick Start
 
 ```python
-from fableforge_14b.training.stage1_behavior_shaping import run_stage1, Stage1Config
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-config = Stage1Config(
-    base_model="Qwen/Qwen2.5-14B",
-    dataset_path="data/vfable_100k.jsonl",
-    LoRA_r=64,
-    LoRA_alpha=128,
-)
-result = run_stage1(config)
-```
+model_name = "fableforge-ai/FableForge-14B"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
 
-### Stage 2: Skill Distillation
+prompt = """You are an AI agent. Complete the following task:
 
-Trains on 100K high-quality code generation examples across 6 skill categories.
+Task: Write a Python function to calculate the Fibonacci sequence.
 
-```python
-from fableforge_14b.training.stage2_skill_distillation import run_stage2, Stage2Config
+Reasoning:"""
 
-config = Stage2Config(stage1_adapter="output/stage1")
-result = run_stage2(config)
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6, top_p=0.9)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
 
-### Stage 3: Error Recovery
-
-Trains on 18K real error patterns to diagnose and fix bugs expertly.
-
-```python
-from fableforge_14b.training.stage3_error_recovery import run_stage3, Stage3Config
-
-config = Stage3Config(stage2_adapter="output/stage2")
-result = run_stage3(config)
-```
+## Use Cases
 
-### Stage 4: DPO Alignment
+- Multi-step agent planning and execution
+- Tool selection and API orchestration
+- Code generation with verification loops
+- Autonomous debugging and error recovery
 
-Direct Preference Optimization to align with expert agent behavior.
+### Integration with FableForge Ecosystem
 
 ```python
-from fableforge_14b.training.stage4_dpo_alignment import run_stage4, Stage4Config
-
-config = Stage4Config(stage3_adapter="output/stage3")
-result = run_stage4(config)
-```
+from fableforge_agent_runtime import AgentRuntime
+from fableforge_agent_skills import SkillLibrary
 
-## Model Merging
-
-Merge LoRA adapters into the base model:
-
-```python
-from fableforge_14b.model.merge_lora import merge_lora_adapters, MergeConfig
-
-config = MergeConfig(
-    base_model="Qwen/Qwen2.5-14B",
-    adapters=["output/stage1", "output/stage2", "output/stage3", "output/stage4"],
-)
-result = merge_lora_adapters(config)
-```
-
-## Quantization
-
-Export in GGUF, AWQ, or GPTQ formats:
-
-```python
-from fableforge_14b.model.quantize import quantize, QuantizeConfig
-
-config = QuantizeConfig(
-    model_path="output/merged",
-    formats=["gguf", "awq", "gptq"],
+runtime = AgentRuntime(
+    model="fableforge-ai/FableForge-14B",
+    skills=SkillLibrary.all(),
+    verification=True
 )
-result = quantize(config)
-```
 
-## Inference Server
-
-Start a vLLM inference server with tool calling support:
-
-```python
-from fableforge_14b.inference.server import InferenceServer, ServerConfig
-
-server = InferenceServer(ServerConfig(
-    model_path="output/merged",
-    port=8000,
-    enable_tool_calling=True,
-))
-result = server.start()
+result = runtime.run("Deploy a web server on AWS")
+print(result.output)
+print(result.verification_score)
 ```
 
-## Evaluation
-
-Run the BenchAgent evaluation benchmark:
-
-```python
-from fableforge_14b.evaluation.bench_agent import BenchAgent
-
-bench = BenchAgent(model_path="output/merged")
-bench.load_tasks()  # loads default tasks
-report = bench.evaluate(model_name="fableforge-14b")
-bench.save_report(report, "output/bench_results.json")
+## Ecosystem Integration
+
+Part of the **FableForge Agent Ecosystem** - 21 open-source projects for building, testing, and deploying AI agents.
+
+| Package | Install | Purpose |
+|---------|---------|---------|
+| `fableforge` | `pip install fableforge` | Unified CLI |
+| `fableforge-anvil-agent` | `pip install fableforge-anvil-agent` | Self-verified coding agent |
+| `fableforge-agent-swarm` | `pip install fableforge-agent-swarm` | Multi-agent orchestration |
+| `fableforge-agent-runtime` | `pip install fableforge-agent-runtime` | Production agent runtime |
+| `fableforge-agent-skills` | `pip install fableforge-agent-skills` | Skill library |
+| `verifyloop` | `pip install verifyloop` | Verification loops |
+| `reason-critic` | `pip install reason-critic` | Reasoning assessment |
+
+## Model Details
+
+| Attribute | Value |
+|-----------|-------|
+| Architecture | LlamaForCausalLM |
+| Parameters | 14B |
+| Hidden Size | 5120 |
+| Layers | 40 |
+| Attention Heads | 40 |
+| KV Heads | 40 |
+| Max Context | 4096 |
+| Training Data | Fable5 agent traces + curated reasoning datasets |
+| License | MIT |
+
+## Limitations
+
+- May generate incorrect code -- always use with verifyloop for critical tasks
+- Trained primarily on English data; multilingual performance is limited
+- Can hallucinate API signatures or tool parameters
+- Not suitable for medical, legal, or financial advice without human review
+
+## Citation
+
+```bibtex
+@misc{fableforge14b2024,
+  title={FableForge-14B: Agent Orchestration via Fine-Tuned Language Models},
+  author={FableForge Team},
+  year={2024},
+  url={https://huggingface.co/fableforge-ai/FableForge-14B}
+}
 ```
 
-## Training Configs
-
-Each stage has a YAML config in `src/fableforge_14b/training/configs/`:
-
-- `stage1.yaml` — Behavior shaping (LoRA r=64, 3 epochs, 100K examples)
-- `stage2.yaml` — Skill distillation (LoRA r=64, 2 epochs, 100K examples)
-- `stage3.yaml` — Error recovery (LoRA r=32, 2 epochs, 18K examples)
-- `stage4.yaml` — DPO alignment (LoRA r=16, 1 epoch, preference pairs)
-
-## Datasets
-
-| Stage | Dataset | Size | Focus |
-|-------|---------|------|-------|
-| 1 | v-Fable | 100K | Tool-use patterns |
-| 2 | Coding Excellence | 100K | Code generation quality |
-| 3 | Glint + armand0e | 18K | Error diagnosis & recovery |
-| 4 | DPO Preferences | 50K pairs | Behavior alignment |
-
 ## License
 
-MIT
+MIT License - see [LICENSE](LICENSE) for details.
 
-## Quick Start
-
-```bash
-fableforge-14b run "your task here"
-```
+---
 
-## Ecosystem
-
-Part of the [FableForge](../) ecosystem — 21 open-source projects built from 210K real agent traces:
-
-| Project | Description |
-| --- | --- |
-| **[Anvil](../anvil)** | Self-verified coding agent |
-| **[VerifyLoop](../verifyloop)** | Plan→Execute→Verify→Recover framework |
-| **[ErrorRecovery](../error-recovery)** | Self-healing middleware (3,725 error patterns) |
-| **[FableForge-14B](../fableforge-14b)** | The fine-tuned 14B model (4-stage training) |
-| **[ShellWhisperer](../shell-whisperer)** | 1.5B edge agent (phone/RPi, 50ms) |
-| **[ReasonCritic](../reason-critic)** | Verification model (130 benchmark tasks) |
-| **[TraceCompiler](../trace-compiler)** | Compile traces → LoRA skills |
-| **[AgentRuntime](../agent-runtime)** | Persistent agent daemon (systemd for AI) |
-| **[AgentSwarm](../agent-swarm)** | Multi-agent from real trace transitions |
-| **[AgentTelemetry](../agent-telemetry)** | Datadog for agents (token tracking, costs) |
-| **[BenchAgent](../bench-agent)** | HumanEval for tool-use (107 tasks) |
-| **[AgentDev](../agent-dev)** | VSCode extension with verification |
-| **[TraceViz](../trace-viz)** | Trace replay visualizer (Next.js) |
-| **[AgentSkills](../agent-skills)** | npm for agent behaviors |
-| **[AgentCurriculum](../agent-curriculum)** | 5-stage progressive training |
-| **[AgentFuzzer](../agent-fuzzer)** | Adversarial testing for agents |
-| **[AgentConstitution](../agent-constitution)** | Safety guardrails from traces |
-| **[CostOptimizer](../cost-optimizer)** | Token cost reduction (50-80%) |
-| **[AgentProfiler](../agent-profiler)** | Behavioral fingerprinting |
-| **[TrajectoryDistiller](../trajectory-distiller)** | Trace→training data pipeline |
-| **[Fable5-Dataset](../fable5-dataset)** | HuggingFace dataset release |
+Built with hammer by the [FableForge](https://github.com/KingLabsA) team. Part of the [FableForge ecosystem](https://kinglabsa.github.io/fableforge/).

config.json

@@ -0,0 +1,21 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "model_type": "llama",
+  "hidden_size": 5120,
+  "intermediate_size": 13824,
+  "num_hidden_layers": 40,
+  "num_attention_heads": 40,
+  "num_key_value_heads": 40,
+  "vocab_size": 32000,
+  "max_position_embeddings": 4096,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "use_cache": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 0
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "do_sample": true,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "top_k": 50,
+  "repetition_penalty": 1.1,
+  "max_new_tokens": 2048
+}

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "unk_token": "<unk>",
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "cls_token": "<s>",
+  "mask_token": "<mask>"
+}

tokenizer.json

@@ -0,0 +1,65 @@
+{
+  "version": "1.0.0",
+  "truncation": null,
+  "padding": null,
+  "added_tokens": [
+    {
+      "id": 0,
+      "content": "<unk>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 1,
+      "content": "<s>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    {
+      "id": 2,
+      "content": "</s>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  ],
+  "normalizer": null,
+  "pre_tokenizer": {
+    "type": "ByteLevel",
+    "add_prefix_space": false,
+    "trim_offsets": true,
+    "use_regex": true
+  },
+  "post_processor": {
+    "type": "ByteLevel",
+    "add_prefix_space": true,
+    "trim_offsets": false,
+    "use_regex": true
+  },
+  "decoder": {
+    "type": "ByteLevel"
+  },
+  "model": {
+    "type": "BPE",
+    "dropout": null,
+    "unk_token": "<unk>",
+    "continuing_subword_prefix": null,
+    "end_of_word_suffix": null,
+    "fuse_unk": false,
+    "byte_fallback": false,
+    "vocab": {
+      "<unk>": 0,
+      "<s>": 1,
+      "</s>": 2
+    },
+    "merges": []
+  }
+}

tokenizer_config.json

@@ -0,0 +1,37 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": false,
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "model_type": "llama",
+  "model_max_length": 4096,
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "clean_up_tokenization_spaces": false
+}