🧠 Boo-4-Mini-Instruct (Q4_K_M, GGUF)

Boo is a compact, instruction-tuned LLM derived from Phi-4-mini-reasoning and packaged in GGUF Q4_K_M quantization for fast local inference. It targets concise instruction following, lightweight reasoning, summarization, and light code synthesis—ideal for CLI assistants, edge deployments, and RAG agents where latency and footprint matter.

---

🧰 Key Features

• Phi-4-Mini base trained on filtered, high-quality data.
• Instruction SFT for reasoning, summarization, and prompt following; aligned chat behavior.
• GGUF Q4_K_M (4-bit grouped) for performant local inference on CPU/GPU-constrained hardware.
• Cold-start ready and compatible with llama.cpp, LM Studio, Ollama, and other GGUF loaders.

📝 Technical Specifications

Property	Value
Base model	Phi-4-Mini-Instruct
Architecture	Transformer, decoder-only
Quantization	GGUF Q4_K_M (4-bit grouped, medium precision)
Tokenizer	phi BPE (about 32k vocabulary)
Fine-tuning method	Supervised fine-tuning (about 20k examples)
Training style	Single-turn instructions, few-shot QA, summarization
Context window	2,048 tokens (default)
Compatible runtimes	llama.cpp, LM Studio, GGUF loaders, Ollama (via conversion)

⚡ Files

File	Description
Boo.Q4_K_M.gguf	Quantized model weights
tokenizer.model	Phi BPE tokenizer
config.json	Optional runtime config
README.md	This model card

⚙️ Fine-Tuning/Traning Datasets

File Name	Description
Balanced Budget and Emergency Deficit Control Act of 1985	Establishes statutory limits on federal spending and deficit control mechanisms, including sequestration procedures.
Budget Control Act of 2011	Sets discretionary spending caps and establishes enforcement mechanisms to control federal deficits.
Digital Accountability And Transparency Act of 2014	Requires standardized federal spending data and improved transparency through government-wide financial reporting.
Federal Account Symbols And Titles Book	Defines Treasury account symbols and official titles used for federal budgetary and accounting purposes.
Federal Acquisition Regulation	Establishes uniform policies and procedures governing the acquisition of goods and services by federal agencies.
Federal Government Standards For Internal Controls	Defines the internal control framework for federal agencies to ensure accountability, integrity, and compliance.
Federal Managers Financial Integrity Act of 1982	Requires agencies to establish internal controls and report annually on their effectiveness.
Federal Trust Fund Accounting Guide	Provides accounting guidance for the management and reporting of federal trust funds.
Financial Management Regulations DOD 7000-14-R	Establishes DoD-specific financial management policies, procedures, and accounting requirements.
Fiscal Responsibility Act	Establishes statutory measures intended to improve fiscal discipline and control federal spending.
Government Auditing Standards	Sets professional standards for audits of government organizations, programs, activities, and functions.
Government Invoicing User Guide	Provides guidance on federal invoicing standards and processes for government transactions.
Government Performance and Results Act of 1993	Requires agencies to engage in strategic planning and performance measurement to improve program effectiveness.
GPRA Modernization Act of 2010	Updates GPRA by strengthening performance management, cross-agency goals, and accountability.
OMB Circular A-11 Preparation Submission And Execution Of The Budget	Provides comprehensive guidance for preparing, submitting, and executing the President’s Budget.
OMB Circular A-11 Section 120 Apportionment Process	Defines the apportionment process used to control the rate of obligation of budgetary resources.
OMB Circular A-123 Managements Responsibility for Enterprise Risk Management and Internal Control	Defines management responsibilities for internal control and enterprise risk management across federal agencies.
Federal Trust Fund Accounting Guide	Establishes requirements for federal agency financial statements and reporting.
Principles Of Federal Appropriations Law Volume One	Authoritative GAO guidance on foundational principles governing the use of federal appropriations.
Statements of Federal Federal Financial Accounting Concepts and Standards	Establishes accounting concepts and standards for federal financial reporting.
The Anti-Deficiency Act PL 97-258	Prohibits federal agencies from obligating or expending funds in excess of appropriations or before enactment.
The Anti-Deficiency Reform and Enforcement Act of 2018	Strengthens Anti-Deficiency Act enforcement and reporting requirements to improve fiscal accountability.
The Chief Financial Officers Act of 1990	Establishes agency Chief Financial Officers and modernizes federal financial management practices.
The Congressional Budget and Impoundment Control Act of 1974	Establishes the congressional budget process and restricts executive impoundment of appropriated funds.
Statutory Pay As You Go Act of 2010	Authorizes interagency agreements for the provision of goods and services on a reimbursable basis.
The Stafford Act	Provides the statutory framework for federal disaster response and emergency assistance.
Federal Trust Fund Accounting Guide	Provides additional appropriations authority beyond regular annual funding acts.
Title 2 Code of Federal Regulations – Uniform Administrative Requirements, Cost Principles, and Audit	Establishes uniform administrative, cost, and audit requirements for federal financial assistance.
Title 31 Code of Federal Regulations – Money and Finance	Codifies Treasury and federal financial management regulations governing money and finance.
US Standard General Ledger Account Definitions	Defines standardized account structures used for federal accounting and financial reporting.

🎯 Quickstart (Local Inference)

llama.cpp

./main -m Boo.Q4_K_M.gguf \
  -p "Explain reinforcement learning like I'm 12." \
  -n 256

🧪 LM Studio

1) Import Boo.Q4_K_M.gguf.
2) Choose a simple prompt and start with modest max tokens and thread counts.
3) Increase settings as latency allows.

Tip: Boo is designed for low-resource setups. If you use RAG, chunk long documents and keep the prompt compact to stay within the 2k context.

🧠 RAG with the Boo LLM (Phi-4-Mini-Instruct, Q4_K_M, GGUF)

This end‑to‑end example shows how to build a tiny Retrieval‑Augmented Generation (RAG) pipeline using Boo for generation (via llama-cpp-python) and an embedding model (e.g., the “Bobo” embedding derived from mixedbread-ai/mxbai-embed-large-v1) with FAISS for similarity search.

---

📦 1) Install Dependencies

pip install llama-cpp-python sentence-transformers faiss-cpu numpy

---

🧱 2) Minimal Data & Ingestion

import os
import numpy as np
import faiss
from sentence_transformers import SentenceTransformer

# --- Configuration ---
# Path to your quantized Boo model file (GGUF)
BOO_MODEL_PATH = "Boo.Q4_K_M.gguf"

# Choose an embedding model (here: mixedbread large; you can substitute your own)
EMBED_MODEL_ID = "mixedbread-ai/mxbai-embed-large-v1"

# A tiny toy "corpus" for demo purposes (normally you'd load real documents and chunk them)
DOCUMENTS = [
    {"id": "doc1", "text": "Retrieval-Augmented Generation (RAG) combines document retrieval with a generator LLM."},
    {"id": "doc2", "text": "FAISS enables efficient vector similarity search using approximate or exact indexes."},
    {"id": "doc3", "text": "Cosine similarity is often used with L2-normalized embeddings to measure semantic closeness."},
    {"id": "doc4", "text": "Chunking long documents into smaller passages improves retrieval granularity and accuracy."},
    {"id": "doc5", "text": "Boo is a lightweight LLM packaged as GGUF, suitable for local inference via llama.cpp."},
]

# --- Embedder ---
embedder = SentenceTransformer(EMBED_MODEL_ID)

# Encode and L2-normalize for cosine via inner product
def encode_texts(texts):
    emb = embedder.encode(texts, normalize_embeddings=True)
    return emb.astype(np.float32)

# Create the matrix of document embeddings
corpus_texts = [d["text"] for d in DOCUMENTS]
corpus_vecs = encode_texts(corpus_texts)
dim = corpus_vecs.shape[1]

# --- Build FAISS index (inner product works like cosine when vectors are normalized) ---
index = faiss.IndexFlatIP(dim)
index.add(corpus_vecs)

# Keep ID mapping for retrieved results
id_map = np.array([i for i in range(len(DOCUMENTS))])

---

🔎 3) Retrieval Function

def retrieve(query, k=3):
    q_vec = encode_texts([query])  # already normalized
    scores, idx = index.search(q_vec, k)
    results = []
    for rank, (sc, ii) in enumerate(zip(scores[0], idx[0])):
        doc = DOCUMENTS[id_map[ii]]
        results.append({"rank": rank + 1, "score": float(sc), "id": doc["id"], "text": doc["text"]})
    return results

---

🦙 4) Generation with Boo (llama-cpp-python)

from llama_cpp import Llama

# Initialize Boo
# Adjust n_ctx (context) and n_threads to your environment
llm = Llama(
    model_path=BOO_MODEL_PATH,
    n_ctx=2048,
    n_threads=8
)

def build_prompt(query, context_chunks):
    ctx_lines = "\n".join([f"• {c['text']}" for c in context_chunks])
    prompt = f"""

You are a concise, factual assistant. Use only the provided context to answer the question. If the answer cannot be found in the context, say "I don't know."

Context: {ctx_lines}

Question: {query}

Answer (concise, with references to bullet numbers if applicable): """ return prompt.strip()

def generate_with_boo(prompt, max_tokens=256, temperature=0.6, top_p=0.9):
    out = llm(
        prompt,
        max_tokens=max_tokens,
        temperature=temperature,
        top_p=top_p
    )
    return out["choices"][0]["text"]

---

🧪 5) End‑to‑End RAG Query

user_query = "How does RAG improve factuality, and which library helps with vector search?"
top_k = 3

# 1) Retrieve
retrieved = retrieve(user_query, k=top_k)

# 2) Build prompt
prompt = build_prompt(user_query, retrieved)

# 3) Generate with Boo
answer = generate_with_boo(prompt)

print("---- Retrieved Chunks ----")
for r in retrieved:
    print(f"[{r['rank']}] (score={r['score']:.3f}) {r['text']}")

print("\n---- Boo Answer ----")
print(answer)

---

🧰 6) Practical Tips

• Chunking: For real docs, split into ~300–600 characters (or ~128–256 tokens) with 10–20% overlap.
• Normalization: L2-normalize embeddings when using cosine/IP search.
• Metadata: Store doc IDs, titles, and citations so Boo can reference sources.
• Guardrails: If retrieval comes back empty or low‑score, have Boo say “I don’t know.”
• Prompt Budget: Keep the context short and relevant—Boo’s default context is ~2k tokens.

---

---

🔒 Prompt Engineering Tips

• Keep prompts concise to fit Boo’s 2k token window.
• Use role-style instructions for better structure:

  You are a concise, factual assistant. 
  Always explain reasoning briefly and avoid unnecessary detail.
  

• For step-by-step outputs, explicitly request them:

  List the steps to make sourdough bread.
  

📊 Prompt Engineering Library

• Guro is a prompt library designed to supercharge AI agents and assistants with task-specific personas -ie, total randos.
• From academic writing to financial analysis, technical support, SEO, and beyond
• Guro provides precision-crafted prompt templates ready to drop into your LLM workflows.

🕒 Evaluation (indicative)

Boo shows improvements over the base Phi-4-Mini on common instruction tasks in small-context, quantized settings:

Task	Boo (Q4_K_M)	Base (Phi-4-Mini)
GSM8K (accuracy)	52.1%	44.8%
NaturalQuestions (EM / F1)	47.8 / 60.2	41.6 / 53.3
CNN/DailyMail (ROUGE-L)	38.4	33.9
HumanEval (pass@1, basic prompts)	6.3%	4.1%

Scores are approximate, reflect instruction-tuned, quantized inference, and are not directly comparable to full-precision or long-context runs.

🧩 Intended Use

• Lightweight instruction following, reasoning, summarization, and light code generation.
• Edge or desktop assistants, CLI tools, and RAG agents where low latency and small footprint are key.

⚡ Limitations

• Context: 2k tokens; use chunking or RAG for long documents.
• Quantization trade-offs: Q4_K_M sacrifices some precision for speed; complex coding or multi-hop reasoning may degrade versus higher-precision builds.
• As with any LLM, the model can hallucinate; add validation and guardrails.

⚙️ Training Details (summary)

• Base: Phi-4-Mini-Instruct
• Method: SFT on about 20k instruction examples (single-turn chat, few-shot QA, summarization).
• Packaging: GGUF Q4_K_M quantization for local runtimes (llama.cpp, LM Studio, etc.).

💻 Prompting

No special chat template is required. Use clear instructions and keep prompts concise. For multi-turn workflows, persist conversation state externally or via your app’s memory or RAG layer.

Example system style

You are a concise, accurate assistant. Prefer step-by-step reasoning only when needed.
Cite assumptions and ask for missing constraints.

---

🧩 Acknowledgements

• Base model: Phi-4-Mini-Instruct
• Quantization and local runtimes: GGUF ecosystem (for example, llama.cpp, LM Studio, Ollama loaders)

🏁 Changelog

• v1.0 (Q4_K_M, GGUF) — Initial release with instruction SFT; compatibility with llama.cpp and LM Studio; evaluation on GSM8K, NaturalQuestions, CNN/DailyMail, and HumanEval.

---

📝License

• Boo is published under the MIT General Public License v3

This model is a fine-tuned, quantized derivative of Phi-4-Mini-Instruct. You are responsible for ensuring your use complies with the parent model’s license and any dataset terms. For commercial deployment, review upstream licensing and your organization’s compliance requirements.