Phase 3 Formulator: QLoRA adapter (r=32, alpha=64) trained on 1,215 formulation problems

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,251 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-7B-Instruct
+tags:
+  - game-theory
+  - formulation
+  - qwen2
+  - lora
+  - qlora
+  - sft
+  - economics
+  - strategic-reasoning
+  - math
+  - decision-theory
+library_name: peft
+pipeline_tag: text-generation
+datasets:
+  - Alogotron/GameTheory-Formulator
+language:
+  - en
+model-index:
+  - name: GameTheory-Formulator-Model
+    results:
+      - task:
+          type: text-generation
+          name: Game Theory Formulation
+        dataset:
+          name: GameTheory-Formulator
+          type: Alogotron/GameTheory-Formulator
+        metrics:
+          - name: Valid Formulation Rate
+            type: accuracy
+            value: 100.0
+          - name: Eval Loss
+            type: loss
+            value: 0.8492
+          - name: Train Loss
+            type: loss
+            value: 1.0992
+---
+
+# 🎯 GameTheory-Formulator-Model
+
+**Phase 3 of the Alogotron Game Theory AI Pipeline** — A QLoRA adapter that teaches language models to translate real-world scenarios into formal game theory formulations.
+
+## Overview
+
+| Property | Value |
+|---|---|
+| **Base Model** | [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
+| **Method** | QLoRA (4-bit NF4 quantization + LoRA) |
+| **Task** | Real-world scenario → Formal game theory formulation |
+| **Dataset** | [Alogotron/GameTheory-Formulator](https://huggingface.co/datasets/Alogotron/GameTheory-Formulator) (1,215 examples) |
+| **Training** | SFT, 1 epoch, ~24 minutes on 2x RTX 3090 |
+| **Eval Accuracy** | **100.0% valid formulations** on held-out set |
+
+## The Alogotron Game Theory Pipeline
+
+This model is part of a 3-phase training pipeline:
+
+| Phase | Model | Task | Method |
+|---|---|---|---|
+| Phase 1 | [GameTheory-Solver](https://huggingface.co/Alogotron/GameTheory-Solver) | Solve formal GT problems | SFT on 2,913 problems → 94% accuracy |
+| Phase 2 | [GameTheory-Reasoner](https://huggingface.co/Alogotron/GameTheory-Reasoner) | Enhanced reasoning | GRPO on same dataset |
+| **Phase 3** | **GameTheory-Formulator** (this model) | **Real-world → formal GT** | **SFT on 1,215 formulation problems** |
+
+## What This Model Does
+
+Given a real-world scenario (business competition, political negotiation, security analysis, etc.), this model:
+
+1. **📋 Formulation Steps** — Walks through the reasoning to identify the game structure
+2. **🎮 Formal Game Model** — Identifies players, strategies, payoffs, information structure, and solution concept
+3. **🧮 Solution** — Solves the formulated game (Nash equilibrium, dominant strategies, etc.)
+4. **🌍 Real-World Interpretation** — Translates the mathematical solution back to actionable insights
+
+## Training Details
+
+### QLoRA Configuration
+| Parameter | Value |
+|---|---|
+| LoRA rank (r) | 32 |
+| LoRA alpha | 64 |
+| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
+| Quantization | 4-bit NF4 with double quantization |
+| Trainable params | 80.7M / 7.7B (1.05%) |
+
+### Training Hyperparameters
+| Parameter | Value |
+|---|---|
+| Epochs | 1 |
+| Batch size (per device) | 2 |
+| Gradient accumulation | 4 |
+| Effective batch size | 16 |
+| Learning rate | 5e-5 (cosine schedule) |
+| Optimizer | paged_adamw_8bit |
+| Max sequence length | 2048 |
+| Packing | Enabled |
+| Gradient checkpointing | Enabled |
+| Hardware | 2x NVIDIA RTX 3090 (24GB each) |
+
+### Training Metrics
+| Metric | Value |
+|---|---|
+| Train loss | 1.0992 |
+| Eval loss | 0.8492 |
+| Training time | 24.3 minutes |
+| Dataset size | 1215 examples |
+| Train split | 1093 examples |
+| Eval split | 122 examples |
+
+## Evaluation Results
+
+Tested on **20 held-out examples** across 6 domains and 3 difficulty levels:
+
+| Metric | Score |
+|---|---|
+| **Valid Formulations** | **100.0%** |
+| All sections present | 100.0% |
+| All GT elements identified | 100.0% |
+| Avg response length | 1821 chars |
+
+### By Domain
+| Domain | Valid |
+|---|---|
+| Business | 8/8 (100%) |
+| Security | 5/5 (100%) |
+| Politics | 2/2 (100%) |
+| Auctions | 2/2 (100%) |
+| Technology | 2/2 (100%) |
+| Social | 1/1 (100%) |
+
+### By Difficulty
+| Difficulty | Valid |
+|---|---|
+| Easy | 5/5 (100%) |
+| Medium | 9/9 (100%) |
+| Hard | 6/6 (100%) |
+
+## Usage
+
+### With PEFT + Transformers
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from peft import PeftModel
+import torch
+
+# Load base model in 4-bit
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16,
+    bnb_4bit_use_double_quant=True,
+)
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    "Qwen/Qwen2.5-7B-Instruct",
+    quantization_config=bnb_config,
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
+
+# Load the Formulator adapter
+model = PeftModel.from_pretrained(base_model, "Alogotron/GameTheory-Formulator-Model")
+model.eval()
+
+# Create a prompt
+messages = [
+    {"role": "system", "content": "You are a game theory expert. Given a real-world scenario, formulate it as a formal game theory model. Identify the players, strategies, payoffs, and information structure. Then solve the game and interpret the results."},
+    {"role": "user", "content": "Two coffee shops on the same street must decide whether to offer a loyalty program. If both offer it, they split customers evenly but incur costs. If neither offers it, they split evenly with no extra cost. If only one offers it, that shop attracts 70% of customers."}
+]
+
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(text, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.3, top_p=0.9)
+
+response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+### Example Output
+
+**Input Scenario:**
+> Two airline companies, Stellar and Haven, each control roughly half the market. They are independently deciding their pricing for the upcoming quarter...
+
+**Model Output:**
+
+> ## Formulation Steps
+> 
+> Step 1 - Stellar and Haven are each making pricing decisions that directly affect each other's profits...
+> 
+> Step 2 - Players: Stellar and Haven...
+> 
+> Step 3 - Strategies: Each firm can either 'Maintain Prices' or 'Cut Prices'...
+>
+> ## Formal Game Model
+> 
+> **Game Type:** Simultaneous
+> **Players:** Stellar, Haven
+> **Strategies:** Maintain Prices, Cut Prices
+> **Payoffs:** Both Maintain: (54, 54), Both Cut: (18, 18)...
+> **Solution Concept:** Nash Equilibrium
+>
+> ## Solution
+> 
+> Both firms will cut prices. Cutting is a dominant strategy for each...
+>
+> ## Real-World Interpretation
+> 
+> This is a classic Prisoner's Dilemma. Both companies rationally choose to cut prices, resulting in lower profits than cooperation would yield...
+
+## Dataset
+
+Trained on [Alogotron/GameTheory-Formulator](https://huggingface.co/datasets/Alogotron/GameTheory-Formulator) — 1,215 expert-crafted formulation problems across 6 domains:
+
+- **Business** (290): Pricing, market entry, production, R&D, supply chain
+- **Security** (230): Cybersecurity, threat modeling, defense allocation
+- **Politics** (195): Elections, negotiations, voting, international relations
+- **Social** (190): Social dilemmas, public goods, coordination, trust
+- **Technology** (165): Platform competition, standards, adoption, innovation
+- **Auctions** (145): First-price, second-price, common value, combinatorial
+
+## Related Models & Datasets
+
+| Resource | Link |
+|---|---|
+| Phase 1: Solver Model | [Alogotron/GameTheory-Solver](https://huggingface.co/Alogotron/GameTheory-Solver) |
+| Phase 2: Reasoner Model | [Alogotron/GameTheory-Reasoner](https://huggingface.co/Alogotron/GameTheory-Reasoner) |
+| Solver Dataset | [Alogotron/GameTheory-Bench](https://huggingface.co/datasets/Alogotron/GameTheory-Bench) |
+| Formulator Dataset | [Alogotron/GameTheory-Formulator](https://huggingface.co/datasets/Alogotron/GameTheory-Formulator) |
+
+## Limitations
+
+- Trained on synthetic formulation data; may not handle all real-world edge cases
+- Formulation quality depends on scenario clarity and completeness
+- Best suited for classical game theory formulations (simultaneous, sequential, auctions)
+- Does not cover cooperative game theory or mechanism design (yet)
+
+## Citation
+
+```bibtex
+@misc{alogotron-formulator-2025,
+  title={GameTheory-Formulator-Model: Real-World Scenario to Game Theory Formulation},
+  author={Alogotron},
+  year={2025},
+  publisher={HuggingFace},
+  url={https://huggingface.co/Alogotron/GameTheory-Formulator-Model}
+}
+```

adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f91a32639559f1cd940c09f7f036b7d6404f3967d06e785a1583d4fc8d9e1fa
+size 323014168

chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

eval_results.json

@@ -0,0 +1,273 @@
+{
+  "summary": {
+    "total": 20,
+    "valid_formulations": 20,
+    "valid_pct": 100.0,
+    "all_sections_pct": 100.0,
+    "all_elements_pct": 100.0,
+    "avg_response_length": 1821,
+    "avg_gen_time": 41.0
+  },
+  "results": [
+    {
+      "id": "FORM-BU-H-0131",
+      "domain": "business",
+      "difficulty": "hard",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1873,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 40.8
+    },
+    {
+      "id": "FORM-SO-E-0105",
+      "domain": "social",
+      "difficulty": "easy",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1877,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 38.0
+    },
+    {
+      "id": "FORM-BU-M-0084",
+      "domain": "business",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1740,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 51.5
+    },
+    {
+      "id": "FORM-BU-M-0174",
+      "domain": "business",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1863,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 49.8
+    },
+    {
+      "id": "FORM-PO-E-0138",
+      "domain": "politics",
+      "difficulty": "easy",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1652,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 37.0
+    },
+    {
+      "id": "FORM-SE-M-0019",
+      "domain": "security",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1966,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 41.4
+    },
+    {
+      "id": "FORM-AU-H-0184",
+      "domain": "auctions",
+      "difficulty": "hard",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 2218,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 41.9
+    },
+    {
+      "id": "FORM-SE-H-0066",
+      "domain": "security",
+      "difficulty": "hard",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1763,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 36.4
+    },
+    {
+      "id": "FORM-AU-E-0041",
+      "domain": "auctions",
+      "difficulty": "easy",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 2142,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 46.8
+    },
+    {
+      "id": "FORM-PO-M-0006",
+      "domain": "politics",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1611,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 30.1
+    },
+    {
+      "id": "FORM-BU-E-0021",
+      "domain": "business",
+      "difficulty": "easy",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1712,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 36.3
+    },
+    {
+      "id": "FORM-BU-H-0181",
+      "domain": "business",
+      "difficulty": "hard",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1973,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 49.7
+    },
+    {
+      "id": "FORM-SE-M-0016",
+      "domain": "security",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1971,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 39.1
+    },
+    {
+      "id": "FORM-BU-M-0078",
+      "domain": "business",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1450,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 44.1
+    },
+    {
+      "id": "FORM-SE-M-0063",
+      "domain": "security",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1736,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 37.8
+    },
+    {
+      "id": "FORM-BU-M-0204",
+      "domain": "business",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1885,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 37.9
+    },
+    {
+      "id": "FORM-TE-M-0135",
+      "domain": "technology",
+      "difficulty": "medium",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1485,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 32.9
+    },
+    {
+      "id": "FORM-SE-H-0108",
+      "domain": "security",
+      "difficulty": "hard",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1669,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 44.1
+    },
+    {
+      "id": "FORM-TE-H-0184",
+      "domain": "technology",
+      "difficulty": "hard",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 2043,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 45.2
+    },
+    {
+      "id": "FORM-BU-E-0101",
+      "domain": "business",
+      "difficulty": "easy",
+      "sections_found": "4/4",
+      "has_all_sections": true,
+      "elements_found": "4/4",
+      "has_all_elements": true,
+      "response_length": 1799,
+      "is_substantial": true,
+      "valid_formulation": true,
+      "gen_time": 38.9
+    }
+  ]
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6bda26c65a0dedb3a99b67c3ffd9ca1a721e10265bf5f71fd8a685eb32e8f7a
+size 5649

training_stats.json

@@ -0,0 +1,26 @@
+{
+  "phase": "Phase 3 - Formulator",
+  "base_model": "Qwen/Qwen2.5-7B-Instruct",
+  "dataset": "/home/beta1/gt-training/formulator/formulator_dataset.json",
+  "dataset_size": 1215,
+  "train_examples": 1093,
+  "eval_examples": 122,
+  "lora_r": 32,
+  "lora_alpha": 64,
+  "target_modules": "all_linear",
+  "epochs": 1,
+  "batch_size_per_device": 2,
+  "grad_accum": 4,
+  "effective_batch": 16,
+  "learning_rate": 5e-05,
+  "lr_scheduler": "cosine",
+  "max_seq_length": 2048,
+  "quantization": "4bit_nf4",
+  "train_loss": 1.0992090911195989,
+  "eval_loss": 0.8491532206535339,
+  "runtime_seconds": 1458.7688,
+  "runtime_minutes": 24.312813333333334,
+  "total_wall_time_seconds": 1469.986917257309,
+  "samples_per_second": 0.311,
+  "num_gpus": 2
+}