Add README.md

README.md

@@ -1,111 +1,65 @@
 ---
-language:
-- en
-license: mit
+library_name: peft
+license: other
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
 tags:
+- llama-factory
 - lora
-- tool-calling
-- llama3
-- instruction-tuning
-- json-generation
-base_model: meta-llama/Meta-Llama-3-8B-Instruct
+- generated_from_trainer
+model-index:
+- name: llama-traces
+  results: []
 ---
 
-# Tool-Calling LoRA for LLaMA-3-8B-Instruct
-
-This is a LoRA (Low-Rank Adaptation) model fine-tuned on tool-calling datasets to enhance the model's ability to generate structured JSON responses for tool execution.
-
-## Model Details
-
-- **Base Model**: meta-llama/Meta-Llama-3-8B-Instruct
-- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
-- **LoRA Rank**: 16
-- **LoRA Alpha**: 32
-- **Training Dataset**: Custom tool-calling dataset with 357 samples
-- **Training Epochs**: 5
-- **Learning Rate**: 5.0e-5
-
-## Usage
-
-### Load the Model
-
-```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
-from peft import PeftModel
-
-# Load base model and tokenizer
-base_model = AutoModelForCausalLM.from_pretrained(
-    "meta-llama/Meta-Llama-3-8B-Instruct",
-    torch_dtype=torch.bfloat16,
-    device_map="auto"
-)
-tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
-
-# Load and merge LoRA
-model = PeftModel.from_pretrained(base_model, "YOUR_USERNAME/llama-traces")
-model = model.merge_and_unload()
-
-# Generate tool-calling responses
-def generate_tool_call(prompt):
-    inputs = tokenizer(prompt, return_tensors="pt")
-    outputs = model.generate(
-        **inputs,
-        max_new_tokens=512,
-        temperature=0.7,
-        do_sample=True,
-        pad_token_id=tokenizer.eos_token_id
-    )
-    return tokenizer.decode(outputs[0], skip_special_tokens=True)
-
-# Example usage
-prompt = "Check the weather in New York"
-response = generate_tool_call(prompt)
-print(response)
-```
-
-### Expected Output Format
-
-The model generates structured JSON responses like:
-```json
-{
-  "trace_id": "002",
-  "steps": [
-    {
-      "action": "call_api",
-      "api": "weather_api",
-      "arguments": {"location": "New York"}
-    },
-    {
-      "action": "respond",
-      "message": "The weather in New York is currently sunny with a temperature of 72°F."
-    }
-  ]
-}
-```
-
-## Training Details
-
-- **Dataset**: Custom tool-calling dataset with instruction/input/output format
-- **Template**: llama3 chat template
-- **Cutoff Length**: 4096 tokens
-- **Batch Size**: 2 (effective batch size: 8 with gradient accumulation)
-- **Optimizer**: AdamW with cosine learning rate scheduling
-- **Warmup Ratio**: 0.1
-
-## Performance
-
-The model shows improved capability in:
-- Generating structured JSON responses
-- Following tool-calling patterns
-- Maintaining context for multi-step tool execution
-- Producing consistent output formats
-
-## Limitations
-
-- Requires the base LLaMA-3-8B-Instruct model to function
-- May generate invalid JSON in some edge cases
-- Performance depends on the quality of the training data
-
-## License
-
-This model is released under the MIT License.
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# llama-traces
+
+This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on the test_tool_calling dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2230
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 8
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 5.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 0.2127        | 2.5031 | 100  | 0.2533          |
+| 0.1242        | 5.0    | 200  | 0.2230          |
+
+
+### Framework versions
+
+- PEFT 0.15.2
+- Transformers 4.52.4
+- Pytorch 2.6.0+cu124
+- Datasets 3.6.0
+- Tokenizers 0.21.1