Update README.md

README.md

@@ -4,18 +4,60 @@ tags:
 - text-generation-inference
 - transformers
 - unsloth
-- llama
 license: apache-2.0
 language:
 - en
 ---
+## Model Description
 
-# Uploaded finetuned  model
+Request Writer Smol has been fine tuned to generate Freedom of Information (FOI) requests to UK public authorities based on the autority name and three keywords. The model has been trained on a synthetic dataset of FOI requests covering various topics and public authorities across the UK.
+The Model demonstrates improved generation of properly formatted, focused FOI requests for specific information that are unlikely to be refused on cost grounds.
 
-- **Developed by:** HMC83
-- **License:** apache-2.0
-- **Finetuned from model :** HuggingFaceTB/SmolLM2-360M-Instruct
+## Model Architecture
 
-This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+- **Base Model**: SmolLM2-360M-Instruct
+- **Fine-tuning Method**: LoRA
+- **LoRA Configuration**:
+  - Rank (r): 8
+  - Alpha: 16
+  - Dropout: 0.1
+  - Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
+- **Training Parameters**: 2.34% of total parameters trained (8.68M trainable parameters)
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+## Finetune training Data
+
+### Dataset Details
+- **Source**: Synthetic FOI requests dataset (HMC83/synthetic_foi_requests)
+- **Size**: 51,308 training examples, ~5,700 validation examples
+- **Format**: Conversational format with system prompts, user inputs, and assistant responses
+
+### Training Configuration
+- **Epochs**: 3
+- **Batch Size**: 32
+- **Learning Rate**: 1e-5
+- **Optimizer**: AdamW 8-bit
+- **Sequence Length**: 4096 tokens
+
+## Limitations and Considerations
+
+Small size of the model (360M parameters) may limit the complexity of any generated requests. The model is trained specifically for UK FOI requests. It has not been trained to generate requests for information about individuals.
+
+## Usage Guidelines
+
+### Input Format
+The model expects a prompt in the form of:
+```
+Generate a formal Freedom of Information request to [authority_name] using these keywords: [keyword1, keyword2, keyword3]
+```
+
+### Output Format
+It will try to generate a concinse, properly structured FOI request, starting with the phrase "Please provide me with a copy of the following information:" followed by 1 to 3 Numbered, specific information requests
+
+## Model Versions
+
+### Available Formats
+- **LoRA Adapters**: `HMC83/request_writer_smol_lora`
+- **Merged 16-bit**: `HMC83/request_writer_smol`
+
+### Disclaimer
+Users are responsible for ensuring that their intended use complies with any applicable laws and regulations. Generated requests should be reviewed and potentially modified before submission to public authorities. Requests should be made in good faith and for legitimate purposes. The model can hallucinate, so any outputs should not be relied upon without being verified. Outputs may also reflect any biases that are present in the underlying training data.