Update README.md

README.md

@@ -325,4 +325,191 @@ def convert_instella_model():
         raise
 
 if __name__ == "__main__":
-    convert_instella_model()
+    convert_instella_model()
+
+    # Documentation
+
+    # Instella Model Conversion to GGUF Format Documentation
+
+## Overview
+
+This script converts Instella models from the Hugging Face format (safetensors) to GGUF format with float16 precision for use with llama.cpp and other compatible inference engines. The conversion preserves the model architecture while ensuring compatibility with GGUF-based inference systems.
+
+## Script Structure
+
+The script consists of two main parts:
+
+1. **convert_instella_bf16.py**: The main script that orchestrates the conversion process
+2. **convert_instella_f16.py**: The generated conversion script that performs the actual conversion
+
+## Requirements
+
+- Python 3.8+
+- PyTorch
+- NumPy
+- safetensors
+
+## Usage
+
+```bash
+python convert_instella_bf16.py
+```
+
+This will:
+1. Install required dependencies
+2. Generate the conversion script
+3. Convert the model in the "huggintuned" directory
+4. Save the output as "huggintuned/model.gguf"
+
+For custom paths, modify the `model_dir` and `output_path` variables in the `convert_instella_model()` function.
+
+## Detailed Function Documentation
+
+### convert_instella_bf16.py
+
+#### `create_instella_conversion_script()`
+
+Generates the conversion script file with all necessary functions for GGUF conversion.
+
+**Returns:**
+- `str`: Path to the generated script file
+
+#### `convert_instella_model()`
+
+Main function that orchestrates the conversion process.
+
+1. Installs required dependencies
+2. Generates the conversion script
+3. Sets input and output paths
+4. Runs the conversion
+5. Verifies the output file
+
+### convert_instella_f16.py (Generated Script)
+
+#### `write_gguf_header(f, num_tensors, num_kv)`
+
+Writes the GGUF header to the output file.
+
+**Parameters:**
+- `f`: File object for writing
+- `num_tensors`: Number of tensors in the model
+- `num_kv`: Number of metadata key-value pairs
+
+#### `write_metadata_kv(f, key, val_type, val)`
+
+Writes a metadata key-value pair to the output file.
+
+**Parameters:**
+- `f`: File object for writing
+- `key`: Metadata key name
+- `val_type`: GGUF type identifier
+- `val`: Value to write
+
+#### `write_tensor_info(f, name, tensor)`
+
+Writes tensor information (name, shape, type) to the output file.
+
+**Parameters:**
+- `f`: File object for writing
+- `name`: Tensor name
+- `tensor`: PyTorch tensor
+
+#### `write_tensor_data(f, tensor)`
+
+Writes tensor data to the output file, converting to float16 format.
+
+**Parameters:**
+- `f`: File object for writing
+- `tensor`: PyTorch tensor
+
+#### `map_tensor_name(name)`
+
+Maps Hugging Face tensor names to GGUF tensor names.
+
+**Parameters:**
+- `name`: Original tensor name
+
+**Returns:**
+- Mapped tensor name for GGUF format
+
+#### `get_model_metadata(config_path)`
+
+Builds metadata for the GGUF model based on the Instella configuration.
+
+**Parameters:**
+- `config_path`: Path to the model's config.json file
+
+**Returns:**
+- Dictionary of metadata key-value pairs
+
+#### `convert_model(model_dir, output_path)`
+
+Main conversion function that processes the model and writes the GGUF file.
+
+**Parameters:**
+- `model_dir`: Directory containing the model files
+- `output_path`: Path to save the GGUF model
+
+## Model Architecture Parameters
+
+The script handles the following Instella model parameters:
+
+| Parameter | Default Value | Description |
+|-----------|---------------|-------------|
+| vocab_size | 50304 | Vocabulary size |
+| hidden_size | 4096 | Dimension of hidden representations |
+| intermediate_size | 11008 | Dimension of MLP representations |
+| num_hidden_layers | 32 | Number of transformer layers |
+| num_attention_heads | 32 | Number of attention heads |
+| num_key_value_heads | 32 | Number of key/value heads for GQA |
+| max_position_embeddings | 2048 | Maximum sequence length |
+| rope_theta | 10000.0 | Base period of RoPE embeddings |
+| rms_norm_eps | 1e-5 | Epsilon for RMS normalization |
+
+## Tensor Mapping
+
+The script maps tensor names from Hugging Face format to GGUF format:
+
+| Hugging Face Name | GGUF Name |
+|-------------------|-----------|
+| model.embed_tokens.weight | token_embd.weight |
+| model.norm.weight | output_norm.weight |
+| lm_head.weight | output.weight |
+| model.layers.{n}.self_attn.q_proj.weight | blk.{n}.attn_q.weight |
+| model.layers.{n}.self_attn.k_proj.weight | blk.{n}.attn_k.weight |
+| model.layers.{n}.self_attn.v_proj.weight | blk.{n}.attn_v.weight |
+| model.layers.{n}.self_attn.o_proj.weight | blk.{n}.attn_output.weight |
+| model.layers.{n}.mlp.gate_proj.weight | blk.{n}.ffn_gate.weight |
+| model.layers.{n}.mlp.up_proj.weight | blk.{n}.ffn_up.weight |
+| model.layers.{n}.mlp.down_proj.weight | blk.{n}.ffn_down.weight |
+| model.layers.{n}.input_layernorm.weight | blk.{n}.attn_norm.weight |
+| model.layers.{n}.post_attention_layernorm.weight | blk.{n}.ffn_norm.weight |
+
+## Precision Handling
+
+The script handles bfloat16 precision models by:
+
+1. Loading the original tensors (which may be in bfloat16)
+2. Converting to float32 for processing
+3. Converting to float16 for GGUF compatibility
+4. Writing the data in binary format
+
+## Error Handling
+
+The script includes error handling for:
+- Missing model files
+- Config file parsing errors
+- Conversion process errors
+- Output file verification
+
+## Notes
+
+- The script is specifically designed for Instella models but may work with similar architectures
+- The default parameters are based on the Instella2Config defaults
+- The script automatically detects and uses the model's configuration when available
+
+## Limitations
+
+- Only supports safetensors format (not PyTorch .bin files)
+- Does not support quantization (outputs float16 precision only)
+- May require adjustments for significantly different model architectures