File size: 1,970 Bytes
08eff87 cb5d751 0f190ee 08eff87 cb5d751 09c5e35 cb5d751 0f190ee cb5d751 09c5e35 cb5d751 0f190ee cb5d751 09c5e35 cb5d751 0f190ee cb5d751 09c5e35 cb5d751 0f190ee cb5d751 09c5e35 0f190ee | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | ---
base_model: unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
- lora
- transformers
- unsloth
license: llama3.2
language:
- en
---
## LLaMaPaca
Model Details
Model Name: LLaMaPaca
Base Model: unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
Adapter Type: LoRA (Low-Rank Adaptation)
Library: PEFT (Parameter-Efficient Fine-Tuning)
Pipeline Tag: text-generation
## Description
LLaMaPaca is a LoRA adapter fine-tuned on the LLaMA 3.2 1B Instruct model using Unsloth's optimized training framework. This adapter enables parameter-efficient customization of the base model for specific tasks or domains while maintaining the core capabilities of LLaMA 3.2.
The adapter was trained using 4-bit quantization via bitsandbytes, making it memory-efficient and suitable for deployment on consumer-grade hardware.
## Technical Specifications
Architecture: LLaMA 3.2 with LoRA adapters
Base Model Size: ~1B parameters
Quantization: 4-bit (bitsandbytes)
Training Framework: Unsloth + PEFT
Adapter Format: PEFT LoRA
## Training Details
Method: LoRA (Low-Rank Adaptation)
Optimization: Unsloth acceleration
Quantization: 4-bit precision with bitsandbytes
Framework: PEFT + Transformers
Intended Use Cases
Instruction following and conversational AI
Domain-specific text generation
Custom task adaptation with minimal resource requirements
Edge deployment scenarios requiring efficient models
Limitations
Performance depends on the quality and quantity of fine-tuning data
May inherit biases from the base LLaMA 3.2 model
4-bit quantization may result in slight accuracy trade-offs
Adapter is specific to the base model architecture
Citation
If you use this model in your research, please cite:
bibtex & TensorVizion
# License
Please refer to the base model license (LLaMA 3.2 Community License) and specify any additional licensing terms for your adapter. |