Model Card for TinyLlama ArXiv Summarizer (LoRA)
Model Details
Model Description
This model is a fine-tuned version of an open-source Large Language Model for summarizing AI research paper abstracts. It uses Parameter-Efficient Fine-Tuning (PEFT) with LoRA to efficiently adapt the base model for domain-specific summarization tasks.
- Developed by: Ransilu Ranasinghe
- Model type: Causal Language Model (LLM)
- Language(s): English
- License: Apache 2.0 (inherits from base model)
- Finetuned from model: TinyLlama-1.1B-Chat
Uses
Direct Use
This model is intended for generating concise summaries of AI research paper abstracts. It can be used in:
- Research paper summarization tools
- AI-powered academic assistants
- Knowledge extraction systems
Out-of-Scope Use
- Not suitable for factual question answering without verification
- Not optimized for non-AI domains
- Should not be used for critical decision-making systems
Bias, Risks, and Limitations
- The model may generate inaccurate or incomplete summaries
- Biases present in the training dataset may be reflected in outputs
- Performance is limited due to small dataset and lightweight fine-tuning
Recommendations
- Use outputs as assistive summaries, not final conclusions
- Validate critical information from original sources
How to Get Started with the Model
Load the base model and apply the LoRA adapter using the Hugging Face Transformers and PEFT libraries.
Training Details
Training Data
- Dataset: ArXiv research paper dataset
- Domain: AI and Machine Learning research abstracts
- Dataset sourced via Hugging Face
Training Procedure
Training Hyperparameters
- Training regime: fp16 mixed precision
- Epochs: 1–2
- Method: PEFT (LoRA)
Evaluation
Metrics
- ROUGE-1
- ROUGE-2
- ROUGE-L
Results
The fine-tuned model shows improved summarization quality compared to the base model, producing more structured and relevant summaries.
Technical Specifications
Model Architecture and Objective
- Transformer-based causal language model
- Fine-tuned for text summarization using instruction-style prompts
Model Card Authors
Ransilu
Model Card Contact
LinkedIn:www.linkedin.com/in/ransiluranasinghe / GitHub profile:https://github.com/RansiluRanasinghe