Improve model card: Add pipeline tag, library name, paper link, GitHub link, and enhanced description
Browse filesThis PR enhances the model card by:
- Adding `pipeline_tag: text-generation` to correctly classify the model's functionality.
- Including `library_name: transformers` to indicate compatibility with the Hugging Face Transformers library, which enables an automated code snippet.
- Adding `uncertainty-quantification` to the tags for better discoverability, aligning with the paper's focus on uncertainty estimation.
- Providing a direct link to the paper: [Training-Free Bayesianization for Low-Rank Adapters of Large Language Models](https://huggingface.co/papers/2412.05723).
- Adding a link to the GitHub repository: https://github.com/Wang-ML-Lab/bayesian-peft.
- Incorporating the paper's abstract as a detailed model description.
Please review and merge this PR if everything looks good.
|
@@ -1,13 +1,22 @@
|
|
| 1 |
---
|
| 2 |
-
|
|
|
|
| 3 |
datasets:
|
| 4 |
- allenai/winogrande
|
| 5 |
- allenai/ai2_arc
|
| 6 |
- google/boolq
|
| 7 |
- wentingzhao/obqa
|
| 8 |
-
|
| 9 |
-
- meta-llama/Llama-3.1-8B
|
| 10 |
tags:
|
| 11 |
- peft
|
| 12 |
- bayesian
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- meta-llama/Llama-3.1-8B
|
| 4 |
datasets:
|
| 5 |
- allenai/winogrande
|
| 6 |
- allenai/ai2_arc
|
| 7 |
- google/boolq
|
| 8 |
- wentingzhao/obqa
|
| 9 |
+
license: llama3.1
|
|
|
|
| 10 |
tags:
|
| 11 |
- peft
|
| 12 |
- bayesian
|
| 13 |
+
- uncertainty-quantification
|
| 14 |
+
pipeline_tag: text-generation
|
| 15 |
+
library_name: transformers
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
This repository hosts the adapter weights for the **Training-Free Bayesianization for Low-Rank Adapters of Large Language Models** (TFB) model, as introduced in the paper [Training-Free Bayesianization for Low-Rank Adapters of Large Language Models](https://huggingface.co/papers/2412.05723).
|
| 19 |
+
|
| 20 |
+
Estimating the uncertainty of responses from Large Language Models (LLMs) remains a critical challenge. While recent Bayesian methods have demonstrated effectiveness in quantifying uncertainty through low-rank weight updates, they typically require complex fine-tuning or post-training procedures. In this paper, we propose Training-Free Bayesianization (TFB), a simple yet theoretically grounded framework that efficiently transforms trained low-rank adapters into Bayesian ones without additional training. TFB systematically searches for the maximally acceptable level of variance in the weight posterior, constrained within a family of low-rank isotropic Gaussian distributions. Our theoretical analysis shows that under mild conditions, this search process is equivalent to KL-regularized variational optimization, a generalized form of variational inference. Through comprehensive experiments, we show that TFB achieves superior uncertainty estimation and generalization compared to existing methods while eliminating the need for complex Bayesianization training procedures.
|
| 21 |
+
|
| 22 |
+
For more details, code, and further experiments, please refer to the official [GitHub repository](https://github.com/Wang-ML-Lab/bayesian-peft).
|