| base_model: | |
| - meta-llama/Llama-3.1-8B | |
| datasets: | |
| - allenai/winogrande | |
| - allenai/ai2_arc | |
| - google/boolq | |
| - wentingzhao/obqa | |
| license: llama3.1 | |
| tags: | |
| - peft | |
| - bayesian | |
| pipeline_tag: text-generation | |
| library_name: transformers | |
| This repository contains a low-rank adapter model, based on [Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B), which was presented in the paper [Training-Free Bayesianization for Low-Rank Adapters of Large Language Models](https://huggingface.co/papers/2412.05723). | |
| **Training-Free Bayesianization (TFB)** is a simple yet theoretically grounded framework that efficiently transforms trained low-rank adapters into Bayesian ones without additional training. TFB systematically searches for the maximally acceptable level of variance in the weight posterior, constrained within a family of low-rank isotropic Gaussian distributions. This approach aims to achieve superior uncertainty estimation and generalization compared to existing methods, while eliminating the need for complex Bayesianization training procedures. | |
| For the code, installation instructions, and further details on how to use the TFB framework, please refer to the official GitHub repository: | |
| [https://github.com/Wang-ML-Lab/bayesian-peft](https://github.com/Wang-ML-Lab/bayesian-peft) |