MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models
Paper • 2406.06046 • Published • 1
# Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("yuzc19/bert-base-uncased-data-influence-model-lambada")
model = AutoModelForSequenceClassification.from_pretrained("yuzc19/bert-base-uncased-data-influence-model-lambada")Quick Links
Data influence models for LAMBADA fine-tuned from bert-base-uncased.
The main branch contains the data influence model for 10k steps.
Paper: MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models
Official codebase: https://github.com/cxcscmu/MATES
- Downloads last month
- 5
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="yuzc19/bert-base-uncased-data-influence-model-lambada")