# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("vikp/cleaner")
model = AutoModelForCausalLM.from_pretrained("vikp/cleaner")Quick Links
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
This model takes in dirty text (from pdfs and epubs), cleans it, and converts it to markdown. It removes page headers/footers, page numbers, etc.
- Downloads last month
- 8
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="vikp/cleaner")