universal-dependencies/universal_dependencies
Updated • 5.59k • 34
How to use KoichiYasuoka/Swallow-7b-plus-upos with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("token-classification", model="KoichiYasuoka/Swallow-7b-plus-upos") # Load model directly
from transformers import AutoTokenizer, AutoModelForTokenClassification
tokenizer = AutoTokenizer.from_pretrained("KoichiYasuoka/Swallow-7b-plus-upos")
model = AutoModelForTokenClassification.from_pretrained("KoichiYasuoka/Swallow-7b-plus-upos")This is a LLaMA model for POS-tagging, derived from Swallow-7b-plus-hf. Every short-unit-word is tagged by UPOS (Universal Part-Of-Speech) and FEATS.
from transformers import pipeline
nlp=pipeline("upos","KoichiYasuoka/Swallow-7b-plus-upos",trust_remote_code=True,aggregation_strategy="simple")
print(nlp("国境の長いトンネルを抜けると雪国であった。"))
安岡孝一: GPT系モデルの系列ラベリングによる品詞付与, 東洋学へのコンピュータ利用, 第38回研究セミナー (2024年7月26日), pp.3-10.
Base model
tokyotech-llm/Swallow-7b-plus-hf