AttributeError: 'list' object has no attribute 'shape'
#2
by BigSalmon - opened
pip install transformers
prefix = "this is some text preceding the cursor,"
suffix = "and this is some text after it."
model_tokenized_input = [50253, *tokenizer(suffix), 50254, *tokenizer(prefix), 50255]
infilled = model.generate(model_tokenized_input)
AttributeError: 'list' object has no attribute 'shape'
I went with:
device = "cuda" if torch.cuda.is_available() else "cpu"
def infill(prefix, suffix):
input_ids = (
torch.tensor(
[
50253,
*tokenizer(suffix)["input_ids"],
50254,
*tokenizer(prefix)["input_ids"],
50255,
]
)
.reshape(1, -1)
.to(device)
)
attention_mask = torch.ones_like(input_ids)
infilled = model.generate(input_ids=input_ids, attention_mask=attention_mask)
filled_text = tokenizer.decode(
infilled[0, ...][input_ids[0].tolist().index(50255) + 1 :], special_tokens=False
)
return filled_text
infill("A rabbit is an", "found in the wild.")
(Though infilling doesn't currently work properly due to the tokenizer, see https://huggingface.co/CarperAI/FIM-NeoX-1.3B/discussions/1)
Thank you! I’ll use this on Tuesday or Wednesday, when they’ve updated the tokenizer.
Tokenizer has been patched. Thank you for your patience.