File size: 1,287 Bytes
3ec5344 2ffe009 3ec5344 2ffe009 3ec5344 2ffe009 3ec5344 2ffe009 3ec5344 2ffe009 3ec5344 2ffe009 3ec5344 2ffe009 3ec5344 2ffe009 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
---
library_name: transformers
datasets:
- youngermax/text-tagging
---
## Model Details
### Model Description
This model identifies multiple topics related to the text in natural language. It is finetuned on youngermax/text-tagging for 3.5 epoch over ~1.3 hours on a free Kaggle P100.
- **Developed by:** Lincoln Maxwell
- **Model type:** Generative Pretrained Transformer
- **Language(s) (NLP):** English
- **Finetuned from model:** DistilGPT2
## Uses
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
### Direct Use
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
```python
input_ids = tokenizer.encode(prompt + '<|topic|>', return_tensors='pt').to('cuda')
# Generate text
output = model.generate(
input_ids,
max_length=1024,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.eos_token_id,
top_k=100,
top_p=0.5,
temperature=1
)
# Decode the output
text = tokenizer.decode(output[0], skip_special_tokens=False, early_stopping=True)
text = text[len(prompt):text.find('<|endoftext|>')]
topics = list(set(list(map(lambda x: x.strip(), text.split('<|topic|>')))[1:]))
``` |