File size: 1,779 Bytes
e281507 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | ---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
library_name: transformers
---
# Pico Mini V1
Pico v1 is a work in progress model. Based off Qwen 2.5 .5b model, it has been fine tuned for automatic COT and self reflection.
When making a output, Pico will create three sections, a reasoning section, a self-reflection section and a output section.
Pico Mini v1 struggles with non-question related tasks (Small talk, roleplay, etc).
Pico Mini v1 can struggle with staying on topic at times.
Here is a example of how you can use it:
```from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load the model and tokenizer from the Hugging Face Model Hub (test/test repository)
output_dir = "test/test"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Loading the model and tokenizer from the Hugging Face Hub...")
model = AutoModelForCausalLM.from_pretrained(output_dir).to(device) # Ensure model is on the same device
tokenizer = AutoTokenizer.from_pretrained(output_dir)
# Define the testing prompt
prompt = "What color is the sky?"
print(f"Testing prompt: {prompt}")
# Tokenize input and move to the same device as the model
inputs = tokenizer(prompt, return_tensors="pt").to(device) # Ensure inputs are on the same device
# Generate response
print("Generating response...")
outputs = model.generate(
**inputs,
max_new_tokens=1550, # Adjust the max tokens if needed
temperature=0.5, # Adjust for response randomness
top_k=50, # Adjust for top-k sampling
top_p=0.9 # Adjust for nucleus sampling
)
# Decode and print the response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("Generated response:")
print(response)
``` |