Instructions to use DrDrek/crewai-finetuned-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DrDrek/crewai-finetuned-model with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DrDrek/crewai-finetuned-model")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DrDrek/crewai-finetuned-model")
model = AutoModelForCausalLM.from_pretrained("DrDrek/crewai-finetuned-model")

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use DrDrek/crewai-finetuned-model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DrDrek/crewai-finetuned-model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DrDrek/crewai-finetuned-model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DrDrek/crewai-finetuned-model

SGLang

How to use DrDrek/crewai-finetuned-model with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DrDrek/crewai-finetuned-model" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DrDrek/crewai-finetuned-model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DrDrek/crewai-finetuned-model" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DrDrek/crewai-finetuned-model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use DrDrek/crewai-finetuned-model with Docker Model Runner:
```
docker model run hf.co/DrDrek/crewai-finetuned-model
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Finetuned model of gemma 2B for crewai library

Crewai Finetuned Model

This is a LoRA finetuned model of gemma-2B for crewai library that produces the Goal and Backstory description automatically in it's agent method or agent() by taking only Role as user input, this helps to generate proper descriptions of those parameters through a llm instead of manually writing it.

You can run the model on a GPU using the following code.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "DrDrek/crewai-finetuned-model"
input_text = "junior software developer"
torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    low_cpu_mem_usage=True,
    return_dict=True,
    torch_dtype=torch.float16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
input_ids = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**input_ids, max_length=128)
output = tokenizer.decode(outputs[0])
#print("llm output:",output)

backstory=(output.split("\n\n"))[1].split("\n\n")[0]
goal=(output.split(backstory)[1].replace("<eos>","")).replace("\n\n","")
print("backstory:",backstory)
print("goal:",goal)
#Output:
#>>backstory: I am a junior software developer with a passion for building innovative and user-friendly applications. I am currently studying Computer Science at the University of Waterloo, and I am always looking for new challenges and opportunities to grow as a developer.
#>>goal: I am a strong believer in the power of technology to improve people's lives, and I am dedicated to using my skills to make a positive impact in the world.I am always looking for new ways to learn and grow, and I am excited to see where my journey takes me.

Training Data

We have 103 rows of descriptions of different roles and their respective goals and backstory which is used to train the models, see this dataset for details.

Evaluation

The model gives a training loss of 0.931 on training data with 25 setps batch size as shown in below figure:

Future Developement

As the Model is trained on tiny dataset it may yeild inaccurate output sometimes, to avoid this we can train the model on a bigger dataset in future.

Downloads last month: 8

Safetensors

Model size

3B params

Tensor type

F16

Model tree for DrDrek/crewai-finetuned-model

Quantizations

1 model

DrDrek
/

crewai-finetuned-model