Instructions to use ajibawa-2023/Code-290k-13B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ajibawa-2023/Code-290k-13B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ajibawa-2023/Code-290k-13B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ajibawa-2023/Code-290k-13B")
model = AutoModelForCausalLM.from_pretrained("ajibawa-2023/Code-290k-13B")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ajibawa-2023/Code-290k-13B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ajibawa-2023/Code-290k-13B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajibawa-2023/Code-290k-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ajibawa-2023/Code-290k-13B

SGLang

How to use ajibawa-2023/Code-290k-13B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ajibawa-2023/Code-290k-13B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajibawa-2023/Code-290k-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ajibawa-2023/Code-290k-13B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ajibawa-2023/Code-290k-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ajibawa-2023/Code-290k-13B with Docker Model Runner:
```
docker model run hf.co/ajibawa-2023/Code-290k-13B
```

Try this with a better model?

by rombodawg - opened Jan 16, 2024

Discussion

rombodawg

Jan 16, 2024

Why dont you do this with a more prominent and popular model? Like mistral ai models (id recommend openchat-3.5-0106), or maybe one of the deepseekcoder-base models. The solar10b models would be a good option also. Im just saying, llama 2 is old news at this point

If you want a larger model i have released a base coding model myself that needs fine tuned on coding instruct bellow:

https://huggingface.co/rombodawg/Everyone-Coder-4x7b-Base

vampires

Jan 16, 2024

Please train for PHP also.

ajibawa-2023

Owner Jan 17, 2024

•

edited Jan 17, 2024

Hello @rombodawg , I plan to release ffts using both base models mistral & mixtral. Solar 10b I am yet to test it. I am not the fastest bit of slow but steady. Despite using old models (llama-2) my coding related models are in top 25 models on Can Ai Code .
Thank you for sharing your base model. I will test it. Also if you have access to 8xA100/H100 then kindly share it which will help me release more powerful models. Thanks

ajibawa-2023

Owner Jan 17, 2024

Hello @vampires I think there shouldn't be any problem with php.

vampires

Jan 17, 2024

Hello @vampires I think there shouldn't be any problem with php.

Great, thank you!

ajibawa-2023 changed discussion status to closed Jan 18, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment