Instructions to use CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML")
model = AutoModelForCausalLM.from_pretrained("CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML

SGLang

How to use CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML with Docker Model Runner:
```
docker model run hf.co/CONCISE/LLaMa_V2-13B-Instruct-Uncensored-GGML
```

LLaMa_V2-13B-Instruct-Uncensored-GGML

Commit History

Update config.json

271cbe3

a4to commited on Aug 17, 2023

Update config.json

705879c

a4to commited on Aug 17, 2023

Upload config.json

83c7703

a4to commited on Aug 17, 2023

Upload Model

8f5643e

a4to commited on Aug 13, 2023

Update README.md

b767c07

a4to commited on Aug 13, 2023

Upload Model

c3d508c

a4to commited on Aug 11, 2023

Model Upload

fdcab5f

a4to commited on Aug 11, 2023

Upload Model

dee8d51

a4to commited on Aug 11, 2023

Update README.md

cef2026

a4to commited on Aug 11, 2023

Update README.md

78bec25

a4to commited on Aug 11, 2023

Update README.md

8b4e193

a4to commited on Aug 11, 2023

initial commit

db9da4d

Connor Etherington commited on Aug 11, 2023

Commit History

Update config.json 271cbe3

Update config.json 705879c

Upload config.json 83c7703

Upload Model 8f5643e

Update README.md b767c07

Upload Model c3d508c

Model Upload fdcab5f

Upload Model dee8d51

Update README.md cef2026

Update README.md 78bec25

Update README.md 8b4e193

initial commit db9da4d

Update config.json

271cbe3

Update config.json

705879c

Upload config.json

83c7703

Upload Model

8f5643e

Update README.md

b767c07

Upload Model

c3d508c

Model Upload

fdcab5f

Upload Model

dee8d51

Update README.md

cef2026

Update README.md

78bec25

Update README.md

8b4e193

initial commit

db9da4d