Instructions to use substratusai/weaviate-gorilla-v4-schema-split with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use substratusai/weaviate-gorilla-v4-schema-split with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="substratusai/weaviate-gorilla-v4-schema-split")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("substratusai/weaviate-gorilla-v4-schema-split")
model = AutoModelForCausalLM.from_pretrained("substratusai/weaviate-gorilla-v4-schema-split")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use substratusai/weaviate-gorilla-v4-schema-split with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "substratusai/weaviate-gorilla-v4-schema-split"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "substratusai/weaviate-gorilla-v4-schema-split",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/substratusai/weaviate-gorilla-v4-schema-split

SGLang

How to use substratusai/weaviate-gorilla-v4-schema-split with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "substratusai/weaviate-gorilla-v4-schema-split" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "substratusai/weaviate-gorilla-v4-schema-split",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "substratusai/weaviate-gorilla-v4-schema-split" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "substratusai/weaviate-gorilla-v4-schema-split",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use substratusai/weaviate-gorilla-v4-schema-split with Docker Model Runner:
```
docker model run hf.co/substratusai/weaviate-gorilla-v4-schema-split
```

Dataset

Finetuned on: https://huggingface.co/datasets/weaviate/WeaviateGraphQLGorilla-SchemaSplit-Train

Prompt template

## Instruction
Your task is to write GraphQL for the Natural Language Query provided. Use the provided API reference and Schema to generate the GraphQL. The GraphQL should be valid for Weaviate.

Only use the API reference to understand the syntax of the request.

## Natural Language Query
{nlcommand}

## Schema
{schema}

## API reference
{apiRef}

## Answer
{output}

Example usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "substratusai/weaviate-gorilla-v4-schema-split"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    load_in_4bit=True,
    device_map='auto',
)

tokenizer = AutoTokenizer.from_pretrained(model_id)

text = """
## Instruction
Your task is to write GraphQL for the Natural Language Query provided. Use the provided API reference and Schema to generate the GraphQL. The GraphQL should be valid for Weaviate.

Only use the API reference to understand the syntax of the request.

## Natural Language Query
```text Get me the top 10 historical events related to 'World War II', and show the event name, description, year, significant impact, and the names and populations of the involved countries. ```

## Schema
{ "classes": [ { "class": "HistoricalEvent", "description": "Information about historical events", "vectorIndexType": "hnsw", "vectorizer": "text2vec-transformers", "properties": [ { "name": "eventName", "dataType": ["text"], "description": "Name of the historical event" }, { "name": "description", "dataType": ["text"], "description": "Detailed description of the event" }, { "name": "year", "dataType": ["int"], "description": "Year the event occurred" }, { "name": "hadSignificantImpact", "dataType": ["boolean"], "description": "Whether the event had a significant impact" }, { "name": "involvedCountries", "dataType": ["Country"], "description": "Countries involved in the event" }{ "class": "Country", "description": "Information about countries", "vectorIndexType": "hnsw", "vectorizer": "text2vec-transformers", "properties": [ { "name": "countryName", "dataType": ["text"], "description": "Name of the country" }, { "name": "population", "dataType": ["int"], "description": "Population of the country" }}}

## API reference
1. Limit BM25 search results Limit the results[] You can limit the number of results returned by a `bm25` search, - to a fixed number, using the `limit: <N>` operator - to the first N "drops" in `score`, using the `autocut` operator `autocut` can be combined with `limit: N`, which would limit autocut's input to the first `N` objects. Limiting the number of results Use the `limit` argument to specify the maximum number of results that should be returned: ```graphql { Get { JeopardyQuestion( bm25: { query: "safety" }, limit: 3 ) { question answer _additional { score } } } } ```

## Answer
```graphql
"""
device = "cuda:0"

inputs = tokenizer(text, return_tensors="pt").to(device)
# this was needed due to a issue with model not taking token_type_ids
# inputs.pop("token_type_ids")
outputs = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Downloads last month: 7