Instructions to use sapromak/OpenCoder-1.5B-Base-32K-via-16K with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sapromak/OpenCoder-1.5B-Base-32K-via-16K with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sapromak/OpenCoder-1.5B-Base-32K-via-16K")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("sapromak/OpenCoder-1.5B-Base-32K-via-16K")
model = AutoModelForCausalLM.from_pretrained("sapromak/OpenCoder-1.5B-Base-32K-via-16K")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use sapromak/OpenCoder-1.5B-Base-32K-via-16K with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sapromak/OpenCoder-1.5B-Base-32K-via-16K"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapromak/OpenCoder-1.5B-Base-32K-via-16K",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/sapromak/OpenCoder-1.5B-Base-32K-via-16K

SGLang

How to use sapromak/OpenCoder-1.5B-Base-32K-via-16K with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sapromak/OpenCoder-1.5B-Base-32K-via-16K" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapromak/OpenCoder-1.5B-Base-32K-via-16K",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sapromak/OpenCoder-1.5B-Base-32K-via-16K" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapromak/OpenCoder-1.5B-Base-32K-via-16K",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use sapromak/OpenCoder-1.5B-Base-32K-via-16K with Docker Model Runner:
```
docker model run hf.co/sapromak/OpenCoder-1.5B-Base-32K-via-16K
```

OpenCoder-1.5B-Base-32K-via-16K

Home Page • Collection • Paper • Thesis

Description

This model is derived from OpenCoder-1.5B-Base by applying an additional context extension fine-tuning with an Adjustment of the Base Frequency parameter of RoPE from 10,000 to 500,000. The number of optimization steps is 512 with a batch size of 128 on sequences of 16,384 length. The repository context is composed based on the Path Distance heuristics, more details on which and other aspects, including all code used, can be found on the Home Page of the project. Note that this model is created with the intent to answer specific research questions and not to gain the maximum possible performance on the repository-level code completion setup. Consider it more as a baseline.

The associated research was initialized and conducted by the JetBrains Research association.

Exact Match on the inproject lines of the large-context subset of the Project-Level Code Completion task from the Long Code Arena benchmark. This checkpoint (solid orange curve) demonstrates its best performance at a context length of 32,768. "1K" refers to 1,024 tokens. The star markers denote the context length used during the repository-level pre-training stage.

Quickstart

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "sapromak/OpenCoder-1.5B-Base-32K-via-16K"
tokenizer_name = "infly/OpenCoder-1.5B-Base"

model = AutoModelForCausalLM.from_pretrained(model_name,
                                             torch_dtype=torch.bfloat16,
                                             device_map="auto",
                                             trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name, trust_remote_code=True)

inputs = tokenizer("# write a quick sort algorithm", return_tensors="pt")
outputs = model.generate(**inputs.to(model.device), max_new_tokens=256)

result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)