Instructions to use Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4

SGLang

How to use Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4 with Docker Model Runner:
```
docker model run hf.co/Multilingual-Multimodal-NLP/IndustrialCoder-32B-AWQ-INT4
```

Add model card for InCoder-32B

by nielsr HF Staff - opened Mar 19

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

+95

-0

nielsr

Mar 19

Hi! I'm Niels from the Hugging Face community team. I noticed this model repository was missing a detailed model card.

This PR adds a comprehensive README that includes:

Metadata for pipeline_tag and library_name.
Links to the research paper and GitHub repository.
An overview of the model's specialized industrial capabilities (Chip Design, GPU Optimization, etc.).
Performance benchmarks across general and industrial tasks.
A "Quickstart" usage snippet for the transformers library.
Information about the training pipeline and a disclaimer regarding industrial use.

Add model card for InCoder-32B94e01092

Challenging666

Multilingual-Multimodal-NLP org Mar 19

Thank you so much!

Challenging666 changed pull request status to merged Mar 19

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment