Instructions to use autotrust/JEV-27B-VL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use autotrust/JEV-27B-VL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="autotrust/JEV-27B-VL")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("autotrust/JEV-27B-VL")
model = AutoModelForMultimodalLM.from_pretrained("autotrust/JEV-27B-VL", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use autotrust/JEV-27B-VL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "autotrust/JEV-27B-VL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "autotrust/JEV-27B-VL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/autotrust/JEV-27B-VL

SGLang

How to use autotrust/JEV-27B-VL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "autotrust/JEV-27B-VL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "autotrust/JEV-27B-VL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "autotrust/JEV-27B-VL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "autotrust/JEV-27B-VL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use autotrust/JEV-27B-VL with Docker Model Runner:
```
docker model run hf.co/autotrust/JEV-27B-VL
```

Feature Request: JEV-4B-VL — 12GB VRAM Multimodal Decision Model

by Exo87 - opened 6 days ago

Discussion

Exo87

6 days ago

JEV-27B-VL is impressive but needs ~80GB VRAM, making it inaccessible to most users. A ~4B variant could run/train on a single 12GB GPU while leaving 2–4GB free for a game or other apps.

Proposal:
Release JEV-4B-VL, distilled from JEV-27B-VL, preserving the calibrated decision head and System 1/System 2 modes. Suggested backbone: Qwen3-VL-4B or Gemma 3 4B. Use 4-bit QLoRA + Unsloth for low-VRAM training/inference.

Target:

· 4B params
· 6–10GB VRAM with 4-bit/LoRA
· Trainable on RTX 3060/4070 12GB
· Apache-2.0, Unsloth-compatible scripts, quantized weights

Why feasible:
4B VL models already fit in ~8–10GB with QLoRA; 2B VL QLoRA runs in ~6.7GB. A 4B JEV variant is realistic and would democratize low-latency multimodal decisions on consumer hardware.

AutoTrustAILab

AutoTrust AI Lab org 6 days ago

Hi Marwan (@Exo87 ) ,

Thanks for taking the time to write such a detailed, well-reasoned proposal. This kind of feedback is genuinely helpful to us.

We hear you on hardware accessibility. A quantised version of JEV-27B-VL is already in our plan, which will significantly reduce the memory needed to run it.

The idea of a distilled JEV-4B-VL for 12GB-class GPUs is a great suggestion. We've noted it, along with your points on backbone options and the QLoRA/Unsloth workflow, and will take it into account as we plan future releases.

We'll share updates as plans firm up. Thanks again, and please keep the feedback coming and you can follow us on X @AutoTrustAI for more updates.

— The AutoTrust AI team

cloudyu

AutoTrust AI Lab org 3 days ago

try this new model https://huggingface.co/autotrust/GEV-26B-Decide-NVFP4 only 18G

ironbcc

1 day ago

@Exo87 If the need is a multimodal decision model you can run on consumer hardware now, rather than a JEV distill specifically: I built Seb-9B, an Apache-2.0 model that takes a text, JSON or image state plus one question with fixed options and returns a probability for every option in one forward pass: https://huggingface.co/ironbcc/seb-9b

It's 9B rather than 27B. The GGUF build (https://huggingface.co/ironbcc/seb-9b-GGUF) is 9.8 GB at Q8_0 plus a 0.9 GB vision projector, so 8-bit is too tight for your 12 GB target with headroom. A 4-bit quant would fit, but I've only validated the 8-bit builds and haven't measured accuracy below that.

Differences from what you described: it isn't distilled from JEV-27B-VL, it has no System 2 mode, choice questions take at most 20 options, and I haven't run it head-to-head against JEV-27B-VL. The card compares it with TypeSafe Jev 1.13 on identical rows and reports image results on a held-out set.

deluxenew

about 22 hours ago

а в доту будет шпилить или это сложно для нее?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment