Instructions to use jgpandrade18/vanka_v01 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jgpandrade18/vanka_v01 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="jgpandrade18/vanka_v01") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("jgpandrade18/vanka_v01") model = AutoModelForCausalLM.from_pretrained("jgpandrade18/vanka_v01", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use jgpandrade18/vanka_v01 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jgpandrade18/vanka_v01" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgpandrade18/vanka_v01", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/jgpandrade18/vanka_v01
- SGLang
How to use jgpandrade18/vanka_v01 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "jgpandrade18/vanka_v01" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgpandrade18/vanka_v01", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "jgpandrade18/vanka_v01" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgpandrade18/vanka_v01", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use jgpandrade18/vanka_v01 with Docker Model Runner:
docker model run hf.co/jgpandrade18/vanka_v01
vanka_v01
A 4B model that runs a parallel multi-agent extraction loop: as orchestrator it splits a task's
variables into groups and calls spawn_subagent once per group (in parallel), then calls calculate;
as sub-agent it extracts its assigned variables from the text as JSON, with verbatim evidence.
The final answer comes from deterministic code, not from the model.
Distilled from Qwen3.6-35B-A3B (AWQ) into Qwen/Qwen3.5-4B on
MedCalc-Bench rule-based calculators (calculate = the official MedCalc-Bench implementations).
Research prototype. Not for clinical use. Do not use its outputs to make or support medical decisions.
Code, prompts, tool schemas and the full pipeline: https://github.com/Pires-De-Andrade/multiagent_classification
Results (MedCalc-Bench Verified test, 17 rule-based calculators, n=340)
| Condition | Acc. | Per-variable acc. | No answer | Critical-path tokens |
|---|---|---|---|---|
| Qwen3.5-2B, single call, no training | 9.4% | 52.3% | 68.5% | 597 |
| vanka_v01 | 80.9% | 93.2% | 1.2% | 647 |
| vanka_v01 + 5-vote sub-agents | 81.8% | 93.4% | 0.0% | 703 |
| Teacher (35B), orchestrator | 79.1% | 92.8% | 5.6% | 1561 |
| Teacher (35B), single agent | 58.8% | 88.0% | 20.3% | 666 |
vanka_v01 vs the teacher orchestrator: +1.8 pts, not significant (exact McNemar p=0.53), with ~2.4x fewer critical-path tokens. APACHE II and Caprini are excluded from the main metric (unreliable labels: the official code reproduces only 50% / 45% of their test labels).
Limitations:
- One benchmark, synthetic and PubMed-derived notes; no external validation.
- Training targets come from teacher episodes checked against labels that the benchmark computed from GPT-4 extractions; agreement with the label does not mean clinical correctness.
- The effect of coordination at a fixed model size is not isolated (no trained single-agent 4B control).
- A GRPO stage on the sub-agent role was tried and lowered accuracy (80.9 -> 75.3%); it is not part of this model.
Usage
The model expects the prompts and tools in the repository's harness (src/medswarm/prompts.py), with thinking
disabled. With vLLM:
git clone https://github.com/Pires-De-Andrade/multiagent_classification && cd multiagent_classification
pip install -e . vllm
bash scripts/fetch_data.sh # MedCalc-Bench Verified + official calculator code
python -m medswarm.oracle # label-consistency flags used by the harness
vllm serve jgpandrade18/vanka_v01 --served-model-name jgpandrade18/vanka_v01 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --language-model-only
python -m medswarm.harness --mode orchestrator --model jgpandrade18/vanka_v01 --out runs/vanka.jsonl
python -m medswarm.demo runs/vanka.jsonl --row <row> # agent tree of one episode
adapter/ holds the LoRA adapter for Qwen/Qwen3.5-4B (use it with the base model's chat template).
Training
- Data: 1,081 correct teacher episodes on MedCalc-Bench train (2,426 rows; capped at 200 episodes per calculator; two rejection-sampling passes on rows the teacher first got wrong) → 4,228 samples (orchestrator tool calls and sub-agent answers; assistant-only loss).
- LoRA (rsLoRA) r=64, alpha=16, all linear layers; lr 2e-4, 1 epoch, effective batch 16, bf16, Liger kernels, gradient checkpointing. One RTX 4090: 46 min, 13.2 GiB peak VRAM.
Citation
MedCalc-Bench: Khandekar et al., "MedCalc-Bench: Evaluating Large Language Models for Medical Calculations", NeurIPS 2024 Datasets and Benchmarks.
- Downloads last month
- 37