A one-image smoke test of the four open-weight vision-language models that are candidates to replace Gemini in the analysis layer, run locally on the L40S (vLLM 0.31.0). Each model gets the same image, the same one-line prompt and the same strict JSON schema, at temperature 0. It checks that the model loads, reads an image and returns schema-valid JSON through the repository's own vllm client. It is not a quality comparison: that is the pilot on 15 papers (docs/model-pilot.md).

Input

Hendrick ter Brugghen, Crucifixion with the Virgin and Saint John
Hendrick ter Brugghen, Crucifixion with the Virgin and Saint John, ca. 1625, New York, The Metropolitan Museum of Art (public domain). Figure f001 of paper 68590ebc5313 (JHNA), as extracted: 329 × 376 px.

Prompt: Describe this image as JSON. Schema: medium (string), subject (string), people_count (integer or null), notes (list of strings). No system prompt, no caption, no article text.

Results

ModelCheckpointWeightsThinking Valid JSONPrompt tokensOutput tokens Seconds
Qwen3.6-35B-A3BQwen/Qwen3.6-35B-A3B-FP8 95a723dofficial FP8offyes23015731.53
Qwen3.8-27BQwen/Qwen3.8-27B-FP8 017b9c7official FP8offyes23017712.48
Gemma 4 26B-A4BRedHatAI/gemma-4-26B-A4B-it-FP8-dynamic ed35d7athird-party FP8offyes110610016.95
Mistral Small 3.2 24BRedHatAI/Mistral-Small-3.2-24B-Instruct-2506-FP8 9bc507cthird-party FP8n/a (not a reasoning model)yes1921669.82

Seconds are wall-clock and include waiting behind the pilot's own requests on the same server (one request at a time), so they are not a speed measurement. Prompt tokens are mostly the image: models size images differently (Qwen by a pixel range, Gemma by a token budget, Mistral by a 1,540 px cap), so equal images do not cost equal tokens.

Answers

Qwen3.6-35B-A3B

{
  "medium": "oil painting",
  "subject": "Crucifixion of Jesus Christ",
  "people_count": 3,
  "notes": [
    "Jesus is depicted on the cross, with a white loincloth and dark hair.",
    "To the left, a woman in a grey hooded robe kneels in grief, head bowed.",
    "To the right, a man in red and dark robes looks up at Jesus with an expression of sorrow or devotion.",
    "The background is dark and atmospheric, with subtle vertical lines suggesting rain or light rays.",
    "The composition is vertical, emphasizing the height of the cross and the emotional weight of the scene."
  ]
}

Qwen3.8-27B

{
  "medium": "painting",
  "subject": "The Crucifixion of Jesus Christ",
  "people_count": 3,
  "notes": [
    "The central figure is Jesus on the cross, depicted with a gaunt, emaciated body and a bowed head.",
    "To the left, a figure in a white hooded robe is shown in profile, looking down with hands clasped in prayer or grief.",
    "To the right, a man in a red robe looks up at Jesus, holding his foot.",
    "The overall color palette is dark and somber, dominated by browns, blacks, and muted earth tones, with the red robe providing a stark contrast.",
    "The style is dramatic and expressive, characteristic of the Baroque period."
  ]
}

Gemma 4 26B-A4B

{
  "medium": "oil on canvas",
  "subject": "The Crucifixion of Jesus Christ",
  "people_count": 3,
  "notes": [
    "The central figure is Jesus Christ on the cross.",
    "To the left is a woman, likely the Virgin Mary, draped in white and blue robes.",
    "To the right is a man, likely Saint John the Apostle, wearing a red cloak.",
    "The background is dark and dramatic, typical of Baroque art style."
  ]
}

Mistral Small 3.2 24B

{
  "medium": "Painting",
  "subject": "Religious",
  "people_count": 3,
  "notes": [
    "This image depicts a religious scene, specifically the crucifixion of Jesus Christ.",
    "The central figure is Jesus, nailed to a cross with his arms outstretched.",
    "Two other figures are present at the base of the cross, one on each side. The figure on the left appears to be a woman, possibly the Virgin Mary, and the figure on the right is a man, possibly John the Apostle.",
    "The painting uses a dark, dramatic color palette, emphasizing the somber and tragic nature of the scene.",
    "The background is dark and cloudy, adding to the overall mood of the painting."
  ]
}

Notes