Hugging Face's logo

freeai-org
/
Scalpel-VL-1.6B

Safetensors
English
Chinese
qwen3_vl
Model card Files
xet
 Community

🐱Scalpel-VL-1.6B: A Lightweight Multimodal Vision Model for Feline Posture and Emotion Recognition

Table of Contents

Scalpel-VL overall architecture preview

✨Project Overview

Artificial intelligence should empower not only human beings but also companion animals. Scalpel-VL is a pioneering multimodal vision model tailor-made for precise recognition of feline behaviors, body postures and emotional states, equipped with robust domain-specific conversational capabilities optimized for cat scenarios. Evaluated on our dedicated feline behavior benchmark dataset, the model outperforms Qwen3.5-9B (a model with five times larger parameters) by nearly 10% in overall recognition accuracy.

In terms of inference performance:

  1. Scalpel-VL-1.6B achieves 40% faster inference speed than Qwen3-VL-2B with similar parameter magnitude;
  2. Its end-to-end inference latency is comparable to Qwen3.5-0.8B, a dense model with half its parameter count;
  3. Within models of equivalent parameter scale, Scalpel-VL-1.6B establishes state-of-the-art performance across both prediction accuracy and inference latency.

📚FelineBench: Custom Evaluation Dataset for Cat Recognition

Cat behavior recognition faces notable obstacles, including subtle visual gaps between different body languages, unstable ambient lighting, highly variable cat poses, ambiguous ear and tail motions, and diverse fur colors and textures. FelineBench is a rigorously curated evaluation dataset with fine-grained multi-dimensional human annotations to enable standardized, fair benchmarking across multimodal models. Complete quantitative metrics are displayed below.

FelineBench benchmark visualization

Note： The dataset is not open-source yet.

🔬Feline Posture Recognition Quantitative Results

Format Explanation: Bold = best performance in the column; Italic = second-best performance in the column

Model Weight Type Parameter Size Action Body Ear Tail Face Fur Average Posture Score (Calibrated)
Llama-3-LLaVA-NeXT open 8B 77.40 5.34 40.21 0.00 0.00 0.00 20.49
Qwen3-VL-2B open 2B 82.45 27.63 43.97 42.80 29.57 25.68 42.02
MiniCPM-V-4.5 open 8B 87.26 71.68 34.77 49.10 24.37 63.80 55.16
Qwen3.5-9B open 9B 88.22 53.51 46.86 84.87 58.30 85.98 69.62
Scalpel-VL-1.6B open 1.6B 85.82 76.49 66.53 92.03 69.72 89.64 80.04

🔬Cat Detection & Counting Accuracy

Model Weight Type Parameter Size Cat Detection Count Accuracy
Llama-3-LLaVA-NeXT open 8B 100.00 80.29
Qwen3-VL-2B open 2B 95.19 84.62
MiniCPM-V-4.5 open 8B 100.00 92.07
Qwen3.5-9B open 9B 100.00 90.62
Scalpel-VL-1.6B open 1.6B 100.00 89.42

🔬General Multimodal QA Capability Benchmark

While optimized intensively for feline visual understanding, Scalpel-VL-1.6B retains robust general-purpose multimodal capabilities. It supports common cross-modal tasks including general image comprehension, open-domain knowledge QA, scene parsing and natural language captioning, balancing domain specialization and universal applicability for daily cat-raising scenarios.

Inference Speed Benchmark Comparison

All speed tests are executed under unified hardware configurations:

  1. Scalpel-VL-1.6B delivers 40% higher inference throughput compared with Qwen3-VL-2B;
  2. Its latency matches Qwen3.5-0.8B, enabling high performance with low computation overhead;
  3. The lightweight design fits local deployment, edge computing hardware and real-time streaming cat monitoring via home cameras.

📖Standard Inference Prompt & JSON Output Example

Inference Prompt

You are a professional feline visual recognition specialist with two core tasks: 1. Confirm whether cats exist in the frame and count all visible cats; 2. Describe each cat’s surrounding environment, body posture, physical features (face, ears, tail) and behavioral actions in full detail.

Strictly follow these output rules:

  1. Output pure JSON text only. No explanations, Markdown, code blocks or redundant content allowed.
  2. The output JSON must fully comply with the given Schema; no fields can be added or omitted.
  3. Fill empty entries with 0/false/null/empty strings; all keys must carry valid values with no vacancies.
  4. Clearly distinguish holistic body posture (squatting / side lying / prone / curled up, etc.) from behavioral actions (sleeping / eating / grooming, etc.).
  5. Supplement secondary details including ambient lighting, surrounding creatures, objects and abnormal elements within the frame.
  6. Limit the final summary field within 30 characters to summarize all cats’ conditions concisely.

Mandatory JSON Schema:

{
  "schema_version": "v1.12",
  "cats_visible": "int >= 0",
  "lighting": "Bright | Moderate | Dim | Very Dark",
  "other_beings": [
    {
      "being_id": "human_a | dog_a | ...",
      "being_type": "Human | Dog | Bird | Other Animal",
      "description": "string, 15-25 words describing appearance with age hint (e.g. 'adult male with black hair and short sleeves')"
    }
  ],
  "cats": [
    {
      "cat_id": "cat_a | cat_b | ...",
      "location_on": "Ground | Bed | Sofa | Chair | Desktop | Cabinet Surface | Windowsill | Cat Tree | Cat Bed | Human Lap | Suspended | Other | Unknown",
      "vertical_position": "Ground Level | Low Level | Middle Level | High Level | Suspended | Unknown",
      "nearby_anchors": [
        {
          "anchor_type": "Door | Window | Litter Box | Food Bowl | Water Bowl | Cat Tree | Scratching Post | Sofa | Bed | Table | Home Appliance | Other",
          "visual_id": "string, 15-30 words describing color/material/shape for cross-frame re-identification",
          "proximity": "Contact | Adjacent | Nearby"
        }
      ],
      "nearby_beings": [
        {
          "being_id": "human_a | dog_a | ...",
          "proximity": "Contact | Adjacent | Nearby"
        }
      ],
      "action": "Sleeping | Stationary | Active | Jumping | Eating | Drinking | Grooming | Using Litter Box | Vomiting | Other | Unknown",
      "attention_to": {
        "target_type": "Cat | Creature | null",
        "target_id": "Matching ID | null"
      },
      "posture": {
        "overall_body": "Squatting | Side Lying | Prone | Standing | Arched Back | Raised Hip | Curled Into Ball | Stretched Out | Half-Risen | Suspended | Other | Unknown",
        "ears": {
          "visible": "bool",
          "position": "Upright Neutral | Forward Tilted | Flattened Sideways | Fully Pressed Back | Unilateral Abnormal | Unknown"
        },
        "tail": {
          "visible": "bool",
          "position": "High Raised | Question Mark Curve | Slightly Curved Up | Horizontal | Hanging Down | Tucked Between Legs | Puffed Up | Coiled | Unknown"
        },
        "face": {
          "visible": "bool",
          "eyelid": "Fully Open | Half Closed/Squinting | Fully Shut | Unknown",
          "mouth": "Closed | Slightly Open | Open With Exposed Teeth | Open With Tongue Out | Licking | Unknown"
        },
        "fur_state": "Normal | Puffed Up"
      },
      "abnormalities": [
        {
          "abnormality_tag": "Visible Wound | Suspected Blood Stain | Stained By Excreta | Local Bald Patch | Wet Fur | Messy Fur | Chewing/Holding Foreign Object | Other",
          "visual_description": "string within 15 words",
          "severity_hint": "Monitor Required | Obvious Abnormality | Unknown"
        }
      ],
      "is_partially_occluded": "bool"
    }
  ],
  "interactions": [
    {
      "presence": [
        {
          "type": "Cat | Creature",
          "id": "Matching ID"
        },
        {
          "type": "Prop",
          "prop_type": "Cat Toy | Wand Toy | Laser Spot | Rope/String | Plant | Food | Hazardous Object | Other",
          "visual_id": "string, 15-30 words describing appearance for cross-frame re-identification"
        },
        {
          "type": "Anchor",
          "anchor_type": "Door | Window | Litter Box | Food Bowl | Water Bowl | Cat Tree | Scratching Post | Sofa | Bed | Table | Home Appliance | Other",
          "visual_id": "string, 15-30 words describing appearance for cross-frame re-identification"
        }
      ],
      "interaction_type": "Head Rubbing | Leg Rubbing | Nose Touching | Kneading | Mutual Licking | Side-by-Side Resting | Pouncing | Pawing | Scratching | Biting | Staring Down | Chasing | Physical Contact | Other"
    }
  ],
  "environment_anomalies": [
    {
      "anomaly_tag": "Broken Glass | Suspected Blood Stain | Liquid Spill | Overturned Object | Accessible Hazard | Feces Outside Litter Box | Ground Vomit | Other",
      "visual_description": "string within 15 words",
      "location_in_frame": "Left Frame | Right Frame | Center Frame | Frame Edge | Multiple Spots",
      "severity_hint": "Monitor Required | Obvious Abnormality | Unknown"
    }
  ],
  "summary": "string within 30 characters"
}


Output pure JSON only. Strictly separate holistic body posture (squatting/side lying/prone/curled etc.) and behavioral actions (sleeping/eating/grooming etc.). No fields can be added or removed.

🔖Standard JSON Output Sample 

{"schema_version": "v1.12", "cats_visible": 1, "lighting": "Bright", "other_beings": [], "cats": [{"cat_id": "cat_a", "location_on": "Ground", "vertical_position": "Low Level", "nearby_anchors": [{"anchor_type": "Water Bowl", "visual_id": "matte white ceramic round bowl filled with clear tap water", "proximity": "Contact"}], "nearby_beings": [], "action": "Drinking", "attention_to": {"target_type": null, "target_id": null}, "posture": {"overall_body": "Standing", "ears": {"visible": true, "position": "Upright Neutral"}, "tail": {"visible": false, "position": "Unknown"}, "face": {"visible": true, "eyelid": "Half Closed/Squinting", "mouth": "Licking"}, "fur_state": "Normal", "abnormalities": [], "is_partially_occluded": false}], "interactions": [{"presence": [{"type": "Anchor", "anchor_type": "Water Bowl", "visual_id": "matte white ceramic round bowl filled with clear tap water"}], "interaction_type": "Other"}], "environment_anomalies": [], "summary": "Tortoiseshell cat lowers head drinking water from ceramic bowl"}

🚕Quick Start 

import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

MODEL_PATH = "Wuyuhang11/Scalpel-VL-1.6B"  # or your local model path
IMAGE_PATH = "cat.jpg"

PROMPT = Inference Prompt

processor = AutoProcessor.from_pretrained(
    MODEL_PATH,
    trust_remote_code=True,
)

model = Qwen3VLForConditionalGeneration.from_pretrained(
    MODEL_PATH,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
).eval()

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": IMAGE_PATH},
            {"type": "text", "text": PROMPT},
        ],
    }
]

template_kwargs = dict(
    messages=messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)

try:
    inputs = processor.apply_chat_template(
        **template_kwargs,
        enable_thinking=False,
    )
except TypeError:
    inputs = processor.apply_chat_template(**template_kwargs)

inputs = inputs.to(model.device)

with torch.inference_mode():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=2048,
        do_sample=False,
    )

generated_ids = generated_ids[:, inputs["input_ids"].shape[1]:]
response = processor.batch_decode(
    generated_ids,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)[0]

print(response.strip())

💁Author Profile & FreeAI Open Community

Author Introduction

I am Han Li, a full-time postgraduate student majoring in Computer Science. I have taken part in multiple industrial research projects focused on large multimodal models and published several first-author academic papers. Publication quantity is not my primary pursuit; I focus on iterative optimization of practical, deployable AI technologies.

FreeAI Open Community

If you are interested in this research and related projects, feel welcome to join our open community FreeAI. FreeAI is an open, inclusive hub for AI communication, technical discussion and collaborative development.

Long-Term Vision

Our ambition extends beyond building production-ready AI models for industry usage. We aim to democratize AI deployment across diverse vertical industries, allowing human beings, companion animals and all tangible entities to benefit from AI advancements. This is the core mission of FreeAI.

License

Apache 2.0 — see the LICENSE file for details.

🖱️ Reference 

@misc{wu2026soulgardvl2bvisionlanguagemodeledgebased,
      title={SoulGard-VL-2B: A Vision-Language Model for Edge-Based Feline Behavior Understanding}, 
      author={YuHang Wu and HaoXian Liu and Jia Tao},
      year={2026},
      eprint={2608.22070},
      archivePrefix={arXiv},
      primaryClass={cs.CE},
      url={https://arxiv.org/abs/2608.22070}, 
}
Downloads last month
4
Safetensors
Model size
2B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for freeai-org/Scalpel-VL-1.6B