Singularity87/Cityscapes-Qwen3.5-4B

Cityscapes Qwen3.5 4B — merged BF16 safetensors

This is a fully merged Transformers checkpoint of unsloth/Qwen3.5-4B, fine-tuned to extract structured Cityscapes road-scene facts from an image. It predicts:

• presence for 2 surface classes and 11 object classes;
• a 3×3 image zone set for each present class;
• count buckets for the 11 object classes;
• conservative subject on road|sidewalk contact relations.

Both the language model and vision tower LoRA weights are merged into this checkpoint. It is not an adapter repository and does not require PEFT or the original LoRA at inference time.

The model is specialized for this fixed Cityscapes JSON task. It is not a general object detector and the evaluation below does not imply a general vision-language improvement.

Files

The model is stored as two safe-serialized shards:

File	Size	SHA-256
model.safetensors-00001-of-00002.safetensors	5,329,398,688 bytes	0acfb9ce10a5057b9f7ef85a83d41e3999cb608032594d7e9e5a557c8cdc4b03
model.safetensors-00002-of-00002.safetensors	3,990,429,408 bytes	b521e3cf44161c13e23878c5b416fb1f24613d3428f2981bd440a94dc7568c1b

There are 738 model tensors: 690 BF16 and 48 F32. The F32 tensors are the base architecture's numerically sensitive parameters, not unmerged LoRA weights. See release_manifest.json for machine-readable provenance, checksums, package versions, and merge validation.

Quick start with Transformers

Use an explicit schema prompt. The input should place the image before the question, matching training and evaluation.

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "Singularity87/Cityscapes-Qwen3.5-4B"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

image = Image.open("frankfurt_000000_000294_leftImg8bit.png").convert("RGB")

system_prompt = """You extract structured Cityscapes facts from an image.
Return raw JSON only with exactly the top-level keys surfaces, objects, relations.
surfaces must contain road and sidewalk with present and zones.
objects must contain person, rider, car, truck, bus, train, motorcycle,
bicycle, traffic_light, traffic_sign, and pole with present, count, and zones.
Valid zones are upper_left, upper_center, upper_right, middle_left,
middle_center, middle_right, lower_left, lower_center, lower_right.
Valid counts are 0, 1, 2-3, 4-7, 8+, unknown. An absent object is
{"present":false,"count":"0","zones":[]}. Relations use subject, relation,
object; relation is on and object is road or sidewalk."""

messages = [
    {"role": "system", "content": system_prompt},
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {
                "type": "text",
                "text": (
                    "Analyze this urban road image and return one completed "
                    "JSON object in the required schema."
                ),
            },
        ],
    },
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)
inputs = {key: value.to(model.device) for key, value in inputs.items()}

with torch.inference_mode():
    generated = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=False,
    )

new_tokens = generated[:, inputs["input_ids"].shape[1]:]
print(processor.batch_decode(new_tokens, skip_special_tokens=True)[0])

What was done

Data

• Cityscapes fine annotations and left images were converted into an image-first supervised JSON task.
• Training split: 2,975 images.
• Validation split: 500 images.
• Cityscapes images and annotations are not redistributed in this model repository.

The output contract uses:

• surfaces: road, sidewalk;
• objects: person, rider, car, truck, bus, train, motorcycle, bicycle, traffic_light, traffic_sign, pole;
• zones: a 3×3 grid from upper_left through lower_right;
• count buckets: "0", "1", "2-3", "4-7", "8+", "unknown";
• relations: one of the eight instance classes, relation "on", and surface road or sidewalk.

Training

This was 16-bit LoRA SFT, not QLoRA. The base model was loaded in BF16 with neither 4-bit nor 8-bit base quantization. Loss was calculated on assistant tokens only. Images retained Cityscapes resolution through the Unsloth resize="max" vision collator behavior.

Setting	Value
Base revision	3764fa359b9082ea5a1e4a5e3ac3aaf6e9671636
Epochs / optimizer steps	1 / 372
Train / eval batch size	8 / 4
Gradient accumulation	1
Precision	BF16, TF32 enabled
LoRA rank / alpha / dropout	16 / 16 / 0
LoRA scope	language + vision; attention + MLP
Learning rate	1e-4
Scheduler / warmup	cosine / 5%
Optimizer / weight decay	AdamW Torch / 0.001
Maximum sequence length	4,096
Seed	3,407
Hardware	NVIDIA RTX 6000 Ada Generation, 48 GB
Train runtime	5,018.83 seconds

The training command in the source project was equivalent to:

python -m scripts.training.train_qwen35_cityscapes_lora \
  --model-name unsloth/Qwen3.5-4B \
  --epochs 1 \
  --batch-size 8 \
  --eval-batch-size 4 \
  --gradient-accumulation-steps 1 \
  --learning-rate 1e-4 \
  --max-length 4096 \
  --lora-rank 16 \
  --lora-alpha 16 \
  --lora-dropout 0 \
  --assistant-only-loss \
  --finetune-vision-layers \
  --finetune-language-layers \
  --finetune-attention-modules \
  --finetune-mlp-modules \
  --seed 3407

Full language-and-vision merge

The adapter contained 688 tensors: 496 language tensors and 192 vision tensors. A llama.cpp runtime --lora would not apply the vision part, so the complete adapter was merged first:

import torch
from unsloth import FastVisionModel

model, processor = FastVisionModel.from_pretrained(
    model_name="final_adapter",
    max_seq_length=4096,
    dtype=torch.bfloat16,
    load_in_4bit=False,
    load_in_8bit=False,
    load_in_16bit=True,
    use_gradient_checkpointing=False,
)
model.save_pretrained_merged(
    "Cityscapes-Qwen3.5-4B",
    processor,
    save_method="merged_16bit",
    safe_serialization=True,
    max_shard_size="4GB",
)

Header validation found 738 merged tensors: 441 language and 297 vision, with zero tensor names containing lora_ and no adapter_config.json.

How it was evaluated

The formal comparison used the BF16 GGUF export of this same merged checkpoint against the untouched BF16 base model under identical llama.cpp settings:

• all 500 Cityscapes validation images;
• OpenAI-compatible /v1/chat/completions;
• the same explicit schema system prompt and one text-only formatting example for both models;
• image-first request;
• temperature=0, seed=3407, max_tokens=1024;
• thinking disabled;
• no JSON grammar and no response_format;
• 10,000 paired bootstrap samples with seed 3,407.

task_score is the equal-weight mean of presence macro F1, per-class count accuracy, zone micro F1, and relation micro F1.

Results

Strict all-sample scoring:

Metric	Base BF16	Tuned BF16	Delta
JSON valid rate	0.292	1.000	+0.708
Strict schema valid rate	0.214	1.000	+0.786
Task score	0.1840	0.8054	+0.6214

To separate formatting gains from semantic gains, a diagnostic removed only a single whole-response Markdown JSON fence and then selected the 402/500 rows where both outputs passed the same strict schema:

Metric	Base BF16	Tuned BF16	Delta	Paired-bootstrap 95% CI
Presence macro F1	0.8110	0.8989	+0.0879	[0.0619, 0.1156]
Count macro accuracy	0.7280	0.8211	+0.0932	[0.0821, 0.1043]
Zone micro F1	0.3495	0.8447	+0.4953	[0.4857, 0.5046]
Relation micro F1	0.5711	0.6513	+0.0802	[0.0606, 0.1001]
Task score	0.6149	0.8040	+0.1891	[0.1790, 0.2000]

On the stricter unnormalized joint-valid subset (107/500), task score improved from 0.6434 to 0.8134; the delta 95% CI was [0.1494, 0.1918]. The predefined pass gate was satisfied. The largest semantic gain was 3×3 zone localization.

Validation loss at the end of training was 0.0446758; this was not used by itself as evidence that the tuned model beat the base model.

Reproducing the paired evaluation

With a base BF16 server on port 8080 and this tuned BF16 model on port 8081:

python -m scripts.evaluation.evaluate_cityscapes_llamacpp \
  --base-url http://127.0.0.1:8080 \
  --tuned-url http://127.0.0.1:8081 \
  --eval-file data/cityscapes-agent-sft/cityscapes_agent_sft_val.jsonl \
  --prompt-profile explicit-schema \
  --seed 3407 \
  --bootstrap-samples 10000

The exact evaluation prompt SHA-256 was b7b80a3b18578acb8406af229392f9d5585563ecbace3aac724145f2d8187a3f.

Limitations

• Results apply only to the fixed Cityscapes structured JSON task.
• Counts are buckets, not exact detections.
• Relations intentionally cover only direct on road|sidewalk contact.
• The model can still hallucinate or miss small/occluded objects.
• Use of this checkpoint is restricted to non-commercial purposes, and citing the Cityscapes Dataset is a condition of that use. See License and Citation.

Software

• PyTorch 2.10.0
• Transformers 5.5.0
• PEFT 0.19.1
• Safetensors 0.8.0
• TRL 0.24.0
• Unsloth 2026.7.2

License

The effective terms for this checkpoint are the intersection of two licenses, and that intersection is non-commercial.

• The base model unsloth/Qwen3.5-4B is Apache-2.0.
• The Cityscapes Dataset License also applies, because these fine-tuned weights are a derivative work of the dataset. It states that you may not use the dataset or any derivative work for commercial purposes, such as licensing or selling the data, or using the data with a purpose to procure a commercial gain.

This repository is therefore not labelled Apache-2.0. Apache-2.0 on its own would grant commercial rights that the Cityscapes terms withhold. If you need commercial use, take that up with the Cityscapes authors rather than relying on the base model's license.

The Cityscapes license does permit distributing abstract derivative works such as trained models, provided they do not allow the dataset to be recovered, which is what makes publishing this checkpoint possible. It does not permit redistributing the dataset itself, so no Cityscapes images or annotations are included here.

Citation

Referencing the Cityscapes Dataset in any work that uses this model is a condition of the dataset license, not a courtesy.

@inproceedings{Cordts2016Cityscapes,
  title     = {The Cityscapes Dataset for Semantic Urban Scene Understanding},
  author    = {Cordts, Marius and Omran, Mohamed and Ramos, Sebastian and
               Rehfeld, Timo and Enzweiler, Markus and Benenson, Rodrigo and
               Franke, Uwe and Roth, Stefan and Schiele, Bernt},
  booktitle = {Proc. of the IEEE Conference on Computer Vision and Pattern
               Recognition (CVPR)},
  year      = {2016}
}