Miril-Drone-2B-1

An open-weight aerial vision-language model for civilian drones

**Now drones can think and talk :) **

Enabling drones to reason and talk about their environment.

Miril-Drone-2B-1 is a 2B-class aerial VLM by Miril.ai for drone-view imagery. Give it an overhead frame and a clear instruction, and it returns a structured JSON response: a scene caption, a direct visual answer, or a rough operational point for review.

Civilian drones are becoming everyday infrastructure: delivery, first response, agriculture, construction, inspection, mapping, disaster recovery, and infrastructure monitoring. Those workflows need more than video. They need models that can explain what is in the frame, answer questions about the ground below, and produce simple machine-readable outputs that operators and downstream software can inspect.

This is V1 of Miril-Drone. It is built to show that small edge-oriented VLMs can make aerial systems easier to supervise, scale, and make safer.

This is what it does!

If the embedded player does not appear, open the demo compilation video directly.

Disclaimer: this is not (yet) a safety system!

This is mostly just a demo for now: it's a very small model trained on a sub-sampling of the WALDO dataset. It's interesting to show because it points in the direction of what drones will be able to do in the future, but this should by no means be considered a production-ready safety system. We will get there, but this is just pointing in that direction ;) It's also still plain wrong quite a lot! The model is based on Gemma4-2B and while the vision backbone is surprisingly strong for a model so small it's far from the perception capabilities we would need to make this perform perfectly on generalized use cases. Right now for instance small things in drone-imagery like people seen from a distance are lost most of the time, and the VLM only recovers them at lower altitudes.

TL;DR

Miril-Drone-2B-1 is an open-weight aerial VLM for:

• drone-view scene descriptions,
• aerial visual question answering,
• visible object and ground-context review,
• delivery-zone and landing-area screening,
• first-response and inspection triage,
• structured JSON outputs for operator tools and autonomy workflows.

The model accepts an image plus a visible text prompt. It does not receive hidden object counts, bounding boxes, or schema metadata at inference time. If you want JSON, put the JSON requirement in the prompt.

Architecture note: the exported checkpoint inherits Gemma4 multimodal components, including audio and video tokens/processors/towers. They are kept so the checkpoint remains a faithful merged Gemma4 model. This Miril-Drone release, however, was trained and validated for drone-view image-plus-text use. Audio input and native temporal video understanding are not part of the current release contract; the demo video path samples still frames and runs image inference on those frames.

Try the demo Space:

https://huggingface.co/spaces/MirilAI/Miril-Drone-2B-1-Demo

The V1 Pointing Contract

V1 supports operational pointing with rough representative grid cues.

That wording matters. Coordinates are image-relative values in a 0..1000 frame, and many V1 pointing outputs should be read as a coarse review cue, not as a certified landing point, delivery command, or closed-loop control target.

Use V1 pointing for:

• asking where a human should look first,
• highlighting a candidate open area,
• screening an image before another planner or operator takes over,
• feeding a safety-aware review interface.

Do not use V1 pointing as:

• an autonomous flight command,
• a guaranteed safe landing point,
• a centimeter-level or pixel-perfect localization system.

V2 is coming soon in the next generation of the model and is intended to add finer pointing behavior, clearer target categories, and stronger validation around operational coordinates :)

What It Can Do

Use Miril-Drone-2B-1 when you want a small aerial model to turn a drone frame into a usable answer:

• "Describe the drone image."
• "How many cars are visible?"
• "Are people clearly visible?"
• "What areas should be avoided for landing or dropping something?"
• "What would be the safest place to land a drone in this image?"

The answer is returned as bare JSON so it can be consumed by dashboards, review tools, notebooks, routing systems, or VLA-style autonomy stacks.

WALDO Lineage

Miril-Drone-2B-1 is not named WALDO, but it inherits aerial vocabulary from the WALDO perception line.

WALDO30 is an overhead object detector for drone-relevant imagery. Its class vocabulary shaped the way this model talks about aerial scenes:

WALDO class	What it covers in plain English
LightVehicle	Cars, sedans, rickshaws, small personal vehicles, vans, and similar light road vehicles.
Person	People and human figures visible from above.
Building	Buildings, sheds, shacks, pergolas, canopies, tents, and other roofed structures.
UPole	Utility poles, telegraph poles, power-line fixtures, lamp posts, street fixtures, satellite dishes, antennas, and thin truss-like structures.
Boat	Boats, ships, canoes, kayaks, surfboards, and watercraft-like objects.
Bike	Bicycles, motorbikes, mopeds, scooters, and other two-wheeled vehicles.
Container	Storage containers, including shipping containers.
Truck	Trucks, transport vehicles, articulated trucks, semi trucks, flatbeds, and larger industrial road vehicles.
Gastank	Gas tanks, storage tanks, liquid tanks, cylindrical storage containers, and silo-like tanks.
Digger	Diggers, tractors, excavators, and construction vehicles.
Solarpanels	Solar panel arrays and individual panel groups.
Bus	Buses and bus-like passenger vehicles.

The VLM does not output detector boxes by default. It uses this aerial vocabulary to answer natural drone-view questions.

Prompting

The model was trained for image-plus-instruction calls. Keep prompts plain and close to the task. Do not add brand language or lineage language to the prompt; keep that context in documentation.

Use this system prompt for JSON-producing calls:

You are a careful visual assistant for drone imagery. Answer only with valid bare JSON. Do not use Markdown, prose outside JSON, or unrequested keys.

Then send the image with one of these user prompt families.

caption_v1

Use this for a compact whole-frame description.

Describe the drone image.

Required response format: return only valid bare JSON with exactly one key, {"caption": string}. Do not use Markdown or prose outside the JSON object.

Expected output:

{"caption": "Overhead view of a residential block with roads, parked vehicles, buildings, trees, and open ground areas."}

simple_answer_v1

Use this for visible objects, counts, scene type, locations, hazards, and operator questions.

How many light vehicles are visible in the image?

Required response format: return only valid bare JSON with exactly one key, {"answer": string}. Do not use Markdown or prose outside the JSON object.

Expected output:

{"answer": "Several light vehicles are visible along the road and in parking areas."}

operational_coordinate_v2

Use this for rough landing, delivery, inspection, or review cues. Bear in mind the output is rough coordinates and is only meant as an indication for now. The next generation will be able to point more accurately, and we are training larger versions of this model too which will be more accurate! This is foremost meant to showcase how a drone could reason by itself about things like landing spots for commercial drone-delivery applications!

What would be the safest place to land a drone in this image?

Required response format: return only valid bare JSON with exactly these keys: "coordinate_system", "description", "point_2d", "point_semantics", "status", "x", and "y". "description", "point_semantics", and "status" must be strings. "coordinate_system" must be "gemma_relative_0_1000_yx" or null. "point_2d" must be [y, x] with numbers from 0 to 1000, or null. "x" and "y" must be numbers from 0 to 1000, or null. "point_semantics" must be one of "specific_point", "representative_point", or "not_applicable". "status" must be one of "target_found", "conditional_candidate", or "no_safe_area". If you describe any candidate landing, delivery, or review area, you must point to it with approximate coordinates: use "coordinate_system": "gemma_relative_0_1000_yx", numeric "x" and "y", "point_2d": [y, x], and "point_semantics": "representative_point" unless the prompt asks for a specific object point. Use "target_found" for a clearly usable target and "conditional_candidate" for a plausible but uncertain target. Use "no_safe_area" only when no meaningful target area can be pointed to; then set "coordinate_system", "point_2d", "x", and "y" to null and set "point_semantics" to "not_applicable". Never omit required keys. Candidate skeleton: {"coordinate_system": "gemma_relative_0_1000_yx", "description": string, "point_2d": [500, 500], "point_semantics": "representative_point", "status": "conditional_candidate", "x": 500, "y": 500}. No-safe skeleton: {"coordinate_system": null, "description": string, "point_2d": null, "point_semantics": "not_applicable", "status": "no_safe_area", "x": null, "y": null}. Do not use Markdown or prose outside the JSON object.

Coordinate interpretation:

• x=0 is the left edge of the image.
• x=1000 is the right edge.
• y=0 is the top edge.
• y=1000 is the bottom edge.
• point_2d is [y, x].
• x and y duplicate the coordinate for easier downstream parsing.

Drawable V1 example:

{
  "coordinate_system": "gemma_relative_0_1000_yx",
  "description": "Open grass near the right side of the frame, away from visible roofs, trees, vehicles, and roads.",
  "point_2d": [500, 750],
  "point_semantics": "representative_point",
  "status": "target_found",
  "x": 750,
  "y": 500
}

No-area example:

{
  "coordinate_system": null,
  "description": "No clearly safe landing area is visible because roads, roofs, trees, and constrained spaces dominate the frame.",
  "point_2d": null,
  "point_semantics": "not_applicable",
  "status": "no_safe_area",
  "x": null,
  "y": null
}

Example Outputs

Scene Explanation

Prompt:

Describe the drone image.

Output:

{"caption": "Overhead view of a residential area with pitched-roof buildings, parked cars, narrow roads, garden plots, trees, and several open grassy sections."}

Visual Question Answering

Prompt:

What areas should be avoided for landing or dropping something in this scene?

Output:

{"answer": "Avoid rooftops, roads, parked vehicles, tree cover, utility fixtures, narrow yards, and areas close to buildings or fences."}

Operational Pointing

Prompt:

What would be the safest place to land a drone in this image?

Output:

{
  "coordinate_system": "gemma_relative_0_1000_yx",
  "description": "A relatively open grassy area away from the main road, buildings, parked cars, and dense tree cover.",
  "point_2d": [750, 500],
  "point_semantics": "representative_point",
  "status": "conditional_candidate",
  "x": 500,
  "y": 750
}

Read this as a rough review cue. A downstream UI may draw a marker there, but a real drone system still needs separate safety checks, geofencing, obstacle sensing, map context, and human or certified autonomy approval.

Quickstart

from PIL import Image
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "MirilAI/Miril-Drone-2B-1"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

image = Image.open("drone_frame.jpg").convert("RGB")
prompt = '''Describe the drone image.

Required response format: return only valid bare JSON with exactly one key, {"caption": string}. Do not use Markdown or prose outside the JSON object.'''

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": prompt},
        ],
    }
]

rendered = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[rendered], images=[[image]], return_tensors="pt", padding=True).to(model.device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=192, do_sample=False)

generated = output[:, inputs["input_ids"].shape[-1]:]
print(processor.batch_decode(generated, skip_special_tokens=True)[0])

Variants

Related deployment variants (derived from this model):

• Miril-Drone-2B-1-bnb4: CUDA 4-bit bitsandbytes path.
• Miril-Drone-2B-1-bnb8: CUDA 8-bit bitsandbytes path.
• Miril-Drone-2B-1-MLX-4bit: Apple Silicon MLX 4-bit path.
• Miril-Drone-2B-1-MLX-8bit: Apple Silicon MLX 8-bit path.

Safety Notes

Miril-Drone-2B-1 is a perception and reasoning model. It is not a flight controller, autopilot, certified safety system, or substitute for operational validation.

Use it as one component in a larger stack:

• keep a human or validated planner in the loop,
• independently check obstacles, terrain, people, vehicles, trees, wires, and weather,
• enforce geofencing and vehicle limits,
• log model outputs and failures,
• treat invalid JSON, null coordinates, and no_safe_area as no-target outcomes.

The model can be wrong. It can miss small objects, confuse scale, overstate open areas, or produce a coordinate that only makes sense as a rough review marker.

Intended Use

Good fits:

• aerial image review,
• drone operator assistance,
• field inspection triage,
• first-response situational awareness,
• delivery or landing-area pre-screening,
• research on aerial VLMs and edge-oriented perception,
• structured-output demos for physical AI workflows.

Not good fits:

• direct closed-loop flight control,
• safety-critical operation without independent validation,
• surveillance or identification of people,
• claims of certified landing-zone selection.

License

Apache License 2.0. See LICENSE and NOTICE.

This model is derived from google/gemma-4-E2B-it; users should also review the upstream base model terms.