IMPLEMENTATION_REPORT.md

# Phillnet Mini Text-Vision

**Build status:** Complete and validated on 21 August 2026.

This package is a deliberately lean derivative of `ayjays132/Phillnet-Mini-Omni-Max`. It retains **text generation** and **still-image understanding** only. The upstream model card identifies the checkpoint as a custom-code Transformers model and describes a separate SDXL image-generation route; this build removes that route completely rather than merely disabling it at runtime. [1]

> **Scope lock:** This package supports only language generation and visual question answering over still images. It contains no SDXL U-Net, VAE, SDXL text encoders, diffusion scheduler, text-to-image API, text-to-video API, tool runtime, agent runtime, audio endpoint, or video endpoint.

## What was downloaded and what was removed

The full original Hugging Face snapshot was mirrored before transformation. Its source inventory contained 99 files, including 18 files in the `phillnet3_sdxl/` subtree. The final lean package contains 48 files and is 1,785,035,602 bytes on disk, compared with 10,403,517,915 bytes for the complete downloaded snapshot.

| Artifact | Original snapshot | Final text-vision package | Result |
|---|---:|---:|---|
| Total package size | 10,403,517,915 bytes | 1,785,035,602 bytes | **82.84% reduction** |
| SDXL subtree | 8,608,200,761 bytes across 18 files | 0 bytes | **Removed** |
| Core `model.safetensors` | 1,763,655,304 bytes | 1,763,655,304 bytes | **Retained byte-for-byte** |
| Package files | 300 files in downloaded snapshot | 48 files | Runtime surface reduced |

The retained core checkpoint has SHA-256:

```text
f1a913f99f8ce921c1aa79982a09eaee6744448766f09a4756751e2d4b9342fc
```

That hash is identical before and after the stripping process. Therefore, the language and visual-encoder weights that reside in `model.safetensors` were not rewritten or quantized.

## Exact removal policy

The SDXL assets were not left as optional downloads. They were physically excluded from the final package together with the loading paths that referenced them. This prevents the deployment from downloading, initializing, or accidentally exposing a diffusion backend.

| Removed area | Files or behavior removed | Why it is absent |
|---|---|---|
| SDXL checkpoint | `phillnet3_sdxl/`, including two `phillnet-image-*.safetensors` shards | These shards contain the diffusion donor banks and were the primary storage cost. |
| SDXL runtime | `phillnet3_sdxl.py`, `synthesis.py`, `phillnet3_bridge.py`, `generation_acceleration.py` | They create or accelerate the SDXL U-Net, VAE, CLIP encoders, scheduler, and image/video synthesis routes. |
| Generation APIs | `generate_image`, `generate_image_prompt`, `generate_video`, `generate_video_prompt`, `synthesis_losses` | These methods were removed from `DendroForCausalLM`, not merely hidden. |
| Diffusion configuration | SDXL/diffusion-specific fields and validation from `config.json` and `configuration_dendro_omni.py` | A lean model should not advertise, validate, or accept inactive synthesis configuration. |
| Non-core runtimes | Agent, tool, orchestration, Smolagents, legacy showcase, cache, and image-generation files | They are outside the requested text-and-vision serving surface. |
| External processor declaration | Stale `Qwen3VLProcessor` declaration | Replaced by the bundled `DendroVisionProcessor` to make image input local and self-contained. |

The remaining configuration declares `model_capabilities: ["text-generation", "image-understanding"]` and `text_vision_only: true`.

## What remains and why it works

The central retained checkpoint uses a packed single-source SafeTensors layout. It has one top-level tensor rather than many separately named tensors, while `transplant_manifest.json` maps portions of that source to language and visual components. The manifest contains **153 retained `model.visual.*` mappings**. This is distinct from the deleted SDXL subtree: those mappings implement visual encoding for image understanding, whereas SDXL supplied image synthesis.

The new `processing_dendro_omni.py` is the key compatibility layer. It loads text and still images locally, normalizes an image, duplicates a still frame to satisfy the visual tower’s temporal patch dimension, constructs visual patches, inserts exactly the matching number of image placeholder tokens, and emits the `pixel_values`, `image_grid_thw`, and `mm_token_type_ids` tensors needed by the retained visual transformer. The processor is registered in both `config.json` and `preprocessor_config.json`, so `AutoProcessor.from_pretrained(..., trust_remote_code=True)` resolves to `DendroVisionProcessor`.

| Supported path | Status | Implementation |
|---|---|---|
| Text completion | Enabled | `model.generate(...)` with `reasoning_effort="direct"` for low-latency serving. |
| Text forward pass | Enabled | Standard `DendroForCausalLM.forward(...)`. |
| Still-image understanding | Enabled | `DendroVisionProcessor` plus `model.answer_image(...)` or direct multimodal model calls. |
| SDXL image generation | Removed | No weights, no module, no API, no dependency. |
| Video generation | Removed | No API, no diffusion route, no endpoint. |
| Audio / tools / agents | Not exposed | Excluded from the lean HTTP service and removed where they were optional runtime layers. |

## Verification evidence

Verification was intentionally separated into structural, load, forward, and serving checks. The checks prove that the package can load and execute the retained text and visual-input paths. They do **not** constitute a semantic-quality benchmark; output quality must be evaluated separately on representative user tasks.

| Check | Result | Evidence |
|---|---|---|
| Python syntax compilation | Passed | Lean runtime modules, processor, configuration, model, and service compiled. |
| SDXL filesystem scan | Passed | No SDXL-named source or weight artifact remains in the final package. |
| Synthesis-code scan | Passed | No `diffusers`, `StableDiffusion`, `DendroSharedDiffusion`, image/video-generation method, or SDXL adapter reference remains in Python runtime code. |
| Configuration deserialization | Passed | `DendroOmniConfig.from_pretrained(...)` loaded from the stripped directory. |
| Processor discovery | Passed | `AutoProcessor` resolves to `DendroVisionProcessor`. |
| Model loading | Passed | `AutoModelForCausalLM` instantiated `DendroForCausalLM` in BF16 on CPU. |
| Text forward pass | Passed | Finite logits with shape `[1, 27, 248320]`. |
| Image forward pass | Passed | Finite logits with shape `[1, 61, 248320]`. |
| Visual-input contract | Passed | One 32×32 test image produced one visual placeholder and four visual patches. |
| Direct text generation | Passed | Two output tokens completed in 2.224 seconds in the sandbox CPU test. |
| Local deployment health | Passed | `/health` returns exactly the two enabled capabilities and the disabled set. |
| Local text endpoint | Passed | `/v1/chat/completions` completed a direct-mode text request. |
| Local image endpoint | Passed | `/v1/chat/completions` accepted a valid base64 still image plus text question. |
| Public temporary health check | Passed | Temporary proxied endpoint returned the expected health payload. |

## Local use

Install the model dependencies from this directory, then load it using the local custom-code implementation:

```bash
cd Phillnet-Mini-Text-Vision
pip install -r requirements.txt
```

```python
import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_dir = "./Phillnet-Mini-Text-Vision"
processor = AutoProcessor.from_pretrained(model_dir, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_dir,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
).eval()

encoded = processor("Reply with one word: blue", return_tensors="pt")
output = model.generate(
    **encoded,
    max_new_tokens=32,
    reasoning_effort="direct",
    do_sample=False,
)
print(processor.tokenizer.decode(output[0], skip_special_tokens=True))
```

For still-image questions, pass OpenAI-style multimodal message content to the processor:

```python
from PIL import Image

encoded = processor.apply_chat_template(
    [{
        "role": "user",
        "content": [
            {"type": "image", "image": Image.open("example.png").convert("RGB")},
            {"type": "text", "text": "Describe the visible objects."},
        ],
    }],
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)

answer = model.generate(
    **encoded,
    max_new_tokens=128,
    reasoning_effort="direct",
    do_sample=False,
)
print(processor.tokenizer.decode(answer[0], skip_special_tokens=True))
```

## Deployment

The package contains a minimal FastAPI service in `server.py` and a `Dockerfile`. The service exposes only `/health` and `/v1/chat/completions`.

### Run directly

```bash
cd Phillnet-Mini-Text-Vision
pip install -r requirements.txt -r requirements-server.txt
uvicorn server:app --host 0.0.0.0 --port 8000
```

### Run in a container

```bash
docker build -t phillnet-mini-text-vision .
docker run --rm -p 8000:8000 phillnet-mini-text-vision
```

The package was verified on CPU using BF16 weights. Production capacity must account for the 1.76 GB checkpoint plus framework and request-memory overhead; test the target hardware under representative image sizes and concurrent-request volume before enabling unrestricted traffic.

### Health check

```bash
curl http://localhost:8000/health
```

Expected response:

```json
{
  "status": "ok",
  "capabilities": ["text-generation", "image-understanding"],
  "disabled": ["image-generation", "video-generation", "audio", "tools", "agents"]
}
```

### Text request

```bash
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  --data '{
    "messages": [{"role": "user", "content": "Give a one-sentence summary of caching."}],
    "max_tokens": 96,
    "reasoning_effort": "direct"
  }'
```

### Still-image question

The endpoint accepts `image_base64`; remote image URLs are intentionally not fetched, avoiding a server-side request-forgery surface.

```bash
IMAGE_B64=$(base64 -w0 example.png)
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  --data "{
    \"messages\": [{
      \"role\": \"user\",
      \"content\": [
        {\"type\": \"image\", \"image_base64\": \"${IMAGE_B64}\"},
        {\"type\": \"text\", \"text\": \"What is visible in this image?\"}
      ]
    }],
    \"max_tokens\": 128,
    \"reasoning_effort\": \"direct\"
  }"
```

## Temporary deployment

A verified **temporary** endpoint is currently available at:

```text
https://8000-iq1jte46c8ls1hzprn1yn-9c029be5.us5.manus.computer
```

Its health endpoint is:

```text
https://8000-iq1jte46c8ls1hzprn1yn-9c029be5.us5.manus.computer/health
```

This URL is for validation only. It is backed by the session sandbox and will not provide durable production hosting. For a persistent deployment, use the included container definition on a host with enough RAM for the retained BF16 checkpoint and configure authentication, TLS termination, logging, rate limits, and request-size limits at the deployment boundary.

## Reproducibility artifacts

The parent workspace contains the following non-runtime artifacts:

| File | Purpose |
|---|---|
| `build_text_vision_only.py` | Rebuilds the lean package from the fully downloaded original snapshot without executing remote model code. |
| `validate_text_vision_package.py` | Structural, configuration, processor, and checkpoint-inventory validation. |
| `load_text_vision_model.py` | Controlled model-load verification. |
| `run_text_vision_forward.py` | Text, image, and direct-generation forward smoke test. |
| `original_weight_sha256.txt` | Checksums captured from the complete original snapshot. |
| `SOURCE_INVENTORY.md` | Source repository inventory. |

## References

[1]: https://huggingface.co/ayjays132/Phillnet-Mini-Omni-Max "ayjays132/Phillnet-Mini-Omni-Max model card"