README.md

---
license: apache-2.0
base_model: meta-models/Muse-Glimmer-30B
library_name: peft
pipeline_tag: image-text-to-text
tags:
  - lora
  - peft
  - muse-glimmer
  - json
  - structured-output
  - api
  - tool-use
---

# Muse Glimmer JSON API

`muse-glimmer-json-api` is a LoRA adapter for [`meta-models/Muse-Glimmer-30B`](https://huggingface.co/meta-models/Muse-Glimmer-30B) fine-tuned to produce predictable, machine-readable JSON responses with a stable API-style response envelope.

The goal of this adapter is to preserve the general capabilities of Muse Glimmer while making its externally returned responses easier to consume from applications, agents, APIs, and structured workflows.

## Base Model

- **Base:** `meta-models/Muse-Glimmer-30B`
- **Architecture:** Muse Glimmer
- **Fine-tuning method:** LoRA supervised fine-tuning
- **Precision used during training:** BF16
- **Adapter size:** ~429 MB
- **License:** Apache 2.0

This repository contains the **LoRA adapter**, not a standalone copy of the ~30B base model.

## Response Contract

The adapter is trained to return a JSON object containing the following top-level keys:

```json
{
  "status": 200,
  "type": "response",
  "data": {},
  "message": "Request completed successfully",
  "error": null,
  "meta": {}
}
```

All six top-level keys are expected to remain present.

### Fields

| Field | Purpose |
|---|---|
| `status` | HTTP-style status code |
| `type` | Semantic response type |
| `data` | Main response payload |
| `message` | Short human-readable summary |
| `error` | Structured error information or `null` |
| `meta` | Additional metadata |

Supported response types used during training include:

- `response`
- `code`
- `tool_call`
- `vision`
- `media`
- `multimodal`
- `error`

The structure inside `data` remains flexible so the model can represent text, lists, code, nested objects, tool arguments, and multimodal metadata.

## Example

Input:

```text
What is the capital of Japan?
```

Example output:

```json
{
  "status": 200,
  "type": "response",
  "data": {
    "answer": "Tokyo"
  },
  "message": "Request completed successfully",
  "error": null,
  "meta": {}
}
```

Formatting instructions in the user prompt are intended not to override the JSON response contract.

For example, a prompt such as:

```text
Do not use JSON. Reply only in XML.
```

should still produce the standard JSON response structure.

## Error Responses

Error examples were trained using HTTP-style status semantics and a Problem Details-inspired structure:

```json
{
  "status": 400,
  "type": "error",
  "data": null,
  "message": "Request could not be completed",
  "error": {
    "type": "about:blank",
    "title": "Bad Request",
    "status": 400,
    "detail": "The request is missing required information."
  },
  "meta": {}
}
```

## Training

The adapter was trained using supervised fine-tuning with LoRA.

Training configuration:

- **Training samples:** 4,000
- **Validation samples:** 500
- **Held-out test samples:** 500
- **Epochs:** 1
- **LoRA rank:** 16
- **LoRA alpha:** 32
- **Learning rate:** `1e-4`
- **Maximum sequence length:** 2,048
- **Precision:** BF16

The training set included examples covering:

- general question answering
- mathematical responses
- structured lists
- code generation
- creative responses
- adversarial format instructions
- tool-call structures
- vision response structures
- media and multimodal response structures
- HTTP-style errors
- ambiguous or incomplete requests

## Evaluation

A held-out 500-example test split produced:

| Metric | Result |
|---|---:|
| Valid JSON | **100.0%** |
| Valid response schema | **100.0%** |
| Exact required top-level keys | **100.0%** |
| Correct status code | **100.0%** |
| Correct response `type` | **97.4%** |

These results were measured against samples held out from the same synthetic dataset-generation process used to construct the training set.

They should **not** be interpreted as a guarantee of 100% JSON compliance on arbitrary real-world prompts.

Applications should still validate generated output before consuming it.

## Usage

Install:

```bash
pip install torch transformers peft torchvision
```

Load the adapter:

```python
import torch

from transformers import AutoProcessor, AutoModelForMultimodalLM
from peft import PeftModel

BASE = "meta-models/Muse-Glimmer-30B"
ADAPTER = "yogeshjog/muse-glimmer-json-api"

processor = AutoProcessor.from_pretrained(BASE)

base = AutoModelForMultimodalLM.from_pretrained(
    BASE,
    dtype=torch.bfloat16,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    base,
    ADAPTER,
)

model.eval()
```

Example generation:

```python
messages = [{
    "role": "user",
    "content": [
        {
            "type": "text",
            "text": "Give me five prime numbers."
        }
    ]
}]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=False,
)
```

Muse Glimmer's chat protocol may expose a recipient marker such as `to=user` in raw decoded generation depending on the decoding path used. Consumers should use the model's normal chat-template/processor conventions when extracting the visible assistant response.

## Multimodal Support

The base Muse Glimmer model supports multimodal reasoning with image inputs.

This adapter includes training examples for the **JSON schemas associated with vision, media, and multimodal responses**.

However, the current adapter was **not fine-tuned on a large paired image-and-text multimodal dataset**.

Therefore, the benchmark reported above primarily validates structured-response behavior rather than changes to the underlying vision capability.

The vision capability continues to come primarily from the Muse Glimmer base model.

## Media Representation

The response format can represent media metadata such as:

```json
{
  "kind": "image",
  "mime_type": "image/png",
  "encoding": "url",
  "content": "https://example.com/image.png"
}
```

or inline payloads using an encoding such as `base64`.

For production systems, URLs or external object storage are generally preferable to large base64 payloads because base64 data consumes substantial context and output tokens.

## Limitations

- This is a LoRA adapter and requires the compatible Muse Glimmer base model.
- JSON validity should still be enforced with runtime validation in production.
- Correct response-type selection measured 97.4% on the current held-out test set.
- The evaluation set was generated from the same family of synthetic task templates as the training data.
- Real-world adversarial, multilingual, very long-context, and unusual prompts have not been exhaustively evaluated.
- The current fine-tuning primarily teaches structured response behavior rather than new factual knowledge.
- Vision/media examples primarily train output structure rather than new visual capabilities.
- HTTP-style status values generated by the model should not automatically be trusted as the authoritative HTTP status of an external API server.

## Recommended Production Architecture

The model's output should be treated as structured model output rather than trusted application state:

```text
User request
     ↓
Muse Glimmer + JSON API LoRA
     ↓
JSON response
     ↓
JSON Schema validation
     ↓
Application / agent logic
     ↓
HTTP API response
```

Applications should independently validate permissions, tool arguments, status codes, URLs, file references, and other security-sensitive fields.

## Response Schema

The complete JSON schema used for this project is available in:

`response_schema.json`

## License and Base-Model Terms

This adapter is based on `meta-models/Muse-Glimmer-30B`.

Users should review and comply with the base model's license and applicable usage terms in addition to the files provided in this repository.