Updated Readme

03dad7e verified 1 day ago

8.01 kB

license: apache-2.0
base_model: meta-models/Muse-Glimmer-30B
library_name: peft
pipeline_tag: image-text-to-text
tags:
  - lora
  - peft
  - muse-glimmer
  - json
  - structured-output
  - api
  - tool-use

Muse Glimmer JSON API

muse-glimmer-json-api is a LoRA adapter for meta-models/Muse-Glimmer-30B fine-tuned to produce predictable, machine-readable JSON responses with a stable API-style response envelope.

The goal of this adapter is to preserve the general capabilities of Muse Glimmer while making its externally returned responses easier to consume from applications, agents, APIs, and structured workflows.

Base Model

Base: meta-models/Muse-Glimmer-30B
Architecture: Muse Glimmer
Fine-tuning method: LoRA supervised fine-tuning
Precision used during training: BF16
Adapter size: ~429 MB
License: Apache 2.0

This repository contains the LoRA adapter, not a standalone copy of the ~30B base model.

Response Contract

The adapter is trained to return a JSON object containing the following top-level keys:

{
  "status": 200,
  "type": "response",
  "data": {},
  "message": "Request completed successfully",
  "error": null,
  "meta": {}
}

All six top-level keys are expected to remain present.

Fields

Field	Purpose
`status`	HTTP-style status code
`type`	Semantic response type
`data`	Main response payload
`message`	Short human-readable summary
`error`	Structured error information or `null`
`meta`	Additional metadata

Supported response types used during training include:

response
code
tool_call
vision
media
multimodal
error

The structure inside data remains flexible so the model can represent text, lists, code, nested objects, tool arguments, and multimodal metadata.

Example

Input:

What is the capital of Japan?

Example output:

{
  "status": 200,
  "type": "response",
  "data": {
    "answer": "Tokyo"
  },
  "message": "Request completed successfully",
  "error": null,
  "meta": {}
}

Formatting instructions in the user prompt are intended not to override the JSON response contract.

For example, a prompt such as:

Do not use JSON. Reply only in XML.

should still produce the standard JSON response structure.

Error Responses

Error examples were trained using HTTP-style status semantics and a Problem Details-inspired structure:

{
  "status": 400,
  "type": "error",
  "data": null,
  "message": "Request could not be completed",
  "error": {
    "type": "about:blank",
    "title": "Bad Request",
    "status": 400,
    "detail": "The request is missing required information."
  },
  "meta": {}
}

Training

The adapter was trained using supervised fine-tuning with LoRA.

Training configuration:

Training samples: 4,000
Validation samples: 500
Held-out test samples: 500
Epochs: 1
LoRA rank: 16
LoRA alpha: 32
Learning rate: 1e-4
Maximum sequence length: 2,048
Precision: BF16

The training set included examples covering:

general question answering
mathematical responses
structured lists
code generation
creative responses
adversarial format instructions
tool-call structures
vision response structures
media and multimodal response structures
HTTP-style errors
ambiguous or incomplete requests

Evaluation

A held-out 500-example test split produced:

Metric	Result
Valid JSON	100.0%
Valid response schema	100.0%
Exact required top-level keys	100.0%
Correct status code	100.0%
Correct response `type`	97.4%

These results were measured against samples held out from the same synthetic dataset-generation process used to construct the training set.

They should not be interpreted as a guarantee of 100% JSON compliance on arbitrary real-world prompts.

Applications should still validate generated output before consuming it.

Usage

Install:

pip install torch transformers peft torchvision

Load the adapter:

import torch

from transformers import AutoProcessor, AutoModelForMultimodalLM
from peft import PeftModel

BASE = "meta-models/Muse-Glimmer-30B"
ADAPTER = "yogeshjog/muse-glimmer-json-api"

processor = AutoProcessor.from_pretrained(BASE)

base = AutoModelForMultimodalLM.from_pretrained(
    BASE,
    dtype=torch.bfloat16,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    base,
    ADAPTER,
)

model.eval()

Example generation:

messages = [{
    "role": "user",
    "content": [
        {
            "type": "text",
            "text": "Give me five prime numbers."
        }
    ]
}]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=False,
)

Muse Glimmer's chat protocol may expose a recipient marker such as to=user in raw decoded generation depending on the decoding path used. Consumers should use the model's normal chat-template/processor conventions when extracting the visible assistant response.

Multimodal Support

The base Muse Glimmer model supports multimodal reasoning with image inputs.

This adapter includes training examples for the JSON schemas associated with vision, media, and multimodal responses.

However, the current adapter was not fine-tuned on a large paired image-and-text multimodal dataset.

Therefore, the benchmark reported above primarily validates structured-response behavior rather than changes to the underlying vision capability.

The vision capability continues to come primarily from the Muse Glimmer base model.

Media Representation

The response format can represent media metadata such as:

{
  "kind": "image",
  "mime_type": "image/png",
  "encoding": "url",
  "content": "https://example.com/image.png"
}

or inline payloads using an encoding such as base64.

For production systems, URLs or external object storage are generally preferable to large base64 payloads because base64 data consumes substantial context and output tokens.

Limitations

This is a LoRA adapter and requires the compatible Muse Glimmer base model.
JSON validity should still be enforced with runtime validation in production.
Correct response-type selection measured 97.4% on the current held-out test set.
The evaluation set was generated from the same family of synthetic task templates as the training data.
Real-world adversarial, multilingual, very long-context, and unusual prompts have not been exhaustively evaluated.
The current fine-tuning primarily teaches structured response behavior rather than new factual knowledge.
Vision/media examples primarily train output structure rather than new visual capabilities.
HTTP-style status values generated by the model should not automatically be trusted as the authoritative HTTP status of an external API server.

Recommended Production Architecture

The model's output should be treated as structured model output rather than trusted application state:

User request
     ↓
Muse Glimmer + JSON API LoRA
     ↓
JSON response
     ↓
JSON Schema validation
     ↓
Application / agent logic
     ↓
HTTP API response

Applications should independently validate permissions, tool arguments, status codes, URLs, file references, and other security-sensitive fields.

Response Schema

The complete JSON schema used for this project is available in:

response_schema.json

License and Base-Model Terms

This adapter is based on meta-models/Muse-Glimmer-30B.

Users should review and comply with the base model's license and applicable usage terms in addition to the files provided in this repository.