---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
library_name: transformers
base_model:
- FrontiersMind/Lumma-0.6B-Base
---

# Lumma-0.6B-Extract

Based on [Lumma-0.6B-Base](FrontiersMind/Lumma-0.6B-Base), Lumma-0.6B-Extract is a lightweight, single-turn specialized model designed to accurately extract structured information from text in one step, enabling efficient and reliable information extraction.


### Benchmark results

<img src="benchmark_results.jpg" width="1000"/>

We benchmarked Lumma-0.6B-Extract on FrontiersMind's internal structured benchmark, measuring model's performances on ~500 documents of diverse types including invoices,travel itenaries, OCR Extracted text & Emails etc. 
We plan to open-source this benchmark in the coming weeks, along with a extensive leaderboard including most popular open-weight and closed-sourced APIs and a Python library allowing to easily measure model performances on structured extraction.


## Use cases

- **Information extraction**: Extract structured information such as names, dates, organizations, locations, products, attributes, and other fields from unstructured text.
- **Document processing**: Convert unstructured text from documents, emails, reports, and messages into structured JSON.
- **Data processing pipelines**: Transform unstructured text into structured data for downstream applications, databases, search systems, and analytics.
- **Real-time AI applications**: Power applications that require fast and reliable information extraction with low latency.
- **Resource-constrained environments**: Support efficient extraction on mobile devices, edge devices, embedded systems, and hardware with limited compute and memory resources.


> [!NOTE]
> 🚧 **Coming Soon:** We will soon be bringing multilingual Indic language support to **Lumma-0.6B-Extract**.

## Structured Extraction

Lumma-0.6B-Extract uses a JSON template to define the information that should be extracted from the input text.



The extraction process consists of two main steps:

1. **Extraction template**: Provide a JSON template describing the fields and information to extract.
2. **Information extraction**: Provide the input text, and the model returns the requested information in the specified JSON structure.

This approach allows users to define custom extraction schemas depending on their application.

## Example

**Template:**

    {
        "name": "string",
        "company": "string",
        "job_title": "string"
    }

**Input text:**

    John Smith joined Acme Corporation as a Senior Software Engineer.

**Expected output:**

    {
        "name": "John Smith",
        "company": "Acme Corporation",
        "job_title": "Senior Software Engineer"
    }

## Usage

    import torch
    import json
    from transformers import AutoModelForCausalLM, AutoTokenizer

    MODEL_PATH = "FrontiersMind/Lumma-0.6B-Extract"

    tokenizer = AutoTokenizer.from_pretrained(
        MODEL_PATH,
        trust_remote_code=True
    )

    model = AutoModelForCausalLM.from_pretrained(
        MODEL_PATH,
        trust_remote_code=True,
        torch_dtype=torch.bfloat16,
        device_map="auto"
    )

    model.eval()


    input_text = "John Smith joined Acme Corporation as a Senior Software Engineer."

    template = {
        "name": "string",
        "company": "string",
        "job_title": "string",
    }

    prompt = tokenizer.apply_chat_template(
    input_text=input_text,
    template=template,
    tokenize=False,
    add_generation_prompt=True
    )


    inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)

    with torch.inference_mode():
        out = model.generate(
            **inputs,
            max_new_tokens=512,
            do_sample=False,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.pad_token_id,
        )

    raw = tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False)
    pred = raw.split("<|endoftext|>")[0].strip()
    print(json.dumps(json.loads(pred), indent=4))


    


## License

This model is released under the Apache License 2.0.

## Feedback & Suggestions

We’d love to hear your thoughts, feedback, and ideas!

- **Discord**: https://discord.gg/ZGdjCdRt
- **Email**: support@frontiersmind.ai
- **Official Website**: https://www.frontiersmind.ai/
- **LinkedIn**: https://www.linkedin.com/company/frontiersmind/
- **X (Twitter)**: https://x.com/FrontiersMind