README.md

---
language:
- en
- fr
pipeline_tag: text-generation
tags:
- open-science
- information-extraction
- datasets
- software
- scientific-text
library_name: transformers
---

# Baguette-Software-Dataset (600M)

A small, deployable model that reads scientific papers and extracts the **datasets** and **software**
they use or produce, then reconciles those mentions into an article-level open-science record
(PLOS-OSI: data/code production and sharing). 608M parameters — runs on a single GPU.

It works in **two steps**:

1. **Extraction (paragraph → mentions).** For each paragraph, emit the datasets and software mentioned,
   with role (produced / reused / mentioned), type, and a verbatim evidence span. **This is the validated
   step** (see Evaluation).
2. **Analysis (mentions → article record).** Aggregate the per-paragraph mentions, deduplicate, and
   produce one article-level record with a PLOS-OSI summary. **This step is functional but not yet as
   thoroughly validated** as step 1 — treat its article-level judgements as indicative.

> **Scope note.** The production-validated deliverable is **step 1 (extraction)**. Step 2 is included for
> convenience and works, but its individual-artifact reconciliation has a known recall gap and has not been
> validated to the same standard.

## Model

| | |
|---|---|
| Architecture | `LlamaForCausalLM`, 608M (hidden 1024, 48 layers, GQA 16/4 heads) |
| Context | 8192 tokens |
| Tokenizer | 65,536-vocab SYNTH BPE (ChatML control tokens native) |
| Base | Pleias `balanced-600M` (SYNTH-pretrained) |
| Training | distilled from a Qwen3.5-27B teacher on ~531k two-step examples |
| Sibling | [JZSG/baguette-funders-600m](https://huggingface.co/JZSG/baguette-funders-600m) (same arch, acknowledgements task) |

## Prompt format (ChatML)

The model was trained on exactly these two templates — match them.

**Step 1 — extraction:**
```
<|im_start|>user
<text>{paragraph}</text><|im_end|>
<|im_start|>assistant
```
→ `{"is_boilerplate": false, "datasets": [...], "software": [...]}`

**Step 2 — analysis:**
```
<|im_start|>user
<mentions>{aggregated_mentions_json}</mentions><|im_end|>
<|im_start|>assistant
```
→ `{"datasets": [...], "software": [...], "paper_summary": {...}}`

Greedy decoding, stop at `<|im_end|>`. A runnable end-to-end example is in
[`inference_example.py`](./inference_example.py).

## Evaluation

Held-out set: 34 articles / 685 paragraphs, held out of all training, medium/large-skewed (the hard end).
Metric is **agreement with the Qwen3.5-27B teacher** on held-out inputs — not correctness vs a human gold
set (no human gold set exists yet).

- **Step 1 (extraction), 685 paragraphs: 89% agreement**, no over-extraction — production-viable.
- Step 2 (analysis), 34 articles: 85% valid parse, 0 dataset↔software mis-typing; article-level PLOS-OSI
  summary vs teacher ≈ 100% (data axes) / ~87% (code axes). Individual-artifact reconciliation is weaker
  (a recall gap), which is why step 2 is labelled functional-not-validated above.

## Limitations

- **Agreement, not gold.** Numbers are agreement with a teacher model, appropriate for a distillation
  target but not a claim of correctness against human annotation.
- **Step 2 is indicative.** Use step-1 extraction for anything that must be reliable.
- English/French scientific text; other languages untested.
- Emits JSON but is not constrained-decoded — parse defensively.

## License

To be set by the owner before public release. The model is distilled from Qwen3.5-27B and gemma-4 teachers
on a Pleias base; confirm the applicable terms before redistribution.