BioJev-Nano

BioJev-Nano is the 0.8B member of the BioJev biomedical decision-model family.

It is built from Qwen/Qwen3.5-0.8B-Base and follows the full BioJev adaptation pipeline:

Qwen3.5-0.8B-Base
        │
        ▼
Biomedical DAPT
PubMed + PMC
        │
        ▼
General NLI
SNLI + MNLI + ANLI
        │
        ▼
Biomedical NLI
BioNLI + NLI4CT
        │
        ▼
BioJev-Nano

The released checkpoint is a PEFT/QLoRA sequence-classification adapter with three NLI labels:

0 → contradiction
1 → entailment
2 → neutral

BioJev-Nano is primarily used for lightweight inference, controlled ablation experiments, and analysis of how biomedical domain adaptation and NLI specialization affect transfer.

Authors

Creator: Gabriel Henrique Alencar Medeiros
Supervisor: Lina F. Soualmia

BioJev is developed at LITIS / Université de Rouen Normandie.

Training recipe

Biomedical DAPT

The Nano model uses a lightweight biomedical DAPT stage initialized from Qwen3.5-0.8B-Base.

Biomedical token budget: 50,000
Sequence length:          512
Method:                   QLoRA

The biomedical corpus combines PubMed and PMC sources.

General NLI

SNLI      35,000
MNLI      45,000
ANLI R1   20,000
----------------
Total    100,000

Biomedical NLI

BioNLI    30,000
NLI4CT    10,000
----------------
Total     40,000

The released checkpoint corresponds to the full Nano configuration:

DAPT → General NLI → Biomedical NLI

Evaluation

BioJev-Nano was evaluated on five biomedical benchmarks.

Dataset	Macro-F1
BioNLI	89.41
NLI4CT	46.07
ChemProt	14.33
DDI2013	32.55
BioRED	16.16
Relation-transfer mean (ChemProt/DDI2013/BioRED)	21.01

These scores correspond to the full Nano configuration used in the BioJev Sprint 6 ablation study.

Interpretation

The Nano ablations indicate that:

general NLI training provides a strong contribution to transfer toward unseen biomedical tasks;
biomedical NLI training strongly specializes the model toward biomedical inference;
this specialization can reduce transfer to some unseen relation-classification tasks;
biomedical DAPT has smaller and more task-dependent effects at the Nano scale.

BioNLI and NLI4CT can be seen during biomedical decision training depending on the experiment, so they should not be interpreted as clean zero-shot transfer tasks for the full model.

Loading BioJev-Nano

Because BioJev is a three-class PEFT sequence classifier, the base model must be reconstructed with num_labels=3 before loading the adapter.

Do not rely on a default two-label sequence-classification configuration.

Install:

pip install -U torch transformers peft accelerate

For optional 4-bit loading:

pip install -U bitsandbytes

Python example

import torch

from peft import PeftConfig, PeftModelForSequenceClassification
from transformers import AutoModelForSequenceClassification, AutoTokenizer

MODEL = "Gabriel382/BioJev-Nano"

LABEL2ID = {
    "contradiction": 0,
    "entailment": 1,
    "neutral": 2,
}

ID2LABEL = {v: k for k, v in LABEL2ID.items()}

peft_config = PeftConfig.from_pretrained(MODEL)

base = AutoModelForSequenceClassification.from_pretrained(
    peft_config.base_model_name_or_path,
    num_labels=3,
    label2id=LABEL2ID,
    id2label=ID2LABEL,
    dtype=torch.bfloat16 if torch.cuda.is_available() else torch.float32,
    device_map="auto" if torch.cuda.is_available() else None,
)

model = PeftModelForSequenceClassification.from_pretrained(
    base,
    MODEL,
    is_trainable=False,
)

tokenizer = AutoTokenizer.from_pretrained(MODEL)

if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token

model.config.pad_token_id = tokenizer.pad_token_id
model.eval()

premise = "The clinical report describes a bacterial pneumonia."
hypothesis = "The patient has an infectious pulmonary disease."

inputs = tokenizer(
    premise,
    hypothesis,
    return_tensors="pt",
    truncation=True,
    max_length=512,
)

device = next(model.parameters()).device
inputs = {k: v.to(device) for k, v in inputs.items()}

with torch.inference_mode():
    logits = model(**inputs).logits[0].float()
    probabilities = torch.softmax(logits, dim=-1)

for label, idx in LABEL2ID.items():
    print(f"{label:14s}: {probabilities[idx].item():.4f}")

A standalone version of this example is included in example_inference.py.

Jev / System One compatibility API

The BioJev GitHub repository includes a compatibility layer that exposes existing BioJev sequence-classification checkpoints through a Jev/System-One-shaped API:

POST /v1/systemone

Supported question types are:

choice
noul
score

This is an engineering compatibility bridge over the BioJev NLI classifier. BioJev-Nano is not a native Ollama/System-One checkpoint.

Install the BioJev API

Clone the main project:

git clone https://github.com/Gabriel382/BioJev.git
cd BioJev

python -m venv .venv
source .venv/bin/activate

pip install -e .
pip install fastapi uvicorn

Download or clone this model repository and point the server to its local directory.

Example:

python scripts/serve_systemone.py \
  --checkpoint /path/to/BioJev-Nano \
  --model-name biojev-nano \
  --load-in-4bit \
  --host 0.0.0.0 \
  --port 8000

Check the service:

curl http://127.0.0.1:8000/health

System One choice example

curl http://127.0.0.1:8000/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "model": "biojev-nano",
    "state": "The patient has fever, productive cough, and a new lobar infiltrate.",
    "questions": {
      "diagnosis": {
        "type": "choice",
        "instructions": "Which diagnosis is best supported?",
        "criteria": {
          "pneumonia": "Community-acquired pneumonia",
          "asthma": "Acute asthma exacerbation",
          "migraine": "Migraine"
        }
      }
    }
  }'

For each candidate, the bridge constructs a premise/hypothesis NLI pair and computes:

support(candidate) =
entailment_logit - contradiction_logit

Candidate supports are normalized with a softmax.

For robust Qwen3.5/PEFT compatibility, candidates are evaluated sequentially with batch_size=1.

Repository files

A typical BioJev-Nano model repository contains:

BioJev-Nano/
├── adapter_config.json
├── adapter_model.safetensors
├── tokenizer.json
├── tokenizer_config.json
├── chat_template.jinja          # if present in the training checkpoint
├── training_manifest.json       # if present
├── README.md
├── example_inference.py
└── requirements.txt

The PEFT adapter does not duplicate the Qwen3.5-0.8B base weights. The base model is downloaded separately when BioJev-Nano is loaded.

Scientific status

BioJev is a research project.

The model, its confidence estimates, and the System One compatibility layer are not validated clinical decision systems. They must not be used as the sole basis for diagnosis, treatment, or other high-stakes medical decisions.

Citation

A formal paper citation will be added when the BioJev publication is available.

Until then, please cite the BioJev repository and the specific Hugging Face checkpoint used.

Acknowledgements

BioJev-Nano was created by Gabriel Henrique Alencar Medeiros under the supervision of Lina F. Soualmia at LITIS / Université de Rouen Normandie.

BioJev builds on the Qwen3.5 model family and publicly available biomedical and natural-language-inference datasets.

Downloads last month: 6

Model tree for Gabriel382/BioJev-Nano

Base model

Qwen/Qwen3.5-0.8B-Base

Adapter

(40)

this model

Gabriel382
/

BioJev-Nano