You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

JPharmatron-7B

JPharmatron-7B is a 7B large language model designed for pharmaceutical applications and researches.

Model Details

Model Description

The JPharmatron-7B is continually pre-trained using 8.8B tokens from Japanese and English datasets, based on Qwen2.5-7B. Compared to the JPharmatron-7B-base model, JPharmatron-7B has enhanced chat capabilities, obtained from Qwen2.5-7B-Instruct via model merging.

Developed by: EQUES Inc.
Funded by [optional]: GENIAC Project
Model type: Causal decoder-only
Language(s) (NLP): Japanese, English
License: CC-BY-SA-4.0

Model Sources [optional]

Repository: https://github.com/EQUES-Inc/pharma-LLM-eval
Paper [optional]: A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP (IJCNLP-AACL 2025)

Uses

This model is intended for applications in pharmaceutical paperwork and research. It is not validated for medical use or any other risk-sensitive use.

Evaluation

We evaluated our model, JPharmatron-7B, with other general / domain-specific models of a similar size.

Testing Data

JPharmaBench and two existing benchmarks (JMMLU (pharma) and IgakuQA) were used.

Results

Compared to Meditron3-Qwen2.5-7B and Llama3.1-Swallow-8B-Instruct-v0.3, JPharmatron-7B achieved the highest score on all of the five benchmarks.

Citation [optional]

This paper has been accepted to IJCNLP-AACL 2025.

BibTeX:

@inproceedings{ono-etal-2025-japanese,
    title = "A {J}apanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical {NLP}",
    author = "Ono, Shinnosuke  and
      Sukeda, Issey  and
      Fujii, Takuro  and
      Buma, Kosei  and
      Sasaki, Shunsuke",
    editor = "Inui, Kentaro  and
      Sakti, Sakriani  and
      Wang, Haofen  and
      Wong, Derek F.  and
      Bhattacharyya, Pushpak  and
      Banerjee, Biplab  and
      Ekbal, Asif  and
      Chakraborty, Tanmoy  and
      Singh, Dhirendra Pratap",
    booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.ijcnlp-long.72/",
    pages = "1316--1332",
    ISBN = "979-8-89176-298-5",
    abstract = "We present **JPharmatron**, a Japanese domain-specific large language model (LLM) for the pharmaceutical field, developed through continual pre-training on two billion Japanese pharmaceutical tokens and eight billion English biomedical tokens. For rigorous evaluation, we introduce **JPharmaBench**, a benchmark suite consisting of three new benchmarks: YakugakuQA, based on national pharmacist licensing exams; NayoseQA, which tests cross-lingual synonym and terminology normalization; and SogoCheck, a novel task involving cross-document consistency checking.We evaluate our model against open-source medical LLMs and commercial models, including GPT-4o. Experimental results show that **JPharmatron** outperforms existing open models and achieves competitive performance with commercial ones.Interestingly, even GPT-4o performs poorly on SogoCheck, suggesting that cross-sentence consistency reasoning remains an open challenge.**JPharmatron** enables secure and local model deployment for pharmaceutical tasks, where privacy and legal constraints limit the use of closed models. Besides, **JPharmaBench** offers a reproducible framework for evaluating Japanese pharmaceutical natural language processing. Together, they demonstrate the feasibility of practical and cost-efficient language models for Japanese healthcare and pharmaceutical sectors.Our model, codes, and datasets are available on HuggingFace: https://huggingface.co/collections/EQUES/jpharmatron and https://huggingface.co/collections/EQUES/jpharmabench."
}