README.md

metadata

license: mit
language:
  - en
tags:
  - medical
extra_gated_heading: Acknowledge license to access the repository
extra_gated_button_content: Acknowledge license
extra_gated_prompt: ' This model and its associated weights are provided solely for research and educational purposes. They are not intended for clinical use, medical diagnosis, treatment decision-making, or patient management, and must not be used as a substitute for professional medical judgment. The outputs generated by this model do not constitute medical advice and should not be interpreted as diagnostic conclusions, clinical recommendations, or actionable guidance for patient care. Any clinical application would require independent validation, regulatory approval, and oversight by qualified medical professionals in accordance with local laws and institutional policies. The model should only be used in compliance with applicable laws, regulations, and ethical guidelines, including those governing human subjects research and genetic data. '
extra_gated_fields:
  I agree to use this model for non-commercial use ONLY: checkbox

Overview

This model ranks candidate disease-causing genes based on a patient’s phenotypic information. During inference, it does not generate a full free-text response. Instead, it performs gene-token subspace ranking on the logits at the final position and outputs a prioritized list of genes.

Visit https://github.com/discostudy/phenogemini for additional usage examples and demonstration data derived from real-world cohorts.

Step 1: Generate Prompt

Use PhenoGemini Atlas (https://phenogemini.org/) to create a prompt that includes "twin patients". After you input the patient's phenotypes, the Atlas retrieves the most similar patients from the literature and appends them to the final prompt.

Step 2: Inference

Because PhenoGemini-LLM performs ranking based on the token predicted at the final position,
please ensure that the prompt passed to the LLM does NOT contain any trailing spaces, newline characters, or other extraneous whitespace AT THE END.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "DISCOStudy/PhenoGemini"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
).eval()

WRAP_BREAK_1 = "\nWe can find some patients with similar phenotypes, such as:"
WRAP_BREAK_2 = "\nBased on all the information above, and my own general knowledge, this patient is likely to have a mutation in"
GENE_PREFIX = "<|PhenoGemini-Special-Token-Entrez-ID-"

def wrap_prompt(prompt: str, tokenizer) -> str:
    try:
        part1, remainder = prompt.split(WRAP_BREAK_1, 1)
        between, part3 = remainder.split(WRAP_BREAK_2, 1)
        part2 = WRAP_BREAK_1 + between
        part3 = WRAP_BREAK_2 + part3
        messages = [{"role": "user", "content": part1 + " We want to find the pathogenic gene mutation."}]
        text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
        text = text + part2[1:] + "\n</think>" + part3 + " "
        return text
    except ValueError:
        messages = [{"role": "user", "content": prompt}]
        return tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

def detect_gene_token_space(tokenizer):
  """
  Used to detect the boundary of special gene tokens in the tokenizer vocabulary for subsequent masking non-gene tokens.
  """
    vocab = tokenizer.get_vocab()
    gene_ids = sorted(idx for tok, idx in vocab.items() if tok.startswith(GENE_PREFIX))
    if not gene_ids:
        raise RuntimeError("No gene tokens found in tokenizer")
    base_cutoff = gene_ids[0]
    tensor_ids = torch.tensor(gene_ids, dtype=torch.long)
    return base_cutoff, len(gene_ids), tensor_ids

_,  _, gene_token_tensor = detect_gene_token_space(tokenizer)

with torch.no_grad():
    text=wrap_prompt(
      "[GET EXAMPLE PROMPT AT https://github.com/discostudy/phenogemini OR USE `https://phenogemini.org` TO GENERATE A PROMPT OF PATIENT'S PHENOTYPES.]", 
      tokenizer
    )
    inputs = tokenizer([text], return_tensors="pt")
    inputs = {k: v.to(next(model.parameters()).device) for k, v in inputs.items()}
    outputs = model(**inputs)

    # Take logits for the last token only
    logits = outputs.logits[0, -1, :].to("cpu", dtype=torch.float32)

    # Mask non-gene tokens
    gene_token_tensor = gene_token_tensor.to(dtype=torch.long, device=logits.device)
    gene_logits = logits[gene_token_tensor]

    # Sort by logits
    order = torch.argsort(gene_logits, descending=True)

    # Ranked Gene List with Scores (logits)
    sorted_gene_ids = gene_token_tensor[order]
    sorted_gene_tok = tokenizer.convert_ids_to_tokens(sorted_gene_ids.tolist())

Special Gene Token Example
In <|PhenoGemini-Special-Token-Entrez-ID-64087|>, the numeric value after the final hyphen represents the Entrez Gene ID.

Usage Example

Patient Phenotypes:

Gait disturbance, Scoliosis, Tall stature, Musculoskeletal abnormality, Positive wrist sign, Positive thumb sign, Pectus carinatum, Redundant skin, Scapular winging, Prominent vertebral column, Increased intervertebral disc space, Congenital hip dislocation

Prompt Generated From https://phenogemini.org:

For a patient who presents with Gait disturbance, Scoliosis, Tall stature, Musculoskeletal abnormality, Positive wrist sign, Positive thumb sign, Pectus carinatum, Redundant skin, Scapular winging, Prominent vertebral column, Increased intervertebral disc space, Congenital hip dislocation.
We can find some patients with similar phenotypes, such as: 
A patient presents with skeletal dysplasia, abnormal skeletal development starting with a ‘hump’ in the back that later evolved into scoliosis, short stature, abnormal chest appearance, deformed wrists and elbows, progressive difficulty in walking and holding objects with her hands, myopia, malar hypoplasia, hazy cornea with a hyper-pigmented spot in the sclera, short neck, pectus carinatum, flaring lower ribs, kyphoscoliosis with gibbus, markedly redundant umbilical skin, thickened wrists and elbows, fixed flexion deformity of knees, heel prominence, flat feet, overriding of lateral toes, joint hypermobility. This patient has a mutation in <|PhenoGemini-Special-Token-Entrez-ID-2588|>.
A patient presents with weak lower extremities, trigger fingers, tight hamstrings, proximal muscle weakness in his lower extremities that caused him difficulties with squatting, balancing, and walking with a stable gait, short stature, barrel-shaped chest, pectus carinatum, short neck and trunk, longer limbs, hyperextensible joints, thoracic kyphosis, mildly coarse facial features along with a flat midface, flat vertebrae, hypoplastic odontoid with a stable relationship to C1/C2, lumbar hyperlordosis, proximal tapering and shortened metacarpal bones, small and irregular carpal and tarsal bones, platyspondyly, hip dysplasia, flared lower coastal margin, elevated mucopolysaccharide levels in his urine, abnormal band pattern of urine oligosaccharides on electrophoresis, reduced β-galactosidase activity in white blood cells (12.3 nmol/h/mg protein), thoracic kyphosis measuring 4° and narrowing of the spinal canal diameter, diffuse lumbar spinal stenosis, narrow lumbar nerve root exit foramina, anterior beaking of the C3–T2 vertebral bodies, low bone mass for his age, ankle valgus (32° bilaterally at age 10), able to walk >1 km slowly with custom bilateral UCBL in-shoe orthotics, but preferred wheelchair, bilateral triggering fingers (right thumb, right index finger, right long digits, left ring digit), relative ulnar shortening compared to the radius, good wrist strength with normal pronation and supination, genu valgum, right-sided inguinal hernia (surgically repaired at age 7), soft systolic ejection murmur at the mid-left sternal border, redundant cords to the anterior mitral leaflet with leaflet thickening (age 6), mitral regurgitation (grade 2/6 pansystolic murmur at age 8), mild corneal clouding, normal polysomnogram, audiology, cognition assessment, nerve conduction study (age 5–7), stable kyphosis at the thoracolumbar junction, broad-based kyphosis centered on the thoracolumbar junction, anterior subluxation of T12 with respect to L1, full range of motion in the neck, no significant cervical instability clinically or radiographically, elevated ferritin levels, chronic pain in hips and legs with gait deterioration requiring wheelchair use for longer distances in adolescence, sleeping problems related to stiffness, pain, and anxiety, slightly elevated intraocular pressure on the right, collapsed left foot arch compared to the right, weak hands and intrinsics, finger escape during sustained finger adduction, Trendelenburg gait. This patient has a mutation in <|PhenoGemini-Special-Token-Entrez-ID-2720|>.
...
Based on all the information above, and my own general knowledge, this patient is likely to have a mutation in

Output:

Token	Gene Symbol	Probability
<|PhenoGemini-Special-Token-Entrez-ID-2200|>	FBN1	0.560
<|PhenoGemini-Special-Token-Entrez-ID-7043|>	TGFB3	0.120
<|PhenoGemini-Special-Token-Entrez-ID-2588|>	GALNS	0.040
<|PhenoGemini-Special-Token-Entrez-ID-4087|>	SMAD2	0.020
<|PhenoGemini-Special-Token-Entrez-ID-7048|>	TGFBR2	0.020
<|PhenoGemini-Special-Token-Entrez-ID-1280|>	COL2A1	0.017
<|PhenoGemini-Special-Token-Entrez-ID-2317|>	FLNB	0.016

Contact

For questions about individual modules or accessing additional data, please email dr.wunan@pumch.cn, jeffchenmed@gmail.com, or caijh09@gmail.com.