QuantFactory/Typst-Coder-9B-GGUF

This is quantized version of TechxGenus/Typst-Coder-9B created using llama.cpp

Original Model Card

Typst-Coder

[🤖Models] | [🛠️Code] | [📊Data] |

---

• Typst-Coder
  • Introduction
  • Usage

---

Introduction

While working with Typst documents, we noticed that AI programming assistants often generate poor results. I understand that these assistants may perform better in languages like Python and JavaScript, which benefit from more extensive datasets and feedback signals from executable code, unlike HTML or Markdown. However, current LLMs even frequently struggle to produce accurate Typst syntax, including models like GPT-4o and Claude-3.5-Sonnet.

Upon further investigation, we found that because Typst is a relatively new language, training data for it is scarce. GitHub's search tool doesn't categorize it as a language for code yet, and The Stack v1/v2 don’t include Typst. No open code LLMs currently list it as a supported language, either. To address this, we developed this project aimed at collecting relevant data and training models to improve Typst support in AI programming tools.

Usage

An example script is shown below:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TechxGenus/Typst-Coder-9B")
model = AutoModelForCausalLM.from_pretrained(
    "TechxGenus/Typst-Coder-9B",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "user", "content": "Hi!"},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=512)
print(tokenizer.decode(outputs[0]))