---
title: README
emoji: "⚡"
colorFrom: gray
colorTo: blue
sdk: static
pinned: false
---

# SG Systems

**Consulting & forward-deployed engineering for Southeast Asian applied AI.**

We embed senior engineers with client teams and ship working systems in weeks
rather than producing strategy decks. Fixed scope, outcome-based pricing, no
hourly billing.

Serving banking, insurance, fintech, healthcare, retail, government and
mid-market commerce across SEA — where a deployment has to be PDPA-compliant,
locally tuned, and native to the channels people actually use.

[sg-systems.co](https://www.sg-systems.co/)

## Why we publish inference work

Regional deployment runs into two constraints that a general-purpose API does
not solve.

**Data residency.** PDPA and its neighbours make "send the customer's data to a
US endpoint" a compliance conversation before it is an engineering one.
Inference that never leaves the device sidesteps that conversation entirely.

**Hardware reality.** SEA deployments run on the devices people have, not on
datacentre accelerators. A model that needs 15 GB is not a product; one that
fits in 3.7 GB is.

So we work on making capable models run locally. What we publish here are the
artifacts and measurements from that work.

## Projects

**FORGE** — mixed-precision post-training quantization. The result depends less
on the average bit width than on which tensor families are excluded from it:
crushing a Mamba-2 state projection to ternary yields a model that is fluent and
confidently wrong.

**HELIX** — a Metal prefill kernel for the SSM scan in Mamba-2 hybrids,
integrated into `llama.cpp` through a 42-line patch. Chunk-parallel
decomposition; ~3x on the isolated op for shapes upstream's fast path declines.

| repository | |
|---|---|
| [`forge`](https://github.com/sg-systems-co/forge) | quantization |
| [`helix`](https://github.com/sg-systems-co/helix) | Metal kernel + llama.cpp patch |
| [`helix-chat-ui`](https://github.com/sg-systems-co/helix-chat-ui) | SwiftUI client |
| [`forge-helix`](https://github.com/sg-systems-co/forge-helix) | overview and release tooling |

## Models

| | |
|---|---|
| [`Falcon-H1-7B-FORGE-v2`](https://huggingface.co/sgsystems/Falcon-H1-7B-FORGE-v2) | `Falcon-H1-7B-Instruct` at 2.06 bpw / 3.48 GB. Runs on Apple Silicon in 3.73 GB resident at 75 tok/s decode. **English only** — this is an engineering artifact from the compression work, not a SEA-language model. |

**Read the model card before using it.** It needs `repeat_last_n = 2048`;
llama.cpp's default of 64 is too small to suppress the cross-turn repetition
these models fall into, and the usual sampling triple without it reproduces the
bug.

## How we report numbers

Measurements come with their conditions attached, and with the cases that did
not work. Things we published that contradicted our own assumptions:

- On M5, `simdgroup_matrix` at bf16 is **slower** than at fp32 (10.88 vs 16.12
  TFLOP/s). It is not the matrix hardware. Reduced precision only pays through
  `MetalPerformancePrimitives`, and only with both operands bf16.
- Adding a repetition penalty did **not** fix our repetition loops. The penalty
  *window* did, and only above 2048 tokens. Within-turn repetition was 0%
  throughout; the failure was entirely cross-turn.
- The quantized model still loses meta-instruction following, and no perplexity
  comparison against fp16 has been run.

Where a claim is not measured, we say so rather than rounding it up.

## Contact

sebastian@sg-systems.co