LAYA SERVER

LAYA SERVER is a self-hosted API and web interface for Laya's structured decision models, compatible with the TypeSafe Jev API format. It packages the System One inference capabilities of upstream Laya as a standalone service, making them easier to deploy and use in your applications.

Quick Start

Once the image is published on Docker Hub, install Docker and run the following command. Replace latest with a version tag to pin a release.

docker run -d --name laya-server --init --restart unless-stopped \
  -p 8080:8080 \
  -v laya-data:/data \
  -e LAYA_ADMIN_USERNAME=admin \
  -e LAYA_ADMIN_PASSWORD='change-this-admin-password' \
  -e LAYA_HF_ENDPOINT=https://huggingface.co \
  -e LAYA_MODEL_VERSION=v2.2 \
  -e LAYA_MODEL_PRECISION=fp8 \
  x6nux/laya-server:latest

The image does not bundle model weights. On first start it downloads the selected precision from x6nux/laya-ad on Hugging Face into the laya-data volume. Set LAYA_HF_ENDPOINT=https://hf-mirror.com to use a mirror, LAYA_MODEL_PRECISION to choose fp32/fp16/bf16/fp8/int8/int4, and LAYA_PRELOAD=fp8,int4 to preload several precisions. LAYA_AUTO_DOWNLOAD=0 disables the first-start download; admins can then install models from /internal/models/catalog.

Replace the sample password with your own, then sign in at http://YOUR_SERVER_IP:8080. Create a key on the API Keys page; the full key is shown only once. /health/ready checks the model files. Data is stored in the laya-data volume.

YOUR_SERVER_IP is your server's IP address. To use a domain name, configure HTTPS on a reverse proxy and forward requests to port 8080 on the server.

Screenshot

Call the API

curl -X POST http://YOUR_SERVER_IP:8080/v1/systemone \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"state":{"message":{"text":"加微信免费领取涨粉秘籍，名额有限！"}},"questions":{"is_ad":{"type":"choice","instructions":"判断是否广告","criteria":{"ad":"广告","clean":"正常"}}}}'

Successful responses include answers, model, and actual token counts in usage. Choice questions are routed by criteria: {ad, clean} returns the binary decision; the seven kind labels (crypto, gambling, porn, porn_bait, promo, scam, spam_flood) return kind probabilities; {account, message} routes to the scope head and 4-level severity score questions route to the severity head (v2.x four-head models). Older binary-only models return unsupported=true for these questions.

Build and push locally

Images are built and pushed from your machine, not by CI:

docker login
cd laya-server
./build_push.sh v2.2

This builds x6nux/laya-server:v2 (also tags latest) and pushes it to Docker Hub. Override IMAGE, TAG, PLATFORM or PUSH_LATEST=0 as needed.

Usage statistics and storage

Inference responses enqueue usage statistics without waiting for SQLite writes. A single background thread per process reuses a connection, writes batches of up to 100 events, and coalesces last_used_at updates by API key. Partial batches flush after up to one second when the writer is idle; slow storage can delay statistics further. The usage field in each inference response is unchanged. duration_ms measures model prediction and result validation, not queue time or database writes.

Statistics are best-effort, eventually consistent telemetry, not a billing or quota ledger. Each process buffers up to 1,000 queued events plus one in-flight batch. A full queue drops new statistics with rate-limited warnings. SQLite failures roll back where possible, log and drop the affected batch, then reconnect for later batches; failed batches are not retried because an uncertain commit could double-count usage. Neither condition turns successful inference into an HTTP 503. API-key authentication and revocation still use the database synchronously.

Graceful shutdown attempts to drain the queue for up to five seconds. An in-flight SQLite call cannot be cancelled safely and may outlive that wait; forced termination, a shutdown timeout, or storage errors can lose unflushed statistics. Keep /data on low-latency persistent storage: background batching reduces write pressure but does not fix slow authentication reads or database locking.

License and Feedback

This project is licensed under Apache License 2.0. For issues and suggestions, visit GitHub Issues.