README.md

metadata

base_model: PLM-Team/PLM-1.8B-Instruct
language:
  - en
library_name: transformers
license: apache-2.0
quantized_by: PLM-Team
pipeline_tag: text-generation

🖲️ PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing

👉 Project PLM Website

---

The PLM (Peripheral Language Model) series introduces a novel model architecture to peripheral computing by delivering powerful language capabilities within the constraints of resource-limited devices. Through modeling and system co-design strategy, PLM optimizes model performance and fits edge system requirements. PLM employs Multi-head Latent Attention and squared ReLU activation to achieve sparsity, significantly reducing memory footprint and computational demands. Coupled with a meticulously crafted training regimen using curated datasets and a Warmup-Stable-Decay-Constant learning rate scheduler, PLM demonstrates superior performance compared to existing small language models, all while maintaining the lowest activated parameters, making it ideally suited for deployment on diverse peripheral platforms like mobile phones and Raspberry Pis.

Here we present the static quants of https://huggingface.co/PLM-Team/PLM-1.8B-Instruct

Provided Quants

Link	Type	Size/GB	Notes
https://huggingface.co/PLM-Team/PLM-1.8B-Instruct-gguf/blob/main/PLM-1.8B-Instruct-F16.gguf	F16	3.66GB	Recommanded
...	...	...	...

Usage (llama.cpp)

(Content omitted for brevity - same as original)

Usage (transformers)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("PLM-Team/PLM-1.8B-Instruct")
model = AutoModelForCausalLM.from_pretrained("PLM-Team/PLM-1.8B-Instruct", torch_dtype=torch.bfloat16)

# Input text
input_text = "Tell me something about reinforcement learning."
inputs = tokenizer(input_text, return_tensors="pt")

# Completion
output = model.generate(inputs["input_ids"], max_new_tokens=100)
print(tokenizer.decode(output[0], skip_special_tokens=True))