metadata
license: apache-2.0
tags:
- huggingface
- transformers
- language-model
- MobileLLM
- Japanese
- lightweight
- fast-inference
- high-school-project
pipeline_tag: text-generation
license: apache-2.0
summer-S1
This repository contains a lightweight and fast model developed by Japanese high school students, based on MobileLLM-R1-950M-MLX.
Overview
This model improves specific computational layers to achieve:
- Fast inference in low-resource environments
- Memory-efficient sequential data processing
- Efficient knowledge transfer from pretrained models
It is the result of a high school research project aiming to create a compact, pretrained language model.
Model Details
- Base model:
robbiemu/MobileLLM-R1-950M-MLX - Improved layers: Selected layers optimized for speed and memory efficiency
- Model size: Hidden layer size × 2 (efficiency-focused)
- Lightweight parameters: Knowledge transfer from pretrained models
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load the high school project model
model_name = "summerstars/summer-S1"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True).to(device)
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
# Text generation
prompt = "This model was developed by high school students"
input_ids = tokenizer(prompt, return_tensors="pt").to(device)
output_ids = model.generate(input_ids.input_ids, max_length=100)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))