Add model card

README.md

@@ -0,0 +1,73 @@
+---
+language:
+- ja
+library_name: pytorch
+pipeline_tag: text-generation
+datasets:
+- KeisukeMiyamoto/lambda-corpus
+- KeisukeMiyamoto/SyntheticTextbook-jp
+- KeisukeMiyamoto/SyntheticTalk-jp
+base_model: KeisukeMiyamoto/lambda-1-360m-mid
+base_model_relation: finetune
+tags:
+- lambda
+- pytorch
+- causal-lm
+- text-generation
+- conversational
+- instruction-tuned
+- supervised-fine-tuning
+- decoder-only
+- custom-code
+- grouped-query-attention
+- rotary-position-embedding
+- byte-level-bpe
+---
+
+# lambda-1-360m-it
+
+lambda-1-360m-it is an experimental Japanese instruction-tuned language model based on [`lambda-1-360m-mid`](https://huggingface.co/KeisukeMiyamoto/lambda-1-360m-mid) and supervised fine-tuned on a synthetic Japanese conversation corpus.
+
+All training code is publicly available at [KeisukeMiyamoto1324/lambda](https://github.com/KeisukeMiyamoto1324/lambda).
+
+## Model Details
+
+| Item | Value |
+|---|---:|
+| Parameters | 359.9M |
+| Architecture | Decoder-only Transformer |
+| Context length | 1,024 tokens |
+| Tokenizer | Byte-level BPE |
+| Vocabulary size | 65,536 |
+| Layers | 32 |
+| Hidden size | 960 |
+| Attention heads | 15 |
+| Key-value heads | 5 |
+| FFN size | 4,096 |
+
+## Training Data
+
+The base model was pretrained on [`KeisukeMiyamoto/lambda-corpus`](https://huggingface.co/datasets/KeisukeMiyamoto/lambda-corpus) and further pretrained on [`KeisukeMiyamoto/SyntheticTextbook-jp`](https://huggingface.co/datasets/KeisukeMiyamoto/SyntheticTextbook-jp).
+
+This model was then supervised fine-tuned on [`KeisukeMiyamoto/SyntheticTalk-jp`](https://huggingface.co/datasets/KeisukeMiyamoto/SyntheticTalk-jp) for 5,000 steps.
+
+## Usage
+
+```bash
+git clone https://github.com/KeisukeMiyamoto1324/lambda.git
+cd lambda
+python3 -m venv venv
+source venv/bin/activate
+pip3 install -r requirements.txt
+
+python3 src/inference_it/inference.py \
+  --model-dir "KeisukeMiyamoto/lambda-1-360m-it" \
+  --prompt "人工知能とは何ですか" \
+  --max-new-tokens 128
+```
+
+## Limitations
+
+This model is not safety-aligned. It may generate incorrect, biased, unsafe, or low-quality text.
+
+The model was trained primarily on synthetic Japanese text and has not been evaluated on standard benchmarks.