|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: |
|
|
- google-t5/t5-base |
|
|
language: |
|
|
- en |
|
|
library_name: transformers |
|
|
--- |
|
|
|
|
|
# LaaLM — Linux as a Language Model |
|
|
|
|
|
## Model Description |
|
|
|
|
|
LaaLM (Linux as a Language Model) is a fine-tuned T5-base model that simulates the textual output of Linux shell commands. |
|
|
|
|
|
Given: |
|
|
|
|
|
- a textual description of the current system state (working directory, files, environment), and |
|
|
|
|
|
- a bash command, |
|
|
|
|
|
the model predicts what the command would output if executed in a Linux environment. |
|
|
|
|
|
This is intended as a ***research and experimentation model***, not a real shell replacement. |
|
|
|
|
|
## How It Works |
|
|
|
|
|
Input format: |
|
|
``` |
|
|
System: <system state description> |
|
|
User: <bash command> |
|
|
``` |
|
|
|
|
|
Output: |
|
|
|
|
|
``` |
|
|
<predicted stdout / stderr> |
|
|
``` |
|
|
|
|
|
The system state is dynamically generated externally and passed to the model as text. |
|
|
The model does not maintain internal state, it only predicts output from the provided context. |
|
|
|
|
|
## Example |
|
|
|
|
|
### Input |
|
|
``` |
|
|
System: Current directory: /home/user |
|
|
Files in current directory: |
|
|
- test.txt (empty) |
|
|
Environment: USER=user, HOME=/home/user |
|
|
User: ls |
|
|
``` |
|
|
### Output |
|
|
``` |
|
|
test.txt |
|
|
|
|
|
``` |
|
|
## Intended Use |
|
|
|
|
|
- Research into learned environment simulation |
|
|
- Studying command semantics and error modeling |
|
|
- Dataset generation experiments |
|
|
- Educational exploration |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- This model does not execute real commands. |
|
|
- Even though it's also trained on cases where it's supposed to error out it may still hallucinate incorrect behavior for unseen commands or edge cases. |
|
|
- File system state must be maintained externally. |
|
|
- Output accuracy depends heavily on how closely the prompt matches the training format. |
|
|
- Not suitable for safety-critical or production systems. |
|
|
|
|
|
## Commands It Knows |
|
|
|
|
|
- pwd |
|
|
- echo |
|
|
- cat |
|
|
- ls |
|
|
- mkdir |
|
|
- touch |
|
|
|
|
|
Any other commands than these might either give wrong results or fail. |
|
|
|
|
|
## Training |
|
|
|
|
|
- Base model: T5-base |
|
|
- Fine-tuned on synthetic Linux command datasets |
|
|
- Mixed-precision training on NVIDIA V100 GPU |
|
|
- Approximately 80k training samples |
|
|
|
|
|
## Quick Usage |
|
|
|
|
|
```python |
|
|
from transformers import T5ForConditionalGeneration, T5Tokenizer |
|
|
import torch |
|
|
|
|
|
model_id = "LaaLM/LaaLM-v1" |
|
|
|
|
|
tokenizer = T5Tokenizer.from_pretrained(model_id) |
|
|
model = T5ForConditionalGeneration.from_pretrained(model_id).cuda() |
|
|
model.eval() |
|
|
|
|
|
def run_command(system_state, command): |
|
|
prompt = f"System: {system_state}\nUser: {command}" |
|
|
inputs = tokenizer(prompt, return_tensors="pt").to("cuda") |
|
|
|
|
|
with torch.no_grad(): |
|
|
output = model.generate( |
|
|
**inputs, |
|
|
max_new_tokens=128, |
|
|
do_sample=False |
|
|
) |
|
|
|
|
|
return tokenizer.decode(output[0], skip_special_tokens=True) |
|
|
|
|
|
print(run_command("Current directory: /home/user", "pwd")) |
|
|
``` |
|
|
|
|
|
## Shoutouts |
|
|
|
|
|
Very big thanks to [@mradermacher](https://huggingface.co/mradermacher) for spending time and compute to generate GGUF Quantized versions of LaaLM-v1! |
|
|
|
|
|
Check out the quantized versions he made [here](https://huggingface.co/mradermacher/LaaLM-v1-GGUF). With these you can run LaaLM-v1 on low tier GPUs or CPUs! |
