File size: 4,247 Bytes
c7b8451 66fffbe 13960d6 0571a73 c7b8451 e858b1c c7b8451 66fffbe c7b8451 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | ---
license: apache-2.0
language:
- en
base_model:
- StentorLabs/Stentor-30M
pipeline_tag: text-generation
tags:
- python
- code-generation
- tiny-model
- code
library_name: transformers
---
**Model Description**
Stentor Python 30M is a compact language model specifically fine-tuned for Python code generation and autocompletion tasks. Based on the Stentor-30M architecture, this model contains 30 million parameters and is designed to run efficiently on resource-constrained devices including mobile phones and embedded systems.
**Model Details**
- **Developed by:** Experimental fine-tuning project
- **Model type:** Causal language model (LlamaForCausalLM)
- **Language:** Python code, English instructions
- **Parameters:** 30,419,712
- **Context length:** 512 tokens
- **Model size:** 60 MB (FP16), 30 MB (INT8)
- **License:** Apache 2.0
**Training Data**
The model was fine-tuned on a curated dataset of 872 Python examples, including:
- Basic algorithms (factorial, prime numbers, list operations)
- Class implementations (Stack, BankAccount, Rectangle, Circle)
- Recursive functions (quicksort, Fibonacci)
- String manipulation (palindrome, anagram, vowel counting)
- MBPP (Mostly Basic Python Problems) dataset tasks
All examples follow a consistent format with "### Task:" instruction and "### Solution:" code block.
**Training Process**
The fine-tuning process involved multiple stages:
1. Base model: Stentor-30M
2. Initial fine-tuning on 50k examples
3. Multiple correction rounds with progressively lower learning rates
4. Final detoxification training with learning rate 3e-7 to remove undesirable patterns
**Evaluation Results**
The model was evaluated on several test categories:
| Category | Pass Rate | Notes |
|----------|-----------|-------|
| Basic functions | 80% | Factorial, prime check, etc. |
| Classes from training set | 100% | Stack, BankAccount, Rectangle |
| New complex classes | 33% | Graph, Queue, inheritance |
| Function signatures (MBPP) | 100% | Correctly generates def statements |
**Capabilities**
- Generates Python functions from natural language descriptions
- Implements basic algorithms (factorial, prime check, palindrome)
- Creates class definitions with methods (Stack, BankAccount, Rectangle)
- Handles recursive functions (quicksort, Fibonacci)
- Produces syntactically correct function signatures
**Limitations**
- May produce repeated or redundant code after the main solution
- Struggles with complex data structures (graphs, trees, queues)
- Does not reliably handle class inheritance patterns
- Can generate incorrect list indexing operations
- May continue generating text beyond the intended solution
- Limited to 512 token context window
- Not suitable for production use without output post-processing
**Recommended Use Cases**
- Code autocompletion in lightweight IDEs
- Educational tool for Python beginners
- Rapid prototyping of simple functions
- Embedded systems with limited computational resources
- Offline code assistance on mobile devices
**Not Recommended For**
- Complex algorithm implementation
- Production code generation without human review
- Tasks requiring deep contextual understanding
- Generating large codebases
- Security-critical applications
**Usage Example**
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
model_path = "stas122/stentor_python_30m"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path)
prompt = "### Task: Write a function that checks if a number is even\n\n### Solution:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.2)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
```
**Ethical Considerations**
This model is intended for educational and development assistance purposes. Users should verify all generated code before deployment, particularly for security-sensitive applications. The model may occasionally produce incorrect or inefficient code and should not be relied upon as the sole source of truth for programming tasks.
**Contact**
For questions or feedback about this model, please open an issue on the Hugging Face repository. |