| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - StentorLabs/Stentor-30M |
| pipeline_tag: text-generation |
| tags: |
| - python |
| - code-generation |
| - tiny-model |
| - code |
| library_name: transformers |
| --- |
| **Model Description** |
|
|
| Stentor Python 30M is a compact language model specifically fine-tuned for Python code generation and autocompletion tasks. Based on the Stentor-30M architecture, this model contains 30 million parameters and is designed to run efficiently on resource-constrained devices including mobile phones and embedded systems. |
|
|
| **Model Details** |
|
|
| - **Developed by:** Experimental fine-tuning project |
| - **Model type:** Causal language model (LlamaForCausalLM) |
| - **Language:** Python code, English instructions |
| - **Parameters:** 30,419,712 |
| - **Context length:** 512 tokens |
| - **Model size:** 60 MB (FP16), 30 MB (INT8) |
| - **License:** Apache 2.0 |
|
|
| **Training Data** |
|
|
| The model was fine-tuned on a curated dataset of 872 Python examples, including: |
|
|
| - Basic algorithms (factorial, prime numbers, list operations) |
| - Class implementations (Stack, BankAccount, Rectangle, Circle) |
| - Recursive functions (quicksort, Fibonacci) |
| - String manipulation (palindrome, anagram, vowel counting) |
| - MBPP (Mostly Basic Python Problems) dataset tasks |
|
|
| All examples follow a consistent format with "### Task:" instruction and "### Solution:" code block. |
|
|
| **Training Process** |
|
|
| The fine-tuning process involved multiple stages: |
|
|
| 1. Base model: Stentor-30M |
| 2. Initial fine-tuning on 50k examples |
| 3. Multiple correction rounds with progressively lower learning rates |
| 4. Final detoxification training with learning rate 3e-7 to remove undesirable patterns |
|
|
| **Evaluation Results** |
|
|
| The model was evaluated on several test categories: |
|
|
| | Category | Pass Rate | Notes | |
| |----------|-----------|-------| |
| | Basic functions | 80% | Factorial, prime check, etc. | |
| | Classes from training set | 100% | Stack, BankAccount, Rectangle | |
| | New complex classes | 33% | Graph, Queue, inheritance | |
| | Function signatures (MBPP) | 100% | Correctly generates def statements | |
|
|
| **Capabilities** |
|
|
| - Generates Python functions from natural language descriptions |
| - Implements basic algorithms (factorial, prime check, palindrome) |
| - Creates class definitions with methods (Stack, BankAccount, Rectangle) |
| - Handles recursive functions (quicksort, Fibonacci) |
| - Produces syntactically correct function signatures |
|
|
| **Limitations** |
|
|
| - May produce repeated or redundant code after the main solution |
| - Struggles with complex data structures (graphs, trees, queues) |
| - Does not reliably handle class inheritance patterns |
| - Can generate incorrect list indexing operations |
| - May continue generating text beyond the intended solution |
| - Limited to 512 token context window |
| - Not suitable for production use without output post-processing |
|
|
| **Recommended Use Cases** |
|
|
| - Code autocompletion in lightweight IDEs |
| - Educational tool for Python beginners |
| - Rapid prototyping of simple functions |
| - Embedded systems with limited computational resources |
| - Offline code assistance on mobile devices |
|
|
| **Not Recommended For** |
|
|
| - Complex algorithm implementation |
| - Production code generation without human review |
| - Tasks requiring deep contextual understanding |
| - Generating large codebases |
| - Security-critical applications |
|
|
| **Usage Example** |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| model_path = "stas122/stentor_python_30m" |
| tokenizer = AutoTokenizer.from_pretrained(model_path) |
| model = AutoModelForCausalLM.from_pretrained(model_path) |
| |
| prompt = "### Task: Write a function that checks if a number is even\n\n### Solution:\n" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.2) |
| response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| ``` |
|
|
| **Ethical Considerations** |
|
|
| This model is intended for educational and development assistance purposes. Users should verify all generated code before deployment, particularly for security-sensitive applications. The model may occasionally produce incorrect or inefficient code and should not be relied upon as the sole source of truth for programming tasks. |
|
|
| **Contact** |
|
|
| For questions or feedback about this model, please open an issue on the Hugging Face repository. |
