| --- |
| license: cc0-1.0 |
| datasets: |
| - HuggingFaceFW/fineweb-edu |
| - openbmb/Ultra-FineWeb-L3 |
| language: |
| - en |
| tags: |
| - LLM |
| - LM |
| - Dront |
| - GPT |
| - GPT2 |
| - 200m |
| --- |
| |
|  |
|
|
| # Dront 200m is a GPT-like model trained from scratch on approximately 5 billion tokens. |
|
|
| Context: 1024 tokens |
|
|
| Tokenizer: GPT2 50k |
|
|
| License: cc0-1.0 |
|
|
| # Perplexity |
|
|
| | Model | openbmb/Ultra-FineWeb-L3 | FineWeb-Edu-L4-5 | |
| |---------|--------|--------| |
| | Dront-200m | 11.33 | 13.23 | |
| | VDrontV2-0.1b | 17.52 | 11.01 | |
| | GPT-2 120m | 10.53 | 8.54 | |
|
|
| # Generate Example |
|
|
| Prompt: The Python is a... |
|
|
| ```text |
| |
| The Python is a powerful tool for building complex code, particularly in scenarios involving complex data structures. |
| It supports the development of complex queries, such as identifying a specific value or identifying a specific |
| attribute. This approach supports efficient data handling and high-level analysis. The core of Python is the Visual |
| Basic Language (VML), which is widely used across various programming languages to build complex algorithms. |
| |
| Python is a powerful tool for generating complex code by manipulating data structures. It is especially useful |
| for handling large datasets and complex data types. For example, a Python script can be written to handle a wide |
| range of data types, including tables, graphs, and functions, making it suitable for both data analysis and modeling. |
| |
| In summary, Python is a powerful tool for creating complex systems with complex data structures. Its ability |
| to process large datasets, generate complex models, and automate tasks makes it a valuable tool for developers |
| working on complex problems. By combining Python and VML, you can significantly improve your programming skills and improve your overall coding experience. |
| |
| For those interested in learning more about Python, additional resources are available through dedicated |
| tutorials. These include detailed tutorials on the basics, practical tips for developing Python, and guidance |
| on how to use Python for code development. |
| |
| Python is a powerful tool for building complex programs, especially in environments where data manipulation |
| is critical. It provides a clear overview of the fundamentals of Python, including its syntax, function, and syntax. |
| While Python is not without limitations, it offers a solid foundation for building complex applications. |
| |
| For those seeking to learn Python, several online platforms offer access to tutorials, tutorials, and tutorials |
| on topics like object-oriented programming. These platforms provide a rich collection of tutorials, tutorials, and |
| tutorials designed to help users build and test their skills. |
| |
| For those interested in learning Python, resources such as tutorials and tutorials can further support learning. |
| These resources are available in both English and Spanish, offering guidance on how to use them effectively. |
| |
| In summary, Python is a powerful tool for building complex code, improving performance, and improving code quality. |
| With consistent effort and careful planning, it can become a powerful tool for building complex applications. |
| The benefits of learning Python are well documented, making it a versatile tool that supports both beginners and experienced developers. |
| |
| To begin learning Python, individuals should explore the official Python website at http://www.python.org/learn-to-learn- |
| Python-learn-with-Python.html. This resource offers a comprehensive overview of Python’s features and benefits, along with |
| step-by-step guidance on how to integrate it into their own projects. |
| |
| The Python community has developed a range of tutorials and tutorials designed to help users learn Python from beginners. |
| These include tutorials on using Python to write and debug code, examples of advanced Python libraries, and a guide on using |
| Python for creating and managing complex code. Additionally, the platform includes tutorials on Python for creating and testing |
| code, which helps learners develop strong coding abilities and improve their coding skills. |
| |
| Overall, learning Python is a rewarding and effective method for building complex code. Whether you're new to Python or |
| seeking to expand your understanding, learning Python is a rewarding journey. The benefits of learning Python are vast |
| and varied, making it a valuable addition to your training regimen. |
| |
| For those interested in exploring Python’s potential in education, resources like The Python Tutorial Guide |
| provide extensive information on Python, including tutorials and tutorials on Python, and additional tutorials on Python. |
| |
| If you have questions about Python, feel free to contact us directly. We welcome your questions and |
| suggestions, and we encourage you to share your thoughts and concerns. |
| |
| The Python Tutorial Guide offers a comprehensive overview of Python, covering its core components, functionality, |
| and integration with other programming languages. It covers essential topics such as the fundamentals of Python, the |
| fundamentals of programming, and the advantages of using Python in real-world projects. The guide also explores the |
| principles of Python, including its use cases, features, and advantages in Python, helping users understand the language’s purpose and benefits. |
| |
| The Python Tutorial Guide is a comprehensive resource that covers the fundamentals of Python, including its structure, |
| implementation, and core concepts. It covers core concepts such as loops, loops, and loops, providing a structured way |
| to learn Python quickly and efficiently. |
| |
| Users can also explore Python’s capabilities through a tutorial or online course, which provides detailed explanations |
| and examples. The guide explains the basics of Python, including its syntax, use cases, and the use of tools like Python. |
| Additionally, the guide explains how to create and deploy Python programs using Python, including the use of Python libraries, |
| libraries, and tools for creating and deploying programs. |
| |
| |
| ``` |
|
|
|
|
| # Easy Start ( ! Local !) |
|
|
| ```Python |
| |
| import torch |
| import sys |
| import warnings |
| from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer |
| |
| warnings.filterwarnings('ignore') |
| |
| torch.backends.cudnn.benchmark = True |
| torch.backends.cuda.matmul.allow_tf32 = True |
| torch.set_float32_matmul_precision('high') |
| |
| MODEL_NAME = "Dront-200m" # LOCAL FOLDER |
| DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu") |
| |
| print("Loading model...") |
| tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, local_files_only=True) |
| if tokenizer.pad_token is None: |
| tokenizer.pad_token = tokenizer.eos_token |
| |
| model = AutoModelForCausalLM.from_pretrained( |
| MODEL_NAME, |
| local_files_only=True, |
| torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32, |
| device_map="auto" |
| ) |
| model.eval() |
| print("Model loaded") |
| |
| |
| def generate_text_streaming(prompt): |
| inputs = tokenizer(prompt, return_tensors="pt").to(DEVICE) |
| |
| # Создаем стример для потокового вывода |
| streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True) |
| |
| print("Output: ", end="", flush=True) |
| |
| with torch.no_grad(): |
| outputs = model.generate( |
| **inputs, |
| max_new_tokens=1024, |
| temperature=0.45, |
| do_sample=True, |
| top_p=0.95, |
| repetition_penalty=1.1, |
| pad_token_id=tokenizer.pad_token_id, |
| eos_token_id=tokenizer.eos_token_id, |
| streamer=streamer, # Добавляем стример |
| use_cache=True |
| ) |
| |
| print() # Новая строка после завершения генерации |
| return outputs |
| |
| |
| print("Text generation started! Type 'exit' to quit.") |
| while True: |
| try: |
| user_input = input("Input: ").strip() |
| if user_input.lower() in ['exit', 'quit', 'q']: |
| print("Goodbye!") |
| break |
| if not user_input: |
| continue |
| |
| generate_text_streaming(user_input) |
| |
| except KeyboardInterrupt: |
| print("\nGoodbye!") |
| break |
| except Exception as e: |
| print(f"Error: {e}") |
| continue |
| |
| ``` |
|
|
| Bye! |
