Upload README.md with huggingface_hub

README.md

@@ -11,9 +11,9 @@ source_datasets:
 - amkyawdev/mm-llm-coder-dataset
 ---
 
-# Combined Myanmar LLM Dataset
+# Combined Myanmar LLM Code Dataset
 
-A comprehensive dataset combining three Myanmar-related datasets for training large language models, optimized for code generation and Myanmar language understanding. This dataset follows the ADP (Agent Data Protocol) format with execution feedback.
+A comprehensive dataset combining three Myanmar-related datasets for training large language models, optimized for code generation and Myanmar language understanding.
 
 [English](#english) | [မြန်မာဘာသာ](#myanmar)
 
@@ -25,231 +25,102 @@ A comprehensive dataset combining three Myanmar-related datasets for training la
 
 This dataset combines three source datasets for training LLMs with Myanmar language and coding capabilities:
 
-| Dataset | Description | Samples |
-|---------|-------------|---------|
-| `amkyawdev/myanmar-llm-data` | Myanmar language conversations, translations, Q&A | 20,327 |
-| `amkyawdev/mm-llm-coder-agent-dataset` | Agent workflow for coding tasks | 1,000,020 |
-| `amkyawdev/mm-llm-coder-dataset` | Code generation tasks | 2,000,000 |
+| Source | Dataset | Description | Type | Samples |
+|--------|---------|-------------|------|---------|
+| chat-skill.md | [amkyawdev/myanmar-llm-data](https://huggingface.co/datasets/amkyawdev/myanmar-llm-data) | Myanmar conversations, translations, Q&A | Chat/Skill | ~54,553 |
+| agent-skill.md | [amkyawdev/mm-llm-coder-agent-dataset](https://huggingface.co/datasets/amkyawdev/mm-llm-coder-agent-dataset) | Agent workflow for coding tasks | Agent/Skill | ~1,000,020 |
+| code-skill.md | [amkyawdev/mm-llm-coder-dataset](https://huggingface.co/datasets/amkyawdev/mm-llm-coder-dataset) | Code generation and debugging | Code/Skill | ~2,000,000 |
 
-**Total Samples: 3,020,347**
+**Total Samples: ~3,020,347**
 
-### Storage Files
+### Data Sources
 
-This dataset also connects to storage bucket files for additional data:
+#### 1. chat-skill.md - Myanmar LLM Data (`amkyawdev/myanmar-llm-data`)
 
-| File | Location | Size |
-|------|----------|------|
-| ADP Execution Feedback | `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-adp-execution-feedback.jsonl` | 50.8 MB |
-| Myanmar LLM Clean | `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-clean.jsonl` | 2.79 GB |
-| Myanmar LLM Formatted | `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-formatted.jsonl` | 24.8 MB |
-| Myanmar LLM Data Formatted | `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-data-formatted.jsonl` | 3.7 kB |
+Multi-turn conversations in Burmese and English:
+- **Format**: `messages` (role + content), `tags`
+- **Link**: [View Dataset](https://huggingface.co/datasets/amkyawdev/myanmar-llm-data)
+
+#### 2. agent-skill.md - Coder Agent Dataset (`amkyawdev/mm-llm-coder-agent-dataset`)
+
+Agent workflows with tool usage:
+- **Format**: Agent workflows with `tools_used`, `code_snippets`, `execution_result`
+- **Link**: [View Dataset](https://huggingface.co/datasets/amkyawdev/mm-llm-coder-agent-dataset)
+
+#### 3. code-skill.md - Coder Dataset (`amkyawdev/mm-llm-coder-dataset`)
+
+Code generation and debugging tasks:
+- **Format**: Code Q&A conversations
+- **Link**: [View Dataset](https://huggingface.co/datasets/amkyawdev/mm-llm-coder-dataset)
 
 ### Features
 
 - **Myanmar Language Support**: Native Burmese (မြန်မာစာ) conversations and translations
-- **Code Generation**: Python, JavaScript, TypeScript, and other programming languages
+- **Code Generation**: Python, JavaScript, TypeScript and other programming languages
 - **Agent Workflows**: Multi-step coding tasks with tool usage
-- **Execution Feedback**: Results from code execution including errors and test results
 - **Quality Metrics**: Ratings, validation status, and complexity scores
 
-### Dataset Structure
-
-Each sample contains:
-
-```python
-{
-# Core Fields
-"messages": [
-{"role": "system", "content": "You are a helpful assistant."},
-{"role": "user", "content": "User input here"},
-{"role": "assistant", "content": "Response here"}
-],
-# Task Definition
-"instruction": "Task instruction",
-"category": "Task category",
-"language": "en or my",
-"difficulty": "beginner, intermediate, or advanced",
-"response": "Expected response/output",
-"task_type": "Type of task",
-# Execution Feedback
-"execution_feedback": {
-"status": "completed or pending_validation",
-"result": "Execution result",
-"error_type": "runtime_error, syntax_error, etc.",
-"error_message": "Error details",
-"execution_time_ms": 1000,
-},
-# Extended Fields
-"framework": "React, Express, etc.",
-"runtime": "Node.js, Python, etc.",
-"database": "MongoDB, PostgreSQL, etc.",
-"validated": True/False,
-"rating": 0.0 to 1.0,
-"complexity_score": 1 to 10,
-# Metadata
-"metadata": {
-"created_at": "2024-01-01T00:00:00",
-"difficulty": "beginner/intermediate/advanced",
-}
-}
-```
-
 ### Usage
 
 ```python
 from datasets import load_dataset
 
-# Load the entire dataset
-dataset = load_dataset("amkyawdev/mm-llm-code-v3")
+# Load chat-skill dataset (Myanmar conversations)
+chat_ds = load_dataset("amkyawdev/myanmar-llm-data")
+print("Chat data:", chat_ds)
 
-# Load specific split
-train_ds = load_dataset("amkyawdev/mm-llm-code-v3", split="train")
+# Load agent-skill dataset (Agent workflows)
+agent_ds = load_dataset("amkyawdev/mm-llm-coder-agent-dataset")
+print("Agent data:", agent_ds)
 
-# Access a single sample
-sample = train_ds[0]
-print(sample["messages"])
-print(sample["execution_feedback"])
+# Load code-skill dataset (Code generation)
+code_ds = load_dataset("amkyawdev/mm-llm-coder-dataset")
+print("Code data:", code_ds)
 
-# Load from storage bucket directly
-from datasets import load_dataset
-ds = load_dataset("hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-clean.jsonl")
-```
-
-### Storage Bucket Access
-
-Access data directly from storage bucket:
-
-```python
-# ADP Execution Feedback (50.8 MB)
-from datasets import load_dataset
-adp_ds = load_dataset("hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-adp-execution-feedback.jsonl")
-
-# Clean dataset (2.79 GB)
-clean_ds = load_dataset("hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-clean.jsonl")
-
-# Formatted dataset (24.8 MB)
-formatted_ds = load_dataset("hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-formatted.jsonl")
+# Access specific samples
+chat_sample = chat_ds["train"][0]
+print("Messages:", chat_sample["messages"])
+print("Tags:", chat_sample["tags"])
 ```
 
 ### Use Cases
 
-- **Myanmar Language Models**: Training LLMs that understand Burmese/Myanmar language
-- **Code Generation**: Training models for programming tasks
+- **Myanmar Language Models**: Train LLMs that understand Burmese
+- **Code Generation**: Train models for programming tasks
+- **Agent Workflows**: Train coding agents with tool usage
+- **Debugging**: Fix common coding errors
 - **Multilingual Tasks**: Translation between English and Myanmar
-- **Chatbots**: Building conversational AI for Myanmar speakers
-- **Agent Workflows**: Training coding agents with execution feedback
-
-### Data Format
-
-This dataset is available in multiple formats:
-
-1. **HuggingFace Dataset**: Standard format with all fields
-2. **ADP Format**: JSONL format with execution feedback
-   - Location: `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-adp-execution-feedback.jsonl`
-3. **Clean Format**: Cleaned and processed data
-   - Location: `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-clean.jsonl`
-4. **Formatted Format**: Pre-formatted for training
-   - Location: `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-formatted.jsonl`
 
 ### License
 
 Apache 2.0 License
 
-### Citation
-
-```bibtex
-@dataset{combined_myanmar_llm,
-title={Combined Myanmar LLM Dataset},
-author={amkyawdev},
-year={2024},
-url={https://huggingface.co/datasets/amkyawdev/mm-llm-code-v3}
-}
-```
-
 ---
 
 ## မြန်မာဘာသာ
 
 ### အနှစ်ချူပါ
 
-ဒီ dataset သည် မြန်မာစာ နှင့် ကုဒ်ရေးလုပ်တဲ့ LLM များကို လေ့ကျင့်ဖို့အတွက် သုံးခုေကာင်း ဒေါင်းလုဒ်များကို ပေါင်းစပ်ထားပပါ။ ADP (Agent Data Protocol) format နှင့် ပါဝင်ပပါ။
-
-| ဒေါင်းလုဒ် | ဖော်ပါ | ပါဝင်မှု |
-|---------|----------|----------|
-| `amkyawdev/myanmar-llm-data` | မြန်မာစာပါးဆက်၊ ဘာသာပြန်၊ Q&A | ၂၀,၃၂၇ |
-| `amkyawdev/mm-llm-coder-agent-dataset` | ကုဒ်ရေးလုပ်တဲ့ agents များ | ၁,၀၀၀,၀၂၀ |
-| `amkyawdev/mm-llm-coder-dataset` | ကုဒ်ထုတ်လုပ်တဲ့ အလုပ်များ | ၂,၀၀၀,၀၀၀ |
-
-**ပါဝင်မှု စုစုပါး: ၃,၀၂၀,၃၄၇**
-
-### Storage ဖိုင်များ
-
-Storage bucket မှာရှိတဲ့ ဖိုင်များနှင့်လည်း ပါဝင်ပပါး။
-
-| ဖိုင် | တည်နေရာ | အရွယ်အစား |
-|------|----------|------|
-| ADP Execution Feedback | `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-adp-execution-feedback.jsonl` | ၅၀.၈ MB |
-| Myanmar LLM Clean | `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-clean.jsonl` | ၂.၇၉ GB |
-| Myanmar LLM Formatted | `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-formatted.jsonl` | ၂၄.၈ MB |
-| Myanmar LLM Data Formatted | `hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-data-formatted.jsonl` | ၃.၇ kB |
+ဒီ dataset သည် မြန်မာစာ နှင့် ကုဒ်ရေးလုပ်တဲ့ LLM များကို လေ့ကျင့်ဖို့အတွက် dataset ၃ ခုကို ပေါင်းစပ်ထားပါ။
 
-### ပါဝင်တဲ့ အချက်များ
+| Source | Dataset | Description | Samples |
+|--------|---------|-------------|---------|
+| chat-skill.md | `amkyawdev/myanmar-llm-data` | မြန်မ���စာပါးဆက် | ~54,553 |
+| agent-skill.md | `amkyawdev/mm-llm-coder-agent-dataset` | Agent workflow | ~1,000,020 |
+| code-skill.md | `amkyawdev/mm-llm-coder-dataset` | ကုဒ်ထုတ်လုပ်ခြင်း | ~2,000,000 |
 
-- **မြန်မာစာ ပါဝင်မှု**: မြန်မာစာ ပါးဆက်များနှင့် ဘာသာပြန်များ
-- **ကုဒ်ထုတ်လုပ်ခြင်း**: Python, JavaScript, TypeScript နှင့် အခြားပရိုဂရမ်ဘာသာများ
-- **Agent Workflows**: အလုပ်အများအဆင့်ဆင့်လုပ်တဲ့ ကုဒ်ရေးလုပ်တဲ့ အလုပ်များ
-- **Execution Feedback**: ကုဒ်လုပ်ခါင်းရလာဒ်၊ အမှားများ၊ စမ်းသပ်ချက်များ
-- **အရည်အသွေး**: Rating၊ validation status၊ complexity scores
-
-### ဖွဲ့စည်းပါ
-
-```python
-# အခြေခံ ဖိုင်များ
-"messages": [...],
-"instruction": "...",
-"category": "...",
-"language": "my သို့မဟုတ် en",
-"difficulty": "beginner, intermediate, advanced",
-"response": "...",
-"task_type": "...",
-# Execution Feedback
-"execution_feedback": {
-"status": "completed သို့မဟုတ် pending_validation",
-"result": "...",
-"error_type": "...",
-"error_message": "...",
-},
-# နောက်ထပ် ဖိုင်များ
-"metadata": {...}
-}
-```
+**ပါဝင်မှု စုစုပါး: ~3,020,347**
 
 ### သုံးပါ
 
 ```python
 from datasets import load_dataset
-dataset = load_dataset("amkyawdev/mm-llm-code-v3")
-train_ds = dataset["train"]
-sample = train_ds[0]
 
-# Storage bucket မှာရှိတဲ့ ဖိုင်များကို သုံးဖို့
-from datasets import load_dataset
-adp_ds = load_dataset("hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-adp-execution-feedback.jsonl")
-clean_ds = load_dataset("hf://buckets/amkyawdev/mm-llm-storage/myanmar-llm-clean.jsonl")
+chat_ds = load_dataset("amkyawdev/myanmar-llm-data")
+agent_ds = load_dataset("amkyawdev/mm-llm-coder-agent-dataset")
+code_ds = load_dataset("amkyawdev/mm-llm-coder-dataset")
 ```
 
-### သုံးပြုနည်း အမျိုးမျိုး
-
-- **မြန်မာစာ LLM**: မြန်မာစာနားလည်တဲ့ LLM များကို လေ့ကျင့်ခြင်း
-- **ကုဒ်ထုတ်လုပ်ခြင်း**: ပရိုဂရမ်ရေးလုပ်တဲ့ မော်ဒယ်များကို လေ့ကျင့်ခြင်း
-- **ဘာသာပြန်**: အင်္ဂလိပ်နဲ့ မြန်မာပါးကြား ပြန်ဆိုခြင်း
-- **ခွန်းဖြေ**: မြန်မာစာပါးဆက်ပါ AI များကို ဆောက်လုပ်ခြင်း
-- **Agent Workflows**: Execution feedback နဲ့ ကုဒ် agents များကို လေ့ကျင့်ခြင်း
-
 ### License
 
 Apache 2.0 License
-
-### Dataset URL
-
-https://huggingface.co/datasets/amkyawdev/mm-llm-code-v3