| | --- |
| | license: apache-2.0 |
| | base_model: kmd2525/v8_stage3_xml-merged |
| | tags: |
| | - structured-output |
| | - json |
| | - csv |
| | - yaml |
| | - xml |
| | - toml |
| | - sft |
| | - sequential-format-learning |
| | language: |
| | - en |
| | - ja |
| | --- |
| | |
| | # v8 Final Model - Sequential Format Learning |
| |
|
| | ## Model Description |
| |
|
| | This is the **final model** of the Sequential Format Learning (v8 strategy) for structured data output. |
| | Trained through 4 stages to achieve optimal performance across all structured formats. |
| |
|
| | ### Training Strategy |
| |
|
| | Based on Person U's approach that achieved 0.84 on the leaderboard: |
| | - Train one format at a time, building format-specific expertise progressively |
| | - Merge LoRA to base model after each stage |
| | - TOML learning comes from other formats (Person T discovery) |
| |
|
| | ### Completed Training Stages |
| |
|
| | | Stage | Formats | Samples | Target | |
| | |-------|---------|---------|--------| |
| | | Stage 1 | JSON, CSV | 800 | 100% parse success | |
| | | Stage 2 | YAML | 500 | 100% parse success | |
| | | Stage 3 | XML | 500 | 95%+ parse success | |
| | | Stage 4 | Mixed (all formats) | 1000 | TOML 90%+, overall optimization | |
| |
|
| | ### Stage 4 (Final) Details |
| |
|
| | - **Base Model**: `kmd2525/v8_stage3_xml-merged` (Stage 3 merged model) |
| | - **Dataset**: 1000 samples, balanced across all 5 formats (200 each) |
| | - **Goal**: Final tuning for TOML and overall balance |
| |
|
| | ### Training Parameters |
| |
|
| | - MAX_SEQ_LEN: 1024 |
| | - EPOCHS: 2 |
| | - Learning Rate: 2e-05 (lower for final stage) |
| | - LoRA R: 64, Alpha: 128 |
| |
|
| | ### Expected Performance |
| |
|
| | | Format | Target | |
| | |--------|--------| |
| | | JSON | 100% | |
| | | CSV | 100% | |
| | | YAML | 100% | |
| | | XML | 95%+ | |
| | | TOML | 90%+ | |
| |
|
| | **Target LB Score: 0.8+** |
| |
|
| | ### Usage |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | |
| | model = AutoModelForCausalLM.from_pretrained("kmd2525/v8_final_model") |
| | tokenizer = AutoTokenizer.from_pretrained("kmd2525/v8_final_model") |
| | |
| | # Example inference |
| | messages = [ |
| | {"role": "user", "content": "Convert the following to JSON: name=Alice, age=30"} |
| | ] |
| | input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| | inputs = tokenizer(input_text, return_tensors="pt") |
| | outputs = model.generate(**inputs, max_new_tokens=512) |
| | print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| | ``` |
| |
|
| | ### Acknowledgments |
| |
|
| | This model was developed based on insights from the StructEval competition community, |
| | particularly Person U's sequential format learning approach and Person T's discovery about TOML learning. |
| |
|
