.opencode/plans/fix_custom_model_info.md

Implementation Plan: Fix Model Information for Custom GGUF Models

Overview

Fix the bug where "Model Information" remains empty when a custom GGUF model is loaded.

Selected Approach: Option A - Store metadata in Gradio State variables UI Style: Cards/panels with dense layout Priority: Bug fix first, then UI improvements

---

Bug Analysis

Problem

get_model_info() reads from static AVAILABLE_MODELS["custom_hf"] which has:

• repo_id = None
• filename = None

The actual values entered by the user are never stored or passed to the info display function.

Solution

Store actual custom model metadata in dedicated Gradio State variables and pass them to get_model_info() when model_key == "custom_hf".

---

Implementation Steps

Step 1: Add Custom Model Metadata State

Location: UI section (~line 1730), alongside other states

# Custom model metadata state (stores actual repo_id and filename when loaded)
custom_model_metadata = gr.State({
    "repo_id": None,
    "filename": None,
    "size_mb": 0,
})

Step 2: Modify get_model_info() Function

Location: ~line 904

Current signature:

def get_model_info(model_key: str, n_threads: int = 2):

New signature:

def get_model_info(model_key: str, n_threads: int = 2, custom_metadata: dict = None):

Logic change (inside function):

if model_key == "custom_hf" and custom_metadata:
    # Use actual metadata from loaded custom model
    repo_id = custom_metadata.get("repo_id", "Not loaded")
    filename = custom_metadata.get("filename", "Not selected")
    size_mb = custom_metadata.get("size_mb", 0)
    
    # Parse quantization from filename
    quant = parse_quantization(filename) if filename else "Unknown"
    
    info_text = (
        f"## 🤖 Custom HF GGUF Model\n\n"
        f"### 📊 Model Metadata\n"
        f"| Property | Value |\n"
        f"|----------|-------|\n"
        f"| **Repository** | `{repo_id}` |\n"
        f"| **GGUF File** | `{filename}` |\n"
        f"| **Quantization** | `{quant}` |\n"
        f"| **File Size** | {size_mb:.1f} MB |\n"
        f"| **Context** | 8,192 tokens |\n"
        f"| **Threads** | {n_threads} |\n\n"
        f"⚠️ Note: Custom models use conservative defaults (CPU-only, smaller context)."
    )
else:
    # Use existing logic for predefined models
    m = AVAILABLE_MODELS[model_key]
    # ... existing code ...

Step 3: Update load_custom_model_selected() to Store Metadata

Location: Event handler section (~line 1927)

Current function (simplified):

def load_custom_model_selected(repo_id, selected_file_display, files_data):
    filename = selected_file_display.split(" | ")[0].replace("📄 ", "").strip()
    llm, load_msg = load_custom_model_from_hf(repo_id, filename, n_threads)
    if llm is None:
        return gr.update(visible=True, value=error), gr.update(visible=True), None
    else:
        return gr.update(visible=True, value=success), gr.update(visible=False), llm

New function:

def load_custom_model_selected(repo_id, selected_file_display, files_data):
    filename = selected_file_display.split(" | ")[0].replace("📄 ", "").strip()
    
    # Extract size from files_data
    size_mb = 0
    for f in files_data:
        if f["name"] == filename:
            size_mb = f.get("size_mb", 0)
            break
    
    llm, load_msg = load_custom_model_from_hf(repo_id, filename, n_threads)
    
    if llm is None:
        return (
            gr.update(visible=True, value=f"❌ {load_msg}"),
            gr.update(visible=True),
            None,
            {"repo_id": None, "filename": None, "size_mb": 0},  # Clear metadata
        )
    else:
        # Create metadata dict
        metadata = {
            "repo_id": repo_id,
            "filename": filename,
            "size_mb": size_mb,
        }
        return (
            gr.update(visible=True, value=f"✅ {load_msg}"),
            gr.update(visible=False),
            llm,
            metadata,  # Return metadata to store in state
        )

Update the click event handler:

load_btn.click(
    fn=load_custom_model_selected,
    inputs=[model_search_input, custom_file_dropdown, custom_repo_files],
    outputs=[custom_status, retry_btn, custom_model_state, custom_model_metadata],  # Added metadata
)

retry_btn.click(
    fn=load_custom_model_selected,
    inputs=[model_search_input, custom_file_dropdown, custom_repo_files],
    outputs=[custom_status, retry_btn, custom_model_state, custom_model_metadata],  # Added metadata
)

Step 4: Update Model Info Display Event Handler

Location: ~line 1711, update_settings_on_model_change() function

Current:

def update_settings_on_model_change(model_key, n_threads):
    info, _, _, _ = get_model_info(model_key, n_threads=n_threads)
    # ... return info ...

New:

def update_settings_on_model_change(model_key, n_threads, custom_metadata):
    info, _, _, _ = get_model_info(model_key, n_threads=n_threads, custom_metadata=custom_metadata)
    # ... return info ...

Update the event handler:

model_dropdown.change(
    fn=update_settings_on_model_change,
    inputs=[model_dropdown, n_threads_display, custom_model_metadata],  # Added metadata
    outputs=[info_output, max_tokens, reasoning_checkbox, n_ctx_display, 
             thinking_accordion, thinking_output, enable_reasoning],
)

Step 5: Update Submit Button Handler

Location: ~line 2010

Update inputs to include custom_model_metadata:

submit_btn.click(
    fn=summarize_streaming,
    inputs=[file_input, model_dropdown, enable_reasoning, max_tokens, temperature_slider, 
            top_p, top_k, language_selector, thread_config_dropdown, custom_threads_slider, 
            custom_model_state, custom_model_metadata],  # Added metadata
    outputs=[thinking_output, summary_output, info_output, metrics_state, system_prompt_debug],
    show_progress="full"
)

Step 6: Update summarize_streaming() Function

Location: ~line 1080

Update function signature:

def summarize_streaming(
    file_obj,
    model_key: str,
    enable_reasoning: bool = True,
    max_tokens: int = 2048,
    temperature: float = 0.6,
    top_p: float = None,
    top_k: int = None,
    output_language: str = "en",
    thread_config: str = "free",
    custom_threads: int = 4,
    custom_model_state: Any = None,
    custom_model_metadata: dict = None,  # NEW parameter
) -> Generator[Tuple[str, str, str, dict, str], None, None]:

Update model info generation:

# Get base model info with current thread configuration
info_text, _, _, _ = get_model_info(model_key, n_threads=n_threads, custom_metadata=custom_model_metadata)

---

Files to Modify

1. app.py - Main changes:
   • Line ~1730: Add custom_model_metadata state
   • Line ~904: Modify get_model_info() signature and logic
   • Line ~1927: Update load_custom_model_selected() to return metadata
   • Line ~1711: Update update_settings_on_model_change() to accept metadata
   • Line ~1080: Update summarize_streaming() signature
   • Line ~2010: Update submit button event handler inputs

---

Testing Plan

1. Select "🔧 Custom HF GGUF..." from model dropdown
2. Type "llama" in search box and select a model
3. Verify file dropdown auto-populates
4. Select a GGUF file
5. Click "Load Selected Model"
6. Verify Model Information now shows:
   • Repository: actual repo ID
   • GGUF File: actual filename
   • Quantization: parsed from filename
   • File Size: actual size
   • Context: 8192 tokens
   • Note about conservative defaults
7. Generate a summary
8. Verify model info remains correct during and after generation
9. Switch to a different predefined model
10. Verify model info updates correctly
11. Switch back to custom model
12. Verify it shows "Not loaded" state until new custom model is loaded

---

Lines to Modify

Function/Component	Line Range	Changes
State declarations	~1730	Add custom_model_metadata
get_model_info()	~904-947	Add custom_metadata param, handle custom_hf
load_custom_model_selected()	~1927-1960	Return metadata dict
Load button click	~1970	Add custom_model_metadata to outputs
update_settings_on_model_change()	~1711-1720	Accept metadata param
Model dropdown change	~1721	Add custom_model_metadata to inputs
summarize_streaming() signature	~1080	Add custom_model_metadata param
Submit button click	~2010	Add custom_model_metadata to inputs

---

Expected Result

After the fix, when a custom GGUF model is loaded:

• ✅ Model Information displays actual repo_id and filename
• ✅ Quantization level is parsed and shown
• ✅ File size is displayed
• ✅ Context window shows correct value
• ✅ Information updates correctly when switching models

---

Ready to implement the bug fix? Say "implement bug fix" and I'll proceed!