Qwen

/

Qwen3-Coder-480B-A35B-Instruct

like 1.34k

Follow

Qwen 87.7k

hello,

I try to run Qwen/Qwen3-Coder-480B-A35B-Instruct on 8x H200 with VLLM, during model downloading its never complete, the *.incomplete file stuck on 9.3GB when downloading model-00001-of-00049.safetensors

Did someone facing same issue?

Stuck Process:
Downloading 'model-00001-of-00049.safetensors' to '/models/models--Qwen--Qwen3-Coder-480B-A35B-Instruct-FP8/blobs/da5e1fb3695be02926ec09f8d46d5fe0499ab5bad792973e20467936c157f965.incomplete'

------------ additional info ------------
initContainers:
- name: model-downloader
image: python:3.10-slim
command: ["/bin/bash", "-c"]
args:
- |
#pip install huggingface-hub tqdm hf-transfer
pip install huggingface_hub tqdm hf-transfer

        python -c "
        import os
        import sys
        import time
        from huggingface_hub import snapshot_download, logging
        from pathlib import Path
        #from urllib3.exceptions import InsecureRequestWarning

        # Enable progress bars
        logging.set_verbosity_info()

        #model_id = 'Qwen/Qwen2.5-Coder-32B-Instruct-AWQ'
        model_id = 'Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8'
        cache_dir = '/models'

        print('='*60)
        print(f'Model Download Monitor')
        print(f'Model: {model_id}')
        print(f'Cache Directory: {cache_dir}')
        print('='*60)

        # Create cache directory if it doesn't exist
        Path(cache_dir).mkdir(parents=True, exist_ok=True)

        # Check if model already exists
        model_path = Path(cache_dir) / 'models--Qwen--Qwen3-Coder-480B-A35B-Instruct-FP8'
        #model_path = Path(cache_dir) / 'models--Qwen--Qwen2.5-Coder-32B-Instruct'
        #model_path = Path(cache_dir) / 'models--Qwen--Qwen2.5-Coder-32B-Instruct-AWQ'
        if model_path.exists():
            print(f'Model directory exists. Checking integrity...')
            existing_files = list(model_path.rglob('*.safetensors'))
            print(f'Found {len(existing_files)} safetensor files')

        try:
            print(f'Starting download at {time.strftime(\"%Y-%m-%d %H:%M:%S\")}')

            # Download with better progress tracking
            local_dir = snapshot_download(
                repo_id=model_id,
                cache_dir=cache_dir,
                token=os.environ.get('HF_TOKEN'),
                local_dir_use_symlinks=False,
                force_download=False,  # Changed from resume_download
                max_workers=4
            )

            print(f'\\nDownload completed at {time.strftime(\"%Y-%m-%d %H:%M:%S\")}')
            print(f'Model location: {local_dir}')

            # Verify download
            safetensor_files = list(Path(local_dir).glob('*.safetensors'))
            total_size = sum(f.stat().st_size for f in safetensor_files)
            print(f'\\nVerification:')
            print(f'- Number of safetensor files: {len(safetensor_files)}')
            print(f'- Total size: {total_size / (1024**3):.2f} GB')

            print('\\nModel download successful!')

        except Exception as e:
            print(f'\\nError downloading model: {e}')
            import traceback
            traceback.print_exc()
            sys.exit(1)
        "
    env:
    - name: HF_TOKEN
      value: "hf_xxxxxxxxxxx"
    - name: HF_HUB_ENABLE_HF_TRANSFER
      value: "1"
    - name: HF_HUB_DOWNLOAD_TIMEOUT
      value: "7200"
    - name: HF_TRANSFER_CONCURRENCY
      value: "4"
    resources:
      requests:
        memory: "4Gi"
        cpu: "2"
      limits:
        memory: "8Gi"
        cpu: "4"
    volumeMounts:
    - name: model-storage
      mountPath: /models