"""Pinned, independently validated release workflow for Image21-Turbo-INT8."""
import argparse
import hashlib
import json
import os
import shutil
import subprocess
import sys
import statistics
import time
from urllib.parse import quote
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
ARTIFACTS = ROOT / 'artifacts/turbo'
BASE = ROOT / 'models/turbo-bf16'
QUANT = ROOT / 'models/turbo-int8'
REPO = 'Qwen/Qwen-Image-2.1-Turbo'
REVISION = 'd65dbc9a7e8f6b5479e33dee6030eaab2a906509'
NAME = 'Image21-Turbo-INT8'
DIFFUSERS_COMMIT = 'da1d3829cf08d4f329b526d89e17cc035c049d8d'
SIGMAS = [1.0, 0.978453, 0.95418, 0.926626, 0.89508, 0.845148, 0.704534, 0.414568]
CORE_CASES = {'portrait', 'english_text', 'chinese_text', 'composition', 'texture',
              'rgba', 'edit', 'young_chinese_woman', 'edit_boat'}


def configure_environment():
    os.environ['HF_HOME'] = str(ROOT / '.cache/huggingface')
    os.environ['NUMBA_CACHE_DIR'] = str(ROOT / '.cache/numba')
    os.environ['HF_HUB_DISABLE_XET'] = '1'
    os.environ['HF_HUB_DOWNLOAD_TIMEOUT'] = '120'
    os.environ['HF_HUB_ETAG_TIMEOUT'] = '60'
    os.environ['PYTHONUNBUFFERED'] = '1'
    os.environ['PYTHONUTF8'] = '1'
    os.environ['MODELSCOPE_DOWNLOAD_PARALLELS'] = '8'


def write_json(path, data):
    path = Path(path)
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(json.dumps(data, indent=2, ensure_ascii=False) + '\n', encoding='utf-8', newline='\n')


def selected(name):
    return (name in ('LICENSE', 'README.md', 'model_index.json')
            or name.startswith(('transformer/', 'text_encoder/', 'vae/', 'processor/', 'scheduler/'))
            or name == 'assets/turbo-boat-input.png')


def verify_upstream_file(path, entry):
    from scripts.integrity import sha256
    if path.stat().st_size != entry['size']:
        raise ValueError('Upstream size mismatch: ' + entry['rfilename'])
    if entry.get('lfs'):
        if sha256(path) != entry['lfs']['sha256']:
            raise ValueError('Upstream SHA256 mismatch: ' + entry['rfilename'])
    else:
        data = path.read_bytes()
        if hashlib.sha1(f'blob {len(data)}\0'.encode() + data).hexdigest() != entry['blobId']:
            raise ValueError('Upstream Git blob mismatch: ' + entry['rfilename'])
    return {'path': entry['rfilename'], 'size': entry['size'], 'sha256': sha256(path)}


def download(source):
    import requests
    from concurrent.futures import ThreadPoolExecutor
    from scripts.integrity import sha256
    upstream = ARTIFACTS / 'upstream'
    upstream.mkdir(parents=True, exist_ok=True)
    response = requests.get(f'https://huggingface.co/api/models/{REPO}/revision/{REVISION}?blobs=true', timeout=60)
    response.raise_for_status()
    info = response.json()
    if info['sha'] != REVISION:
        raise ValueError('Unexpected upstream revision')
    write_json(upstream / 'huggingface-model-info.json', info)
    entries = [e for e in info['siblings'] if selected(e['rfilename'])]
    BASE.mkdir(parents=True, exist_ok=True)
    # Official ModelScope weights are accepted only when they match the pinned HF release.
    ms_rows = {}
    if source == 'modelscope':
        from modelscope.hub.api import HubApi
        files = HubApi().get_model_files(REPO, revision='master', recursive=True)
        write_json(upstream / 'modelscope-files.json', files)
        ms_rows = {e['Path']: e for e in files if e['Type'] == 'blob'}
        for e in entries:
            if e['rfilename'].endswith('.safetensors'):
                other = ms_rows.get(e['rfilename'], {})
                if other.get('Sha256') != e['lfs']['sha256'] or other.get('Size') != e['size']:
                    raise ValueError('Official platform weight mismatch: ' + e['rfilename'])
    def fetch(e):
        name = e['rfilename']
        path = BASE / name
        path.parent.mkdir(parents=True, exist_ok=True)
        if path.exists() and path.stat().st_size == e['size']:
            return verify_upstream_file(path, e)
        if name.endswith('.safetensors') and source == 'modelscope':
            from modelscope.hub.file_download import model_file_download
            print('Downloading official ModelScope weight: ' + name, flush=True)
            model_file_download(REPO, name, revision='master', local_dir=str(BASE))
        else:
            from huggingface_hub import hf_hub_download
            print('Downloading pinned HF file: ' + name, flush=True)
            hf_hub_download(REPO, name, revision=REVISION, local_dir=str(BASE))
        return verify_upstream_file(path, e)
    with ThreadPoolExecutor(max_workers=4) as pool:
        rows = list(pool.map(fetch, entries))
    write_json(BASE / 'upstream.json', {'base_model': REPO, 'base_revision': REVISION})
    write_json(ARTIFACTS / 'download-verification.json', {'base_model': REPO, 'revision': REVISION, 'files': rows})
    for name in ('README.md', 'LICENSE', 'model_index.json'):
        shutil.copyfile(BASE / name, upstream / name)
    index = json.loads((BASE / 'model_index.json').read_text())
    if index.get('sample_sigmas') != SIGMAS:
        raise ValueError('Expected the official eight-step pipeline schedule')
    print(f'Verified {len(rows)} official files; schedule: {index["sample_sigmas"]}', flush=True)


def run_module(module, *args):
    subprocess.run([sys.executable, '-m', module, *map(str, args)], cwd=ROOT, check=True)


def quantize():
    from scripts.integrity import verify_files
    from scripts.release import annotate
    verified = json.loads((ARTIFACTS / 'download-verification.json').read_text())
    if verified['revision'] != REVISION or verified['base_model'] != REPO:
        raise ValueError('Source provenance mismatch')
    verify_files(BASE, verified['files'])
    known = {r['path']: r for r in verified['files']}
    for component in ('transformer', 'text_encoder'):
        report_path = QUANT / (component + '-quantization.json')
        if report_path.exists():
            report = json.loads(report_path.read_text())
            if not report['source_files'] or any(known.get(r['path']) != r for r in report['source_files']):
                raise ValueError('Existing quantized component has different source files')
            print('Keeping completed component from verified Turbo source: ' + component, flush=True)
        else:
            run_module('scripts.quantize', '--source', BASE, '--output', QUANT,
                       '--component', component, '--name', NAME)
    from scripts.quantize import copy_support
    copy_support(BASE, QUANT, NAME)
    annotate(QUANT, base_model=REPO, revision=REVISION, diffusers_commit=DIFFUSERS_COMMIT)
    if json.loads((QUANT / 'model_index.json').read_text())['sample_sigmas'] != SIGMAS:
        raise ValueError('Quantization changed the official sampling grid')
    print('Saved and annotated independent Turbo INT8 pipeline.', flush=True)


def prepare_cases():
    cases = json.loads((ROOT / 'benchmarks/cases.json').read_text(encoding='utf-8'))
    cases[-1]['input'] = 'artifacts/turbo/evaluation/bf16/portrait-s42.png'
    cases[-1]['seeds'] = [1000042, 1000123]
    cases += json.loads((ROOT / 'benchmarks/young_woman.json').read_text(encoding='utf-8'))
    cases.append({'id': 'edit_boat', 'category': 'image_editing',
                  'prompt': 'Turn this yacht sketch into a photorealistic side-view photograph of the same vessel sailing on calm blue sea. Preserve the hull, deckhouses, windows, mast, funnel and lifeboat arrangement. Black hull, white superstructures, yellow funnel. Soft natural daylight, clear sky, the entire vessel in frame.',
                  'input': 'artifacts/turbo/evaluation/inputs/boat.png',
                  'seeds': [20261009, 20261010]})
    input_dir = ARTIFACTS / 'evaluation/inputs'
    input_dir.mkdir(parents=True, exist_ok=True)
    shutil.copyfile(BASE / 'assets/turbo-boat-input.png', input_dir / 'boat.png')
    write_json(ROOT / 'benchmarks/turbo.json', cases)
    native = [c for c in cases if c['id'] in ('portrait', 'edit_boat')]
    write_json(ROOT / 'benchmarks/turbo-native.json', native)


def evaluate(int8_only=False):
    prepare_cases()
    jobs = ([j for j in json.loads((ARTIFACTS / 'evaluation/processes.json').read_text())
             if j['precision'] == 'bf16'] if int8_only else [])
    if int8_only and {(j['suite'], j['returncode']) for j in jobs} != {('core', 0), ('native', 0)}:
        raise ValueError('INT8-only rerun requires both complete BF16 baseline processes')
    for suite, resolution, cases in [('', 1024, 'turbo.json'), ('native', 2048, 'turbo-native.json')]:
        for kind, model in ([('int8', QUANT)] if int8_only else [('bf16', BASE), ('int8', QUANT)]):
            output = ARTIFACTS / 'evaluation' / suite / kind
            print(f'Evaluating {suite or "core"}/{kind}, {resolution}px, official eight-step schedule', flush=True)
            log_path = ARTIFACTS / f'evaluate-{suite or "core"}-{kind}.log'
            with log_path.open('w', encoding='utf-8', newline='\n') as log:
                process = subprocess.run([sys.executable, '-u', '-m', 'scripts.benchmark',
                    '--model', str(model), '--output', str(output), '--cases', 'benchmarks/' + cases,
                    '--steps', '8', '--width', str(resolution), '--height', str(resolution)],
                    cwd=ROOT, stdout=log, stderr=subprocess.STDOUT)
            jobs.append({'suite': suite or 'core', 'precision': kind, 'returncode': process.returncode,
                         'log': log_path.relative_to(ROOT).as_posix()})
            write_json(ARTIFACTS / 'evaluation/processes.json', jobs)
            if process.returncode:
                raise RuntimeError('Benchmark failed; see ' + str(log_path))
        output = ARTIFACTS / 'evaluation' / suite
        run_module('scripts.report', '--baseline', output / 'bf16', '--quantized', output / 'int8', '--output', output)
    print('All four fresh-process benchmarks and paired reports completed.', flush=True)


def validate_evidence(root, model_identity):
    """Verify complete pairs, source identity, actual sampling, inputs and derived metrics."""
    from scripts.integrity import sha256, validate_pair
    from scripts.report import load_records, compare_pixels
    from scripts.provenance import validate_roles
    root = Path(root)
    processes = json.loads((root / 'evaluation/processes.json').read_text())
    if {(j['suite'], j['precision'], j['returncode']) for j in processes} != {
            (s, p, 0) for s in ('core', 'native') for p in ('bf16', 'int8')} or len(processes) != 4:
        raise ValueError('Incomplete fresh-process evaluation')
    source = json.loads((root / 'upstream-verification.json').read_text())
    if source['revision'] != REVISION or source['base_model'] != REPO:
        raise ValueError('Wrong upstream evidence')
    source_rows = {r['path']: r for r in source['files']}
    for suite, resolution, cases_file in [('', 1024, 'turbo.json'), ('native', 2048, 'turbo-native.json')]:
        folder = root / 'evaluation' / suite
        cases_path = root / 'benchmarks' / cases_file
        cases = json.loads(cases_path.read_text(encoding='utf-8'))
        expected_cases = CORE_CASES if suite == '' else {'portrait', 'edit_boat'}
        if {c['id'] for c in cases} != expected_cases or len(cases) != len(expected_cases):
            raise ValueError('Incomplete planned evaluation suite')
        for case in cases:
            expected_seeds = ([1000042, 1000123] if case['id'] == 'edit' else
                              [20261009, 20261010] if case['id'] == 'edit_boat' else [42, 123])
            if case.get('seeds', [42, 123]) != expected_seeds:
                raise ValueError('Unexpected benchmark seeds')
        expected = {(c['id'], s) for c in cases for s in c.get('seeds', [42, 123])}
        records = {kind: load_records(folder / kind) for kind in ('bf16', 'int8')}
        env = {kind: json.loads((folder / kind / 'environment.json').read_text()) for kind in records}
        validate_roles(env['bf16']['model_identity'], env['int8']['model_identity'])
        if env['int8']['model_identity'] != model_identity:
            raise ValueError('Measured INT8 differs from staged model')
        for kind, data in env.items():
            identity = data['model_identity']
            if identity['base_model'] != REPO or identity['base_revision'] != REVISION:
                raise ValueError('Evaluation used a different upstream')
            if (data['sample_sigmas'] != SIGMAS or not data.get('prefix_cache_release') or not data['warmup']
                    or not data['offload_aux_fix'] or data['cases_sha256'] != sha256(cases_path)):
                raise ValueError('Invalid benchmark sampling/settings/cases')
            for field, script in [('benchmark_sha256', 'benchmark.py'), ('runtime_helper_sha256', 'runtime.py')]:
                if data[field] != sha256(root / 'scripts' / script):
                    raise ValueError('Benchmark source differs from packaged source')
            if kind == 'bf16':
                expected_files = sorted([r for r in source_rows.values()
                    if r['path'] == 'model_index.json' or r['path'].startswith(
                        ('transformer/', 'text_encoder/', 'vae/', 'processor/', 'scheduler/'))],
                    key=lambda r: r['path'])
                if identity['files'] != expected_files:
                    raise ValueError('BF16 fingerprint is not from the complete verified official files')
        for field in ('gpu', 'cuda', 'packages', 'offload', 'warmup', 'generator_device',
                      'cases_sha256', 'pipeline_sha256'):
            if env['bf16'][field] != env['int8'][field]:
                raise ValueError('Unpaired runtime: ' + field)
        if records['bf16'].keys() != expected or records['int8'].keys() != expected:
            raise ValueError('Missing or extra benchmark cases/seeds')
        for case in cases:
            for seed in case.get('seeds', [42, 123]):
                a, b = (records[kind][(case['id'], seed)] for kind in ('bf16', 'int8'))
                validate_pair(a, b)
                if (a['prompt'] != case['prompt'] or a['steps'] != 8 or a['sample_sigmas'] != SIGMAS
                        or a['width'] != resolution or a['height'] != resolution
                        or a['actual_size'] != [resolution, resolution] or b['actual_size'] != a['actual_size']
                        or a['cfg'] != 1.0 or not a['kv_cache'] or a['offload'] != 'model'
                        or len(a['scheduler_sigmas']) != 9
                        or min(a.get('released_prefix_tensor_bytes', 0), b.get('released_prefix_tensor_bytes', 0)) <= 0):
                    raise ValueError('Invalid actual generation settings')
                from PIL import Image
                for kind, row in [('bf16', a), ('int8', b)]:
                    if row['image'] != f'{case["id"]}-s{seed}.png':
                        raise ValueError('Unexpected image path')
                    with Image.open(folder / kind / row['image']) as image:
                        if list(image.size) != row['actual_size'] or image.mode != row['image_mode']:
                            raise ValueError('Image metadata differs from recorded output')
                if case.get('input'):
                    input_path = root / ('evaluation/inputs/boat.png' if case['id'] == 'edit_boat'
                                         else 'evaluation/bf16/portrait-s42.png')
                    if a['input_sha256'] != sha256(input_path):
                        raise ValueError('Incorrect shared edit input')
                    if case['id'] == 'edit' and seed == 42:
                        raise ValueError('Editing noise replay')
        import csv
        with (folder / 'comparison.csv').open(encoding='utf-8', newline='') as f:
            metrics = list(csv.DictReader(f))
        if len(metrics) != len(expected) or {(r['case_id'], int(r['seed'])) for r in metrics} != expected:
            raise ValueError('Incomplete comparison metrics')
        for r in metrics:
            key = (r['case_id'], int(r['seed']))
            a, b = records['bf16'][key], records['int8'][key]
            values = {'bf16_seconds': a['seconds'], 'int8_seconds': b['seconds'],
                      'speed_ratio_bf16_over_int8': a['seconds'] / b['seconds'],
                      'bf16_peak_allocated_gib': a['peak_allocated_bytes'] / 2**30,
                      'int8_peak_allocated_gib': b['peak_allocated_bytes'] / 2**30,
                      **compare_pixels(folder / 'bf16' / a['image'], folder / 'int8' / b['image'])}
            for field, value in values.items():
                if (r[field] != '' if value is None else abs(float(r[field]) - value) > 1e-8):
                    raise ValueError('Derived comparison mismatch: ' + field)
        summary = json.loads((folder / 'summary.json').read_text())
        derived = {'pairs': len(expected), 'cases': len(cases), 'width': resolution, 'height': resolution,
                   'steps': 8, 'warmup': True}
        for kind in records:
            derived.update({kind + '_mean_seconds': statistics.mean(r['seconds'] for r in records[kind].values()),
                kind + '_max_allocated_gib': max(r['peak_allocated_bytes'] for r in records[kind].values()) / 2**30,
                kind + '_weight_bytes': sum(r['size'] for r in env[kind]['model_identity']['files']
                                           if r['path'].endswith('.safetensors'))})
        if any(k not in summary or abs(summary[k] - v) > 1e-8 for k, v in derived.items()):
            raise ValueError('Summary differs from raw evidence')


def original_url(platform, path):
    if platform == 'modelscope':
        return f'https://modelscope.cn/api/v1/models/iximbox/{NAME}/repo?Revision=master&FilePath=' + quote(path, safe='')
    return f'https://huggingface.co/ixim/{NAME}/resolve/main/' + quote(path, safe='/')


def card(root, platform):
    from scripts.report import load_records
    zh = platform == 'modelscope'
    def tr(en, cn):
        return cn if zh else en
    lines = ['---', 'license: other', 'license_name: qwen-research',
             f'license_link: https://huggingface.co/{REPO}/blob/{REVISION}/LICENSE',
             'library_name: diffusers', 'pipeline_tag: text-to-image', f'base_model: {REPO}',
             'base_model_relation: quantized', 'tags:', '- image-generation', '- image-editing',
             '- bitsandbytes', '- int8', '- qwen-image-2.1-turbo', '---', '', f'# {NAME}', '',
             '**Built with Qwen**', '',
             tr('Independent community INT8 conversion of Qwen-Image-2.1-Turbo. Both the diffusion transformer and text encoder use bitsandbytes LLM.int8 (threshold 6.0); sensitive projections, the vision tower and VAE remain in floating point. This is a complete Diffusers pipeline.',
                'Qwen-Image-2.1-Turbo 的独立社区 INT8 量化版本。扩散 Transformer 与文本编码器均使用 bitsandbytes LLM.int8（阈值 6.0）；敏感投影层、视觉编码器及 VAE 保留浮点精度。提供完整可重载的 Diffusers 管线。'), '',
             tr('The official eight-step `sample_sigmas` grid is preserved in `model_index.json`; CFG=1 and prefix KV caching are used. Changing `num_inference_steps` alone does not override the saved grid. This release requires the fixed Diffusers version below with PR #14950.',
                '原样保留 `model_index.json` 内官方 8 步 `sample_sigmas`，使用 CFG=1 和前缀 KV 缓存。单独修改 `num_inference_steps` 不会覆盖内置采样序列；必须使用下方支持 PR #14950 的固定 Diffusers 版本。'), '',
             tr('## Companion application: IMGEN', '## 配套应用：IMGEN'), '',
             tr('Application source: [iXimNet/IMGEN](https://github.com/iXimNet/IMGEN). IMGEN is a local visual studio for Qwen-Image-2.1 with a browser interface for text-to-image generation, editing with up to 10 reference images, prompt presets, and searchable history with saved parameters.',
                '配套应用代码仓库：[iXimNet/IMGEN](https://github.com/iXimNet/IMGEN)。IMGEN 是面向 Qwen-Image-2.1 的本地生图／改图工作室，通过浏览器界面提供文生图、最多 10 张参考图改图、预置提示词，以及可搜索并恢复参数的历史记录。'), '',
             tr('Choose Hugging Face or ModelScope and select **Image21-Turbo-INT8** in IMGEN to use this model with its official eight-step sampling schedule and CFG=1. This INT8 model requires NVIDIA CUDA. See the [IMGEN README](https://github.com/iXimNet/IMGEN#install) for installation and usage; the application code is MIT-licensed, while these model weights remain under the Qwen Research License.',
                '在 IMGEN 中选择 Hugging Face 或 ModelScope，并选择 **Image21-Turbo-INT8**，即可使用本模型及其官方 8 步采样序列和 CFG=1；本 INT8 模型需要 NVIDIA CUDA。安装与使用方法见 [IMGEN README](https://github.com/iXimNet/IMGEN#install)。应用代码采用 MIT 许可证，模型权重仍遵循 Qwen Research License。'), '',
             tr('## Usage', '## 使用方法'), '',
             '```bash', 'pip install -r requirements.txt', '```', '', '```python', 'import torch',
             'from scripts.runtime import load_int8_pipeline', '',
             f'pipe = load_int8_pipeline("{"iximbox/" if zh else "ixim/"}{NAME}"' + (', local_files_only=True)' if zh else ')'),
             'image = pipe(', '    prompt="A ginger cat sitting by a sunlit window, natural photography.",',
             '    width=1024, height=1024, true_cfg_scale=1.0, use_kv_cache=True,',
             '    generator=torch.Generator("cpu").manual_seed(42),', ').images[0]', 'image.save("cat.png")', '```', '',
             tr('Clone/download this repository first to use its `scripts.runtime` loader. On ModelScope, download with `snapshot_download` and pass its local path. The supplied loader loads quantized components sequentially and moves INT8 auxiliary tensors during CPU offload.',
                '先下载本仓库，再导入配套 `scripts.runtime`。ModelScope 模型应先使用 `snapshot_download` 下载，将返回的本地路径传给加载函数。加载器逐个加载量化组件，并在 CPU 卸载时同步移动 INT8 辅助张量。'), '',
             tr('The loader also releases completed denoising prefix KV tensors before VAE decode. The eight-step denoising computation and untiled VAE are preserved; this reduces the native-resolution decode peak.',
                '加载器还会在 VAE 解码前释放去噪阶段已用完的前缀 KV 张量，保持原有 8 步去噪和不分块 VAE，降低原生分辨率解码峰值。'), '',
             tr('For editing, pass `image=Image.open("input.png").convert("RGBA")` with a new independent editing seed. Output PNG retains RGBA. VAE tiling is disabled in this evaluation.',
                '改图时传入 `image=Image.open("input.png").convert("RGBA")`，并使用独立编辑种子。输出 PNG 保留 RGBA；本次评测关闭 VAE 分块。'), '',
             tr('Shared evaluation inputs: ', '评测共用输入：') +
             f'[BF16 portrait]({original_url(platform, "evaluation/bf16/portrait-s42.png")}) · '
             f'[Official boat sketch]({original_url(platform, "evaluation/inputs/boat.png")})', '',
             tr('## Informal BF16 / saved-INT8 evaluation', '## 非正式 BF16／保存后重载 INT8 对照'), '',
             tr('This small community evaluation is for reference and does not represent an official evaluation. Every result is included. One portrait warmup per precision/suite is excluded; editing has no separate warmup. Loading and disk writing are excluded from call latency. Suite means include generation and editing. CUDA allocated peaks exclude other processes and are not minimum VRAM requirements. Pixel drift is not a semantic quality score.',
                '**本次社区小样本评测仅供参考，不代表任何官方评价。** 展示全部结果，不挑选种子。每种精度、每套用例做一次肖像预热，不计入测量；改图没有单独预热。调用耗时不含模型加载与写盘；均值同时包含生成和改图。CUDA 已分配峰值不含其他进程，不等于最低显存要求；像素差异不等于语义质量评分。'), '']
    for suite, title in [('', tr('1024px core suite', '1024px 主评测')), ('native', tr('2048px native-resolution supplement', '2048px 原生分辨率补充'))]:
        folder = root / 'evaluation' / suite
        summary = json.loads((folder / 'summary.json').read_text())
        env = json.loads((folder / 'bf16/environment.json').read_text())
        records = {k: load_records(folder / k) for k in ('bf16', 'int8')}
        lines += [f'### {title}', '',
                  tr(f'{summary["cases"]} cases / {summary["pairs"]} pairs; {summary["width"]} × {summary["height"]}; 8 steps; CFG 1; KV cache; model CPU offload; {env["gpu"]}.',
                     f'{summary["cases"]} 类用例／{summary["pairs"]} 对输出；{summary["width"]} × {summary["height"]}；8 步；CFG 1；KV 缓存；模型 CPU 卸载；{env["gpu"]}。'), '',
                  tr('| Metric | BF16 | INT8 |', '| 指标 | BF16 | INT8 |'), '|---|---:|---:|',
                  tr('| Weights (GB) | ', '| 权重体积（GB） | ') + f'{summary["bf16_weight_bytes"]/1e9:.3f} | {summary["int8_weight_bytes"]/1e9:.3f} |',
                  tr('| Mean call (s) | ', '| 平均调用耗时（秒） | ') + f'{summary["bf16_mean_seconds"]:.3f} | {summary["int8_mean_seconds"]:.3f} |',
                  tr('| Peak CUDA allocated (GiB) | ', '| CUDA 已分配显存峰值（GiB） | ') + f'{summary["bf16_max_allocated_gib"]:.3f} | {summary["int8_max_allocated_gib"]:.3f} |', '']
        if suite == '':
            lines += [tr('Runtime package versions:', '运行环境软件版本：'), '',
                      '| Package | Version |', '|---|---|']
            lines += [f'| {k} | {v} |' for k, v in env['packages'].items()]
            lines += [f'| Python / CUDA | {env["python"]} / {env["cuda"]} |', '',
                      tr('Every preview below links to its unchanged full-resolution original PNG.',
                         '下方所有预览均可点击打开未经修改的原尺寸 PNG。'), '']
        for key, a in records['bf16'].items():
            b = records['int8'][key]
            urls = [original_url(platform, '/'.join(p for p in ('evaluation', suite, k, r['image']) if p))
                    for k, r in [('bf16', a), ('int8', b)]]
            lines += [f'#### {key[0]} · seed {key[1]}', '', '```text', a['prompt'], '```', '',
                      '| BF16 | INT8 |', '|---|---|',
                      f'| [![BF16]({urls[0]})]({urls[0]}) | [![INT8]({urls[1]})]({urls[1]}) |', '',
                      f'{a["seconds"]:.3f}s / {b["seconds"]:.3f}s; allocated peak '
                      f'{a["peak_allocated_bytes"]/2**30:.3f} / {b["peak_allocated_bytes"]/2**30:.3f} GiB.', '']
    observations = root / 'evaluation/qualitative.md'
    if not observations.exists():
        raise ValueError('Inspect generated images and write qualitative.md before staging')
    observation_text = observations.read_text(encoding='utf-8').replace(
        '](cache-release-validation.json)', '](evaluation/cache-release-validation.json)')
    lines += [tr('## Visual observations and limits', '## 样图观察与局限'), '', observation_text, '',
              tr('## Reproducibility and license', '## 复现与许可证'), '',
              tr('Official base: ', '官方源模型：') + f'[{REPO}](https://huggingface.co/{REPO}/tree/{REVISION})，`{REVISION}`.', '',
              f'Diffusers: `{DIFFUSERS_COMMIT}`. ' +
              tr('Exact package versions, actual scheduler sigmas, source and image hashes, warmup settings and independent-process exits are included in `evaluation/`.',
                 '完整软件版本、实际调度器 sigmas、源文件与输出哈希、预热设置及独立进程退出记录均位于 `evaluation/`。'), '',
              '[Reproduction instructions](REPRODUCE.md) · [Raw comparison](evaluation/comparison.csv) · '
              '[Native supplement](evaluation/native/comparison.csv) · [Conversion](conversion.json) · [Manifest](MANIFEST.json)', '',
              tr('Non-commercial research and evaluation only under the accompanying [Qwen Research License](LICENSE). See [Notice](Notice) and [modifications](CHANGES.md). No fine-tuning or trained adapters are included.',
                 '遵循随附 [Qwen Research License](LICENSE)，仅限非商业研究与评测。归属见 [Notice](Notice)，修改说明见 [CHANGES.md](CHANGES.md)。没有微调或附加训练适配器。'), '']
    text = '\n'.join(lines)
    if zh:
        text = text.replace('pipe = load_int8_pipeline("iximbox/' + NAME + '", local_files_only=True)',
                            'from modelscope import snapshot_download\nmodel = snapshot_download("iximbox/' + NAME + '")\npipe = load_int8_pipeline(model, local_files_only=True)')
    return text


def stage(platform):
    from scripts.integrity import sha256
    output = ROOT / 'release' / ('turbo-' + platform)
    if output.exists():
        raise FileExistsError('Refusing to replace staged release: ' + str(output))
    shutil.copytree(QUANT, output, ignore=shutil.ignore_patterns('.cache', '__pycache__'))
    shutil.copytree(ARTIFACTS / 'evaluation', output / 'evaluation')
    for folder, names in [('scripts', ['__init__.py', 'turbo_release.py', 'quantize.py', 'runtime.py', 'benchmark.py',
            'infer.py', 'provenance.py', 'integrity.py', 'release.py', 'report.py', 'upload.py',
            'editing_protocol.py', 'editing_validation.py']),
            ('benchmarks', ['cases.json', 'young_woman.json', 'turbo.json', 'turbo-native.json']),
            ('tests', ['__init__.py', 'test_turbo.py', 'test_integrity.py', 'test_notices.py', 'test_provenance.py', 'test_upload_auth.py'])]:
        (output / folder).mkdir()
        for name in names:
            shutil.copyfile(ROOT / folder / name, output / folder / name)
    shutil.copyfile(ARTIFACTS / 'download-verification.json', output / 'upstream-verification.json')
    shutil.copytree(ARTIFACTS / 'upstream', output / 'upstream')
    requirements = (ROOT / 'requirements.txt').read_text(encoding='utf-8')
    requirements = '\n'.join('diffusers @ https://codeload.github.com/huggingface/diffusers/zip/' + DIFFUSERS_COMMIT
                             if line.startswith('diffusers @') else line for line in requirements.splitlines()) + '\n'
    (output / 'requirements.txt').write_text(requirements, encoding='utf-8', newline='\n')
    (output / 'requirements-turbo.txt').write_text(requirements, encoding='utf-8', newline='\n')
    (output / 'docs').mkdir()
    shutil.copyfile(ROOT / 'docs/turbo-release.md', output / 'docs/turbo-release.md')
    shutil.copyfile(ROOT / 'docs/turbo-release.md', output / 'REPRODUCE.md')
    shutil.copyfile(ARTIFACTS / 'runtime-source.json', output / 'runtime-source.json')
    (output / 'README.md').write_text(card(output, platform), encoding='utf-8', newline='\n')
    if platform == 'modelscope':
        write_json(output / 'configuration.json', {'framework': 'pytorch', 'task': 'text-to-image-synthesis'})
    # Normalize authored text before generating the manifest; inference files already
    # use LF and must remain byte-identical to their measured model fingerprints.
    for path in output.rglob('*'):
        if path.is_file() and (path.suffix in ('.py', '.json', '.jsonl', '.md', '.txt', '.html', '.csv')
                              or path.name == 'Notice'):
            data = path.read_bytes()
            path.write_bytes(data.replace(b'\r\n', b'\n'))
    rows = [{'path': p.relative_to(output).as_posix(), 'size': p.stat().st_size, 'sha256': sha256(p)}
            for p in sorted(output.rglob('*')) if p.is_file()]
    write_json(output / 'MANIFEST.json', rows)
    validate_release(output)
    print('Staged and validated ' + str(output), flush=True)


def validate_release(root):
    from scripts.integrity import verify_files
    from scripts.provenance import model_identity
    root = Path(root)
    rows = json.loads((root / 'MANIFEST.json').read_text())
    expected = {r['path'] for r in rows} | {'MANIFEST.json'}
    actual = {p.relative_to(root).as_posix() for p in root.rglob('*') if p.is_file()}
    if len(rows) != len(expected) - 1 or actual != expected:
        raise ValueError('Release inventory differs from manifest')
    for name in actual:
        if any(part.startswith('.') or part == '__pycache__' for part in Path(name).parts):
            raise ValueError('Unpublishable hidden/cache file: ' + name)
    verify_files(root, rows)
    identity = model_identity(root, verified_rows=rows)
    conversion = json.loads((root / 'conversion.json').read_text())
    if (identity['kind'] != 'int8' or conversion['base_model'] != REPO
            or conversion['base_revision'] != REVISION or conversion['diffusers_commit'] != DIFFUSERS_COMMIT
            or json.loads((root / 'model_index.json').read_text())['sample_sigmas'] != SIGMAS):
        raise ValueError('Invalid Turbo release provenance/sampling')
    known = {r['path']: r for r in rows}
    if any(known.get(r['path']) != r for r in conversion['weight_files']):
        raise ValueError('Conversion weight hashes differ')
    from safetensors import safe_open
    from scripts.quantize import TRANSFORMER_SKIP, TEXT_SKIP
    upstream = json.loads((root / 'upstream-verification.json').read_text())
    source_known = {r['path']: r for r in upstream['files']}
    for component, skip in [('transformer', TRANSFORMER_SKIP), ('text_encoder', TEXT_SKIP)]:
        report = json.loads((root / (component + '-quantization.json')).read_text())
        if report['skip_modules'] != skip or not report['linear8bit_modules'] or report['int8_parameters'] <= 0:
            raise ValueError('Invalid quantization coverage')
        if component == 'text_encoder' and any(n.startswith(('visual.', 'model.visual.')) for n in report['linear8bit_modules']):
            raise ValueError('Vision tower unexpectedly quantized')
        if not report['source_files'] or any(source_known.get(r['path']) != r for r in report['source_files']):
            raise ValueError('Quantized component did not use verified official source')
        for path in (root / component).glob('*.safetensors'):
            with safe_open(str(path), framework='np') as f:
                metadata = f.metadata() or {}
                if metadata.get('base_revision') != REVISION or 'Qwen-Image-2.1-Turbo' not in metadata.get('modification_notice', ''):
                    raise ValueError('Missing Turbo modification notice')
                for name in f.keys():
                    protected = (name.startswith(('visual.', 'model.visual.')) if component == 'text_encoder'
                                 else name.startswith(tuple(s + '.' for s in TRANSFORMER_SKIP)))
                    if protected and f.get_slice(name).get_dtype() not in ('BF16', 'F16', 'F32', 'F64'):
                        raise ValueError('Protected layer is not floating point: ' + name)
    for row in upstream['files']:
        if row['path'] == 'model_index.json' or row['path'].startswith(('processor/', 'scheduler/', 'vae/')):
            if known.get(row['path']) != row:
                raise ValueError('Official unmodified support file changed: ' + row['path'])
    runtime = json.loads((root / 'runtime-source.json').read_text())
    if runtime['diffusers_commit'] != DIFFUSERS_COMMIT:
        raise ValueError('Wrong pinned runtime')
    for suite in ('', 'native'):
        for kind in ('bf16', 'int8'):
            env = json.loads((root / 'evaluation' / suite / kind / 'environment.json').read_text())
            if env['pipeline_sha256'] != runtime['pipeline_sha256']:
                raise ValueError('Evaluated pipeline differs from official runtime archive')
    validate_evidence(root, identity)
    return rows


def upload(platform, dry_run=False):
    folder = ROOT / 'release' / ('turbo-' + platform)
    rows = validate_release(folder)
    print(f'Validated {len(rows)} files for {platform}', flush=True)
    if dry_run:
        return
    names = [r['path'] for r in rows] + ['MANIFEST.json']
    if platform == 'huggingface':
        from huggingface_hub import HfApi, get_token
        token = get_token()
        if not token:
            raise RuntimeError('HF login required in project HF_HOME')
        api = HfApi(token=token)
        if api.whoami()['name'] != 'ixim':
            raise ValueError('Wrong Hugging Face account')
        target = 'ixim/' + NAME
        api.create_repo(target, repo_type='model', private=False, exist_ok=True)
        result = api.upload_folder(repo_id=target, repo_type='model', folder_path=str(folder),
                                  allow_patterns=names, commit_message='Release verified Turbo INT8 with eight-step paired evaluation')
        print(result, flush=True)
    else:
        from modelscope.hub.api import HubApi, ModelScopeConfig
        from modelscope.hub.errors import NotExistError
        from requests.exceptions import HTTPError
        from scripts.upload import modelscope_access
        api = HubApi()
        token = modelscope_access(ModelScopeConfig, 'iximbox')
        if token:
            api.login(token)
            user = ModelScopeConfig.get_user_info()
            if not user or user[0] != 'iximbox':
                raise ValueError('Wrong ModelScope account')
        target = 'iximbox/' + NAME
        try:
            api.get_model(target)
        except (HTTPError, NotExistError) as exc:
            if isinstance(exc, HTTPError) and (exc.response is None or exc.response.status_code != 404):
                raise
            api.create_model(target, visibility=5, license='other', original_model_id=REPO, token=token)
        print(api.upload_folder(repo_id=target, folder_path=str(folder), token=token,
                                allow_patterns=names, max_workers=4,
                                commit_message='Release verified Turbo INT8 with eight-step paired evaluation'), flush=True)


def verify_published(platform):
    from scripts.integrity import sha256
    folder = ROOT / 'release' / ('turbo-' + platform)
    rows = json.loads((folder / 'MANIFEST.json').read_text())
    manifest = folder / 'MANIFEST.json'
    rows += [{'path': 'MANIFEST.json', 'size': manifest.stat().st_size, 'sha256': sha256(manifest)}]
    expected = {r['path'] for r in rows}
    if platform == 'huggingface':
        from huggingface_hub import HfApi
        info = HfApi(token=False).model_info('ixim/' + NAME, files_metadata=True)
        if info.private:
            raise ValueError('Release is not public')
        remote = {e.rfilename: e for e in info.siblings}
        commit = info.sha
        for r in rows:
            e = remote[r['path']]
            if e.size != r['size']:
                raise ValueError('Remote size mismatch: ' + r['path'])
            if e.lfs:
                if e.lfs.sha256 != r['sha256']:
                    raise ValueError('Remote weight hash mismatch: ' + r['path'])
            else:
                data = (folder / r['path']).read_bytes()
                if hashlib.sha1(f'blob {len(data)}\0'.encode() + data).hexdigest() != e.blob_id:
                    raise ValueError('Remote Git hash mismatch: ' + r['path'])
        target = 'ixim/' + NAME
    else:
        from modelscope.hub.api import HubApi
        api = HubApi()
        files = api.get_model_files('iximbox/' + NAME, recursive=True)
        remote = {e['Path']: e for e in files if e['Type'] == 'blob'}
        # ModelScope reports each file's last-change revision. The manifest is
        # updated by every release change, including docs-only followups.
        commit = remote['MANIFEST.json']['Revision']
        files = api.get_model_files('iximbox/' + NAME, revision=commit, recursive=True)
        remote = {e['Path']: e for e in files if e['Type'] == 'blob'}
        for r in rows:
            e = remote[r['path']]
            if e['Size'] != r['size'] or e['Sha256'] != r['sha256']:
                raise ValueError('Remote hash/size mismatch: ' + r['path'])
        import requests
        response = requests.get(f'https://modelscope.cn/api/v1/models/iximbox/{NAME}', timeout=60)
        response.raise_for_status()
        if not response.json().get('Success'):
            raise ValueError('Public ModelScope metadata is unavailable')
        readme = requests.get(f'https://modelscope.cn/api/v1/models/iximbox/{NAME}/repo',
                              params={'Revision': commit, 'FilePath': 'README.md'}, timeout=60)
        readme.raise_for_status()
        if hashlib.sha256(readme.content).hexdigest() != sha256(folder / 'README.md'):
            raise ValueError('ModelScope card is not anonymously accessible with the expected bytes')
        target = 'iximbox/' + NAME
    if expected - set(remote) or set(remote) - expected - {'.gitattributes'}:
        raise ValueError('Published inventory differs')
    report = {'repo_id': target, 'commit': commit, 'files_verified': len(rows), 'files': rows}
    write_json(ARTIFACTS / (platform + '-publication.json'), report)
    print(json.dumps({k: v for k, v in report.items() if k != 'files'}), flush=True)


def main():
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument('action', choices=['download', 'quantize', 'evaluate', 'local', 'retest-int8', 'stage', 'upload', 'verify'])
    parser.add_argument('--source', choices=['huggingface', 'modelscope'], default='modelscope')
    parser.add_argument('--platform', choices=['huggingface', 'modelscope'])
    parser.add_argument('--dry-run', action='store_true')
    parser.add_argument('--download-pid', type=int, help='Wait for an already-running source download before local work')
    args = parser.parse_args()
    configure_environment()
    os.chdir(ROOT)
    ARTIFACTS.mkdir(parents=True, exist_ok=True)
    if args.action == 'download':
        download(args.source)
    elif args.action == 'quantize':
        quantize()
    elif args.action == 'evaluate':
        evaluate()
    elif args.action == 'retest-int8':
        quantize()
        evaluate(int8_only=True)
    elif args.action == 'local':
        if args.download_pid:
            import psutil
            while not (ARTIFACTS / 'download-verification.json').exists():
                if not psutil.pid_exists(args.download_pid):
                    raise RuntimeError('Source download stopped without verified files')
                time.sleep(5)
        quantize()
        evaluate()
    else:
        if not args.platform:
            parser.error('--platform is required for this action')
        if args.action == 'stage':
            stage(args.platform)
        elif args.action == 'upload':
            upload(args.platform, args.dry_run)
        else:
            verify_published(args.platform)


if __name__ == '__main__':
    main()