alex4cip Claude commited on
Commit
a0f4ab8
Β·
1 Parent(s): fc890b6

feat: Add comprehensive hardware environment detection system

Browse files

Major Changes:
- Implement detect_hardware_environment() function
- Support 6 environment types with auto-detection
- Dynamic UI based on detected hardware

Environment Types Supported:
1. HF Spaces - ZeroGPU (NVIDIA H200)
2. HF Spaces - CPU Upgrade (8 vCPU, 32GB RAM)
3. HF Spaces - CPU Basic (2-4 vCPU, 16GB RAM)
4. Local - GPU (CUDA/MPS detection)
5. Local - Apple Silicon (MPS backend)
6. Local - CPU (fallback)

Detection Logic:
- SPACE_ID environment variable β†’ HF Spaces
- Import spaces success β†’ ZeroGPU
- CPU count >= 8 β†’ CPU Upgrade
- CPU count < 8 β†’ CPU Basic
- torch.cuda.is_available() β†’ Local CUDA GPU
- torch.backends.mps.is_available() β†’ Apple Silicon GPU
- Fallback β†’ Local CPU

Environment Info Displayed:
- Platform (HF Spaces / Local)
- Hardware type
- GPU availability and name
- CPU core count
- Operating system
- Detailed description

UI Improvements:
- Dynamic header with environment-specific info
- Hardware-specific features and recommendations
- Response time estimates per environment
- Cost information (HF Spaces tiers)
- Model recommendations (lightweight for CPU)

Benefits:
- Clear visibility of current environment
- Appropriate expectations for performance
- Better user experience with accurate info
- Easier debugging and troubleshooting
- Support for all deployment scenarios

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +200 -70
app.py CHANGED
@@ -1,23 +1,106 @@
1
  """
2
- Flexible version: Works on both ZeroGPU and CPU Upgrade hardware
3
- Automatically detects hardware and adjusts accordingly
4
  """
5
 
6
- # Try to import spaces for ZeroGPU support
7
- try:
8
- import spaces
9
- ZEROGPU_AVAILABLE = True
10
- print("βœ… ZeroGPU support enabled")
11
- except ImportError:
12
- ZEROGPU_AVAILABLE = False
13
- print("ℹ️ ZeroGPU not available, using standard mode")
14
-
15
  import os
 
16
  import gradio as gr
17
  from transformers import AutoModelForCausalLM, AutoTokenizer
18
  from huggingface_hub import snapshot_download
19
  import torch
20
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
  # Load environment variables from .env file
22
  try:
23
  from dotenv import load_dotenv
@@ -338,37 +421,62 @@ def chat_wrapper(message, history):
338
  return response_history
339
 
340
 
341
- # Determine hardware info for UI
342
- hardware_info = "NVIDIA H200 (ZeroGPU)" if ZEROGPU_AVAILABLE else "CPU Upgrade (32GB RAM)"
343
- print(f"βœ… App initialized - Hardware: {hardware_info}")
344
 
345
  # Create Gradio interface
346
  with gr.Blocks(title="πŸ€– Multi-Model Chatbot") as demo:
347
- # Dynamic header based on hardware
348
- if ZEROGPU_AVAILABLE:
349
- header = """
350
- # πŸ€– 닀쀑 λͺ¨λΈ 챗봇 (ZeroGPU)
351
-
352
- **ν•˜λ“œμ›¨μ–΄**: NVIDIA H200 (ZeroGPU - μžλ™ ν• λ‹Ή)
353
-
354
- **νŠΉμ§•**:
355
- - ⚑ GPU κ°€μ†μœΌλ‘œ λΉ λ₯Έ 응닡 (3-5초)
356
- - 🎯 {TOTAL_MODEL_COUNT}κ°€μ§€ ν•œκΈ€ μ΅œμ ν™” λͺ¨λΈ 선택 κ°€λŠ₯ ({PUBLIC_MODEL_COUNT} Public + {GATED_MODEL_COUNT} Gated)
357
- - πŸ”„ λͺ¨οΏ½οΏ½ μ „ν™˜ μ‹œ μžλ™ μž¬λ‘œλ”©
358
- - πŸ’° PRO ꡬ독 μ‹œ ν•˜λ£¨ 25λΆ„ 무료 μ‚¬μš©
359
- """
360
- else:
361
- header = """
362
- # πŸ€– 닀쀑 λͺ¨λΈ 챗봇 (CPU Upgrade)
363
-
364
- **ν•˜λ“œμ›¨μ–΄**: CPU Upgrade (8 vCPU / 32 GB RAM)
365
-
366
- **νŠΉμ§•**:
367
- - 🎯 {TOTAL_MODEL_COUNT}κ°€μ§€ ν•œκΈ€ μ΅œμ ν™” λͺ¨λΈ 선택 κ°€λŠ₯ ({PUBLIC_MODEL_COUNT} Public + {GATED_MODEL_COUNT} Gated)
368
- - πŸ”„ λͺ¨λΈ μ „ν™˜ μ‹œ μžλ™ μž¬λ‘œλ”©
369
- - ⏳ CPU ν™˜κ²½μ΄λ―€λ‘œ 응닡이 λ‹€μ†Œ λŠλ¦½λ‹ˆλ‹€ (30초~1λΆ„)
370
- - πŸ’° μ‹œκ°„λ‹Ή $0.03 (μ›” μ•½ $22)
371
- """
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
372
 
373
  gr.Markdown(header)
374
 
@@ -438,37 +546,59 @@ with gr.Blocks(title="πŸ€– Multi-Model Chatbot") as demo:
438
  msg.submit(submit, [msg, chatbot], [chatbot, msg])
439
  clear.click(lambda: [], outputs=chatbot)
440
 
441
- # Dynamic footer based on hardware
442
- if ZEROGPU_AVAILABLE:
443
- footer = f"""
444
- ---
445
- **참고사항 (ZeroGPU λͺ¨λ“œ)**:
446
- - πŸ€– {TOTAL_MODEL_COUNT}κ°€μ§€ λͺ¨λΈ 쀑 선택 κ°€λŠ₯ (λ“œλ‘­λ‹€μš΄μ—μ„œ 선택)
447
- - ⚑ ZeroGPUλŠ” μš”μ²­ μ‹œ μžλ™μœΌλ‘œ GPUλ₯Ό ν• λ‹Ήν•©λ‹ˆλ‹€
448
- - πŸ’° PRO κ΅¬λ…μžλŠ” ν•˜λ£¨ 25λΆ„ 무료 μ‚¬μš©
449
- - πŸ”„ λͺ¨λΈ λ³€κ²½ μ‹œ λŒ€ν™” 내역이 μ΄ˆκΈ°ν™”λ©λ‹ˆλ‹€
450
- - ⏱️ 첫 응닡은 λͺ¨λΈ λ‘œλ”© μ‹œκ°„ 포함 (~10-15초)
451
-
452
- **ν…ŒμŠ€νŠΈ μ˜ˆμ‹œ**:
453
- - "μ•ˆλ…•ν•˜μ„Έμš”"
454
- - "인곡지λŠ₯에 λŒ€ν•΄ μ„€λͺ…ν•΄μ£Όμ„Έμš”"
455
- - "ν•œκ΅­μ˜ μˆ˜λ„λŠ” μ–΄λ””μΈκ°€μš”?"
456
- """
457
- else:
458
- footer = f"""
459
- ---
460
- **참고사항 (CPU Upgrade λͺ¨λ“œ)**:
461
- - πŸ€– {TOTAL_MODEL_COUNT}κ°€μ§€ λͺ¨λΈ 쀑 선택 κ°€λŠ₯ (λ“œλ‘­λ‹€μš΄μ—μ„œ 선택)
462
- - πŸ”„ λͺ¨λΈ λ³€κ²½ μ‹œ λŒ€ν™” 내역이 μ΄ˆκΈ°ν™”λ©λ‹ˆλ‹€
463
- - ⏳ CPU ν™˜κ²½μ΄λ―€λ‘œ 응닡이 λŠλ¦½λ‹ˆλ‹€ (30초~1λΆ„)
464
- - ⏱️ 첫 응닡은 λͺ¨λΈ λ‘œλ”© μ‹œκ°„ 포함 (~1-2λΆ„)
465
- - πŸ’° 24μ‹œκ°„ λ¬΄μ œν•œ μ‚¬μš© (μ‹œκ°„λ‹Ή $0.03)
466
-
467
- **ν…ŒμŠ€νŠΈ μ˜ˆμ‹œ**:
468
- - "μ•ˆλ…•ν•˜μ„Έμš”"
469
- - "인곡지λŠ₯에 λŒ€ν•΄ μ„€λͺ…ν•΄μ£Όμ„Έμš”"
470
- - "ν•œκ΅­μ˜ μˆ˜λ„λŠ” μ–΄λ””μΈκ°€μš”?"
471
- """
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
472
 
473
  gr.Markdown(footer)
474
 
 
1
  """
2
+ Multi-environment chatbot: Detects and adapts to different hardware environments
3
+ Supports: Local (Mac/Linux/Windows), HF Spaces (CPU Basic/Upgrade, ZeroGPU)
4
  """
5
 
 
 
 
 
 
 
 
 
 
6
  import os
7
+ import platform
8
  import gradio as gr
9
  from transformers import AutoModelForCausalLM, AutoTokenizer
10
  from huggingface_hub import snapshot_download
11
  import torch
12
 
13
+ # ============================================================================
14
+ # Hardware Environment Detection
15
+ # ============================================================================
16
+
17
+ def detect_hardware_environment():
18
+ """
19
+ Comprehensive hardware environment detection
20
+
21
+ Returns:
22
+ dict: {
23
+ 'platform': 'hf_spaces' | 'local',
24
+ 'hardware': 'zerogpu' | 'cpu_upgrade' | 'cpu_basic' | 'local_gpu' | 'local_cpu',
25
+ 'gpu_available': bool,
26
+ 'gpu_name': str or None,
27
+ 'cpu_count': int,
28
+ 'os': 'Darwin' | 'Linux' | 'Windows',
29
+ 'description': str
30
+ }
31
+ """
32
+ env_info = {
33
+ 'platform': 'local',
34
+ 'hardware': 'local_cpu',
35
+ 'gpu_available': False,
36
+ 'gpu_name': None,
37
+ 'cpu_count': os.cpu_count() or 1,
38
+ 'os': platform.system(),
39
+ 'description': ''
40
+ }
41
+
42
+ # Check if running on HF Spaces
43
+ is_hf_spaces = os.environ.get('SPACE_ID') is not None
44
+
45
+ if is_hf_spaces:
46
+ env_info['platform'] = 'hf_spaces'
47
+ space_id = os.environ.get('SPACE_ID', 'unknown')
48
+
49
+ # Check for ZeroGPU
50
+ try:
51
+ import spaces
52
+ env_info['hardware'] = 'zerogpu'
53
+ env_info['gpu_available'] = True
54
+ env_info['gpu_name'] = 'NVIDIA H200 (ZeroGPU)'
55
+ env_info['description'] = f"πŸš€ HF Spaces - ZeroGPU ({space_id})"
56
+ except ImportError:
57
+ # Check CPU tier by memory/CPU count
58
+ cpu_count = env_info['cpu_count']
59
+ if cpu_count >= 8:
60
+ env_info['hardware'] = 'cpu_upgrade'
61
+ env_info['description'] = f"βš™οΈ HF Spaces - CPU Upgrade ({cpu_count} vCPU, 32GB RAM)"
62
+ else:
63
+ env_info['hardware'] = 'cpu_basic'
64
+ env_info['description'] = f"πŸ’» HF Spaces - CPU Basic ({cpu_count} vCPU, 16GB RAM)"
65
+ else:
66
+ # Local environment detection
67
+ if torch.cuda.is_available():
68
+ env_info['hardware'] = 'local_gpu'
69
+ env_info['gpu_available'] = True
70
+ try:
71
+ env_info['gpu_name'] = torch.cuda.get_device_name(0)
72
+ except:
73
+ env_info['gpu_name'] = 'CUDA GPU'
74
+ env_info['description'] = f"πŸ–₯️ Local - GPU ({env_info['gpu_name']})"
75
+ elif torch.backends.mps.is_available():
76
+ env_info['hardware'] = 'local_gpu'
77
+ env_info['gpu_available'] = True
78
+ env_info['gpu_name'] = 'Apple Silicon GPU (MPS)'
79
+ env_info['description'] = f"🍎 Local - Apple Silicon GPU"
80
+ else:
81
+ env_info['hardware'] = 'local_cpu'
82
+ env_info['description'] = f"πŸ’» Local - CPU ({env_info['os']}, {env_info['cpu_count']} cores)"
83
+
84
+ return env_info
85
+
86
+ # Detect hardware environment
87
+ HW_ENV = detect_hardware_environment()
88
+ ZEROGPU_AVAILABLE = HW_ENV['hardware'] == 'zerogpu'
89
+
90
+ # Print environment info
91
+ print("=" * 60)
92
+ print("Hardware Environment Detection")
93
+ print("=" * 60)
94
+ print(f"Platform: {HW_ENV['platform']}")
95
+ print(f"Hardware: {HW_ENV['hardware']}")
96
+ print(f"GPU Available: {HW_ENV['gpu_available']}")
97
+ if HW_ENV['gpu_name']:
98
+ print(f"GPU Name: {HW_ENV['gpu_name']}")
99
+ print(f"CPU Cores: {HW_ENV['cpu_count']}")
100
+ print(f"OS: {HW_ENV['os']}")
101
+ print(f"Description: {HW_ENV['description']}")
102
+ print("=" * 60)
103
+
104
  # Load environment variables from .env file
105
  try:
106
  from dotenv import load_dotenv
 
421
  return response_history
422
 
423
 
424
+ print(f"βœ… App initialized - {HW_ENV['description']}")
 
 
425
 
426
  # Create Gradio interface
427
  with gr.Blocks(title="πŸ€– Multi-Model Chatbot") as demo:
428
+ # Dynamic header based on hardware environment
429
+ header = f"""
430
+ # πŸ€– 닀쀑 λͺ¨λΈ 챗봇 {HW_ENV['description']}
431
+
432
+ **ν™˜κ²½ 정보**:
433
+ - **ν”Œλž«νΌ**: {HW_ENV['platform'].upper().replace('_', ' ')}
434
+ - **ν•˜λ“œμ›¨μ–΄**: {HW_ENV['hardware'].upper().replace('_', ' ')}
435
+ - **GPU**: {'βœ… ' + HW_ENV['gpu_name'] if HW_ENV['gpu_available'] else '❌ CPU only'}
436
+ - **CPU μ½”μ–΄**: {HW_ENV['cpu_count']}
437
+ - **운영체제**: {HW_ENV['os']}
438
+
439
+ **λͺ¨λΈ 선택**:
440
+ - 🎯 {TOTAL_MODEL_COUNT}κ°€μ§€ ν•œκΈ€ μ΅œμ ν™” λͺ¨λΈ ({PUBLIC_MODEL_COUNT} Public + {GATED_MODEL_COUNT} Gated)
441
+ - πŸ”„ λͺ¨λΈ μ „ν™˜ μ‹œ μžλ™ μž¬λ‘œλ”© (μ±„νŒ… νžˆμŠ€ν† λ¦¬ μ΄ˆκΈ°ν™”)
442
+ """
443
+
444
+ # Add hardware-specific features
445
+ if HW_ENV['hardware'] == 'zerogpu':
446
+ header += """
447
+ **ZeroGPU νŠΉμ§•**:
448
+ - ⚑ μ΄ˆκ³ μ† 응닡 (3-5초, GPU 가속)
449
+ - πŸš€ NVIDIA H200 μžλ™ ν• λ‹Ή
450
+ - πŸ’° PRO ꡬ독 μ‹œ ν•˜λ£¨ 25λΆ„ 무료
451
+ """
452
+ elif HW_ENV['hardware'] == 'cpu_upgrade':
453
+ header += """
454
+ **CPU Upgrade νŠΉμ§•**:
455
+ - ⏰ λ¬΄μ œν•œ μ‚¬μš© μ‹œκ°„
456
+ - ⏳ CPU ν™˜κ²½ (응닡 30초~1λΆ„)
457
+ - πŸ’° μ‹œκ°„λ‹Ή $0.03 (μ›” μ•½ $22)
458
+ """
459
+ elif HW_ENV['hardware'] == 'cpu_basic':
460
+ header += """
461
+ **CPU Basic νŠΉμ§•**:
462
+ - πŸ’‘ 무료 ν‹°μ–΄
463
+ - ⏳ CPU ν™˜κ²½ (응닡 1~2λΆ„)
464
+ - πŸ”’ κ²½λŸ‰ λͺ¨λΈ ꢌμž₯ (EXAONE 2.4B, Mistral 7B)
465
+ """
466
+ elif HW_ENV['hardware'] == 'local_gpu':
467
+ header += f"""
468
+ **둜컬 GPU νŠΉμ§•**:
469
+ - πŸ–₯️ 개인 GPU: {HW_ENV['gpu_name']}
470
+ - ⚑ λΉ λ₯Έ 응닡 (GPU 가속)
471
+ - πŸ”“ λ¬΄μ œν•œ μ‚¬μš©
472
+ """
473
+ else: # local_cpu
474
+ header += """
475
+ **둜컬 CPU νŠΉμ§•**:
476
+ - πŸ’» 둜컬 개발 ν™˜κ²½
477
+ - ⏳ CPU ν™˜κ²½ (느린 응닡)
478
+ - πŸ”’ κ²½λŸ‰ λͺ¨λΈ ꢌμž₯
479
+ """
480
 
481
  gr.Markdown(header)
482
 
 
546
  msg.submit(submit, [msg, chatbot], [chatbot, msg])
547
  clear.click(lambda: [], outputs=chatbot)
548
 
549
+ # Dynamic footer based on hardware environment
550
+ footer = f"""
551
+ ---
552
+ **ν˜„μž¬ ν™˜κ²½**: {HW_ENV['description']}
553
+
554
+ **참고사항**:
555
+ - πŸ€– {TOTAL_MODEL_COUNT}κ°€μ§€ λͺ¨λΈ 쀑 선택 κ°€λŠ₯
556
+ - πŸ”„ λͺ¨λΈ λ³€κ²½ μ‹œ λŒ€ν™” λ‚΄μ—­ μ΄ˆκΈ°ν™”
557
+ - ⏱️ 첫 응닡은 λͺ¨λΈ λ‘œλ”© μ‹œκ°„ 포함
558
+ """
559
+
560
+ # Add environment-specific notes
561
+ if HW_ENV['hardware'] == 'zerogpu':
562
+ footer += """
563
+ - ⚑ ZeroGPU μžλ™ GPU ν• λ‹Ή (3-5초 응닡)
564
+ - πŸ’° PRO κ΅¬λ…μž ν•˜λ£¨ 25λΆ„ 무료
565
+ - ⏱️ 첫 λ‘œλ”©: ~10-15초
566
+ """
567
+ elif HW_ENV['hardware'] == 'cpu_upgrade':
568
+ footer += """
569
+ - ⏰ 24μ‹œκ°„ λ¬΄μ œν•œ μ‚¬μš©
570
+ - ⏳ CPU ν™˜κ²½ (30초~1λΆ„ 응닡)
571
+ - πŸ’° μ‹œκ°„λ‹Ή $0.03
572
+ - ⏱️ 첫 λ‘œλ”©: ~1-2λΆ„
573
+ """
574
+ elif HW_ENV['hardware'] == 'cpu_basic':
575
+ footer += """
576
+ - πŸ’‘ 무료 ν‹°μ–΄ (μ œν•œμ )
577
+ - ⏳ CPU ν™˜κ²½ (1~2λΆ„ 응닡)
578
+ - πŸ”’ κ²½λŸ‰ λͺ¨λΈ ꢌμž₯
579
+ - ⏱️ 첫 λ‘œλ”©: ~2-3λΆ„
580
+ """
581
+ elif HW_ENV['hardware'] == 'local_gpu':
582
+ footer += f"""
583
+ - πŸ–₯️ GPU 가속: {HW_ENV['gpu_name']}
584
+ - ⚑ λΉ λ₯Έ 응닡 (5-10초)
585
+ - πŸ”“ λ¬΄μ œν•œ μ‚¬μš©
586
+ - ⏱️ 첫 λ‘œλ”©: ~10-20초
587
+ """
588
+ else: # local_cpu
589
+ footer += f"""
590
+ - πŸ’» 둜컬 CPU ({HW_ENV['cpu_count']} μ½”μ–΄)
591
+ - ⏳ 느린 응닡 (1~3λΆ„)
592
+ - πŸ”’ κ²½λŸ‰ λͺ¨λΈ ꢌμž₯
593
+ - ⏱️ 첫 λ‘œλ”©: ~2-5λΆ„
594
+ """
595
+
596
+ footer += """
597
+ **ν…ŒμŠ€νŠΈ μ˜ˆμ‹œ**:
598
+ - "μ•ˆλ…•ν•˜μ„Έμš”"
599
+ - "인곡지λŠ₯에 λŒ€ν•΄ μ„€λͺ…ν•΄μ£Όμ„Έμš”"
600
+ - "ν•œκ΅­μ˜ μˆ˜λ„λŠ” μ–΄λ””μΈκ°€μš”?"
601
+ """
602
 
603
  gr.Markdown(footer)
604