TroglodyteDerivations commited on
Commit
6705b44
Β·
verified Β·
1 Parent(s): 6a8b697

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +123 -0
README.md ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - openai/gpt-oss-120b
5
+ - deepseek-ai/DeepSeek-V3.1
6
+ tags:
7
+ - chat-interface
8
+ - gpt-oss-120b-chat-interface
9
+ ---
10
+ # Model Card: GPT-OSS-120B Chat Interface
11
+
12
+ ```markdown
13
+ ---
14
+ license: apache-2.0
15
+ tags:
16
+ - mlx
17
+ - chat-ui
18
+ - local-ai
19
+ - gpt
20
+ - python
21
+ - pyqt5
22
+ ---
23
+
24
+ # GPT-OSS-120B Chat Interface
25
+
26
+ ## Model Description
27
+
28
+ This is a modern, feature-rich chat interface for the GPT-OSS-120B model running on Apple MLX framework. The interface provides a user-friendly way to interact with the 120-billion parameter open-source language model locally on Apple Silicon hardware.
29
+
30
+ ## Model Overview
31
+
32
+ - **Model Name:** GPT-OSS-120B (4-bit quantized)
33
+ - **Framework:** Apple MLX
34
+ - **Interface:** PyQt5-based desktop application
35
+ - **Hardware Requirements:** Apple Silicon with sufficient RAM (recommended: M3 Ultra with 512GB RAM)
36
+
37
+ ## Features
38
+
39
+ - 🎨 Modern, responsive UI with PyQt5
40
+ - πŸ’¬ Real-time chat interface with message history
41
+ - ⚑ Local inference on Apple Silicon
42
+ - πŸ“ Markdown support with syntax highlighting
43
+ - πŸ’Ύ Conversation export functionality
44
+ - βš™οΈ Adjustable generation parameters
45
+ - 🎯 Code block detection and formatting
46
+ - πŸ“Š Performance monitoring
47
+
48
+ ## UI Architecture Diagram
49
+
50
+ ```
51
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
52
+ β”‚ MAIN WINDOW β”‚
53
+ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
54
+ β”‚ LEFT PANEL β”‚ CHAT AREA β”‚
55
+ β”‚ β”‚ β”‚
56
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
57
+ β”‚ β”‚ MODEL INFO β”‚ β”‚ β”‚ β”‚ β”‚
58
+ β”‚ β”‚ - Model details β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
59
+ β”‚ β”‚ - Hardware specs β”‚ β”‚ β”‚ β”‚ CHAT MESSAGE (User) β”‚ β”‚ β”‚
60
+ β”‚ β”‚ - Performance metrics β”‚ β”‚ β”‚ β”‚ - Avatar + timestamp β”‚ β”‚ β”‚
61
+ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ - Formatted content β”‚ β”‚ β”‚
62
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
63
+ β”‚ β”‚ β”‚ β”‚ β”‚
64
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
65
+ β”‚ β”‚ GENERATION SETTINGS β”‚ β”‚ β”‚ β”‚ CHAT MESSAGE (Assistant) β”‚ β”‚ β”‚
66
+ β”‚ β”‚ - Max tokens control β”‚ β”‚ β”‚ β”‚ - Avatar + timestamp β”‚ β”‚ β”‚
67
+ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ - Formatted content β”‚ β”‚ β”‚
68
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ - Generation time β”‚ β”‚ β”‚
69
+ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
70
+ β”‚ β”‚ β”‚ ... β”‚ β”‚
71
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚
72
+ β”‚ β”‚ CONVERSATION TOOLS β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
73
+ β”‚ β”‚ - Clear conversation β”‚ β”‚ β”‚ β”‚ INPUT AREA β”‚ β”‚ β”‚
74
+ β”‚ β”‚ - Export chat β”‚ β”‚ β”‚ β”‚ - Multi-line text input β”‚ β”‚ β”‚
75
+ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ - Character counter β”‚ β”‚ β”‚
76
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ - Send button β”‚ β”‚ β”‚
77
+ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
78
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚
79
+ β”‚ β”‚ STATUS INDICATOR β”‚ β”‚ β”‚ β”‚ β”‚
80
+ β”‚ β”‚ - Loading/ready state β”‚ β”‚ β”‚ β”‚ β”‚
81
+ β”‚ β”‚ οΏ½οΏ½ β”‚ β”‚ β”‚ β”‚
82
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚
83
+ β”‚ β”‚ β”‚ β”‚ β”‚
84
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
85
+ ```
86
+
87
+ ## Development
88
+
89
+ This interface was developed using PyQt5 and integrates with the MLX-LM library for efficient inference on Apple Silicon. The UI features a responsive design with:
90
+
91
+ 1. **Threaded Operations:** Model loading and text generation run in background threads
92
+ 2. **Custom Widgets:** Specialized chat message widgets with formatting
93
+ 3. **Syntax Highlighting:** Code detection and highlighting in responses
94
+ 4. **Modern Styling:** Clean, professional interface with appropriate spacing and colors
95
+
96
+ ## DeepSeek Involvement
97
+
98
+ The `gpt_oss_ui.py` Python script was created with assistance from DeepSeek's AI models, which helped design the architecture, implement the PyQt5 interface components, and ensure proper integration with the MLX inference backend.
99
+
100
+ ## Usage
101
+
102
+ 1. Install requirements: `pip install PyQt5 markdown mlx-lm`
103
+ 2. Run the application: `python gpt_oss_ui.py`
104
+ 3. Wait for model to load (first time will download the model)
105
+ 4. Start chatting with the GPT-OSS-120B model
106
+
107
+ ## Performance
108
+
109
+ On an M3 Ultra with 512GB RAM:
110
+ - Model load time: ~2-3 minutes (first time)
111
+ - Inference speed: ~95 tokens/second
112
+ - Memory usage: Optimized with 4-bit quantization
113
+
114
+ ## Limitations
115
+
116
+ - Requires significant RAM for the 120B parameter model
117
+ - Currently only supports Apple Silicon hardware
118
+ - Model loading can be time-consuming on first run
119
+
120
+ ## Ethical Considerations
121
+
122
+ This interface is designed for local use, ensuring privacy as all processing happens on-device. Users should still follow responsible AI practices when using the model.
123
+ ```