LOOFYYLO commited on
Commit
4a12ac6
·
verified ·
1 Parent(s): aa06975

Upload folder using huggingface_hub

Browse files
README.md CHANGED
@@ -1,12 +1,62 @@
1
  ---
2
- title: Interactive Futures Model
3
- emoji: 💻
4
- colorFrom: green
5
- colorTo: blue
6
- sdk: gradio
7
- sdk_version: 6.2.0
8
- app_file: app.py
9
- pinned: false
10
  ---
11
 
12
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: mit
3
+ tags:
4
+ - futures-prediction
5
+ - multi-dimensional
6
+ - mixture-of-experts
7
+ - state-space-model
 
 
8
  ---
9
 
10
+ # Futures Prediction Model (MoE + SSM + FiLM)
11
+
12
+ This repository contains the code and trained weights for a novel architecture designed for multi-dimensional futures prediction. The model was trained on the `futures_dataset_v2.json` dataset.
13
+
14
+ ## Model Description
15
+
16
+ The model architecture is a combination of:
17
+
18
+ * **Mixture of Experts (MoE):** To handle the multi-dimensional nature of futures scenarios.
19
+ * **State Space Model (SSM):** To capture the temporal evolution of futures.
20
+ * **FiLM Conditioning:** To modulate the model's behavior based on the different future axes.
21
+
22
+ The model is trained to predict a 12-dimensional vector of weights, each corresponding to a different future "axis".
23
+
24
+ ## How to Use
25
+
26
+ To use this model, you will need to have PyTorch installed. You can then use the `load_model.py` script to load the model and tokenizer.
27
+
28
+ ```python
29
+ from load_model import load_model_and_tokenizer
30
+
31
+ model, tokenizer = load_model_and_tokenizer()
32
+
33
+ text = "In a future dominated by hyper-automation, societal structures adapt to new forms of labor and community."
34
+ token_ids = tokenizer.encode(text)
35
+ tokens_tensor = torch.LongTensor(token_ids).unsqueeze(0)
36
+
37
+ with torch.no_grad():
38
+ axis_logits, _, _ = model(tokens_tensor)
39
+ axis_predictions = torch.sigmoid(axis_logits)
40
+
41
+ print(axis_predictions)
42
+ ```
43
+
44
+ ## Training Data
45
+
46
+ The model was trained on the `futures_dataset_v2.json` dataset, which contains 3,000 rich, multi-dimensional futures scenarios.
47
+
48
+ ## Training Procedure
49
+
50
+ The model was trained for 100 epochs with a batch size of 16 and a learning rate of 1e-4. The training script `train_futures_model.py` is available in the original repository.
51
+
52
+ ## Citing
53
+
54
+ If you use this model or code, please cite:
55
+
56
+ ```
57
+ @article{futures-representation-learning,
58
+ title={Learning Multi-Dimensional Futures Representations with Mixture-of-Experts and State Space Models},
59
+ author={Your Name},
60
+ year={2024}
61
+ }
62
+ ```
__init__.py ADDED
File without changes
__pycache__/__init__.cpython-312.pyc ADDED
Binary file (117 Bytes). View file
 
__pycache__/load_model.cpython-312.pyc ADDED
Binary file (3.97 kB). View file
 
__pycache__/model.cpython-312.pyc ADDED
Binary file (13.3 kB). View file
 
__pycache__/push_to_hub.cpython-312.pyc ADDED
Binary file (3.06 kB). View file
 
__pycache__/push_to_space.cpython-312.pyc ADDED
Binary file (3.51 kB). View file
 
checkpoint_best.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73abc01892b82fb6a17f2b6c1d8fc93dcc6f50f571639dd8a3f7d42c2becfc0a
3
+ size 24407482
futures_dataset_v2.json ADDED
The diff for this file is too large to render. See raw diff
 
gradio_app.py ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import gradio as gr
2
+ import torch
3
+ from app.load_model import load_model_and_tokenizer
4
+
5
+ # --- 1. Load Model and Tokenizer ---
6
+ # This is done once when the Gradio app starts.
7
+ try:
8
+ print("Loading model and tokenizer...")
9
+ model, tokenizer = load_model_and_tokenizer()
10
+ print("✅ Model and tokenizer loaded successfully.")
11
+ except Exception as e:
12
+ print(f"❌ Failed to load model: {e}")
13
+ model, tokenizer = None, None
14
+
15
+ # --- 2. Define the Prediction Function ---
16
+ # This function is called every time a user interacts with the demo.
17
+ def predict_futures(text):
18
+ """
19
+ Takes raw text input, tokenizes it, gets model predictions,
20
+ and formats the output for the Gradio interface.
21
+ """
22
+ if not model or not tokenizer:
23
+ return "Model not loaded. Please check the logs.", {}
24
+
25
+ try:
26
+ # a. Preprocess: Tokenize the input text
27
+ token_ids = tokenizer.encode(text)
28
+ tokens_tensor = torch.LongTensor(token_ids).unsqueeze(0) # Add batch dimension
29
+
30
+ # b. Predict: Get model's raw output (logits)
31
+ with torch.no_grad():
32
+ axis_logits, _, _ = model(tokens_tensor)
33
+ # c. Post-process: Apply sigmoid to get probabilities (0-1)
34
+ axis_predictions = torch.sigmoid(axis_logits)
35
+
36
+ # d. Format Output: Create a dictionary for the label component
37
+ axis_names = [
38
+ "Hyper-Automation", "Human-Tech Symbiosis", "Abundance", "Individualism",
39
+ "Community Focus", "Global Interconnectedness", "Crisis & Collapse", "Restoration & Healing",
40
+ "Adaptation & Resilience", "Digital Dominance", "Physical Embodiment", "Collaboration"
41
+ ]
42
+
43
+ # Create a dictionary of {label: confidence}
44
+ confidences = {name: float(weight) for name, weight in zip(axis_names, axis_predictions[0])}
45
+
46
+ # You can return a simple message and the formatted labels
47
+ return "Prediction complete.", confidences
48
+
49
+ except Exception as e:
50
+ print(f"Error during prediction: {e}")
51
+ return f"An error occurred: {e}", {}
52
+
53
+ # --- 3. Create and Launch the Gradio Interface ---
54
+ print("Creating Gradio interface...")
55
+
56
+ # Define the input and output components
57
+ input_text = gr.Textbox(
58
+ lines=5,
59
+ label="Input Scenario",
60
+ placeholder="Describe a future scenario here..."
61
+ )
62
+
63
+ output_text = gr.Textbox(label="Status")
64
+ output_labels = gr.Label(label="Predicted Axis Weights", num_top_classes=12)
65
+
66
+ # Build the interface
67
+ demo = gr.Interface(
68
+ fn=predict_futures,
69
+ inputs=input_text,
70
+ outputs=[output_text, output_labels],
71
+ title="Futures Prediction Model",
72
+ description=(
73
+ "Explore multi-dimensional futures. "
74
+ "Write a text describing a potential future scenario and see how the model scores it "
75
+ "across 12 different axes, from 'Hyper-Automation' to 'Crisis & Collapse'."
76
+ ),
77
+ examples=[
78
+ ["In a future dominated by hyper-automation, societal structures adapt to new forms of labor and community."],
79
+ ["Coastal cities adopt divergent strategies as sea levels rise. Singapore invests in autonomous seawall monitoring, while Jakarta facilitates managed retreat."],
80
+ ["A global pandemic leads to a surge in community-focused initiatives and a renewed appreciation for local supply chains."]
81
+ ]
82
+ )
83
+
84
+ if __name__ == "__main__":
85
+ print("Launching Gradio demo...")
86
+ # The launch() command creates a shareable link to the demo.
87
+ demo.launch()
load_model.py ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ from .model import FuturesModel, CustomTokenizer, build_vocabulary
3
+
4
+ def load_model_and_tokenizer(
5
+ model_path='app/checkpoint_best.pt',
6
+ dataset_path='app/futures_dataset_v2.json',
7
+ vocab_size=5000,
8
+ ):
9
+ """Loads the trained FuturesModel and CustomTokenizer."""
10
+
11
+ # 1. Build vocabulary and tokenizer
12
+ print("Building vocabulary from dataset...")
13
+ vocab_dict = build_vocabulary(dataset_path, vocab_size=vocab_size)
14
+ tokenizer = CustomTokenizer(vocab_dict)
15
+ print(f"Vocabulary size: {len(vocab_dict)}")
16
+
17
+ # 2. Initialize the model with the same architecture
18
+ print("Initializing model...")
19
+ model = FuturesModel(
20
+ vocab_size=len(vocab_dict),
21
+ n_axes=12,
22
+ d_model=256,
23
+ n_head=8,
24
+ n_layers=4,
25
+ n_experts=8,
26
+ dropout=0.1
27
+ )
28
+ print(f"Model parameters: {sum(p.numel() for p in model.parameters()):,}")
29
+
30
+ # 3. Load the saved state dictionary
31
+ print(f"Loading model weights from {model_path}...")
32
+ device = 'cuda' if torch.cuda.is_available() else 'cpu'
33
+ checkpoint = torch.load(model_path, map_location=device)
34
+
35
+ # The state dict is nested in the checkpoint
36
+ model.load_state_dict(checkpoint['model_state_dict'])
37
+
38
+ # 4. Set the model to evaluation mode
39
+ model.eval()
40
+ print("Model set to evaluation mode.")
41
+
42
+ return model, tokenizer
43
+
44
+ if __name__ == "__main__":
45
+ print("="*80)
46
+ print("Loading Futures Prediction Model")
47
+ print("="*80)
48
+
49
+ # Correct paths for running from the root directory
50
+ model_path = 'app/checkpoint_best.pt'
51
+ dataset_path = 'app/futures_dataset_v2.json'
52
+
53
+ try:
54
+ model, tokenizer = load_model_and_tokenizer(
55
+ model_path=model_path,
56
+ dataset_path=dataset_path
57
+ )
58
+ print("\n✅ Model and tokenizer loaded successfully!")
59
+
60
+ # Example usage
61
+ print("\n--- Example Usage ---")
62
+ text = "In a future dominated by hyper-automation, societal structures adapt to new forms of labor and community."
63
+ print(f"Input text: '{text}'")
64
+
65
+ token_ids = tokenizer.encode(text)
66
+ tokens_tensor = torch.LongTensor(token_ids).unsqueeze(0) # Add batch dimension
67
+
68
+ print(f"Encoded tokens (first 10): {tokens_tensor[0, :10]}...")
69
+
70
+ with torch.no_grad():
71
+ axis_logits, lm_logits, stats = model(tokens_tensor)
72
+ axis_predictions = torch.sigmoid(axis_logits)
73
+
74
+ print("\nPredicted Axis Weights:")
75
+ axis_names = [
76
+ "HyperAuto", "HumanTech", "Abundant", "Individual",
77
+ "Community", "Global", "Crisis", "Restore",
78
+ "Adapt", "Digital", "Physical", "Collab"
79
+ ]
80
+ for name, weight in zip(axis_names, axis_predictions[0]):
81
+ print(f" - {name:12s}: {weight:.4f}")
82
+
83
+ except Exception as e:
84
+ print(f"\n❌ An error occurred during loading: {e}")
85
+ import traceback
86
+ traceback.print_exc()
87
+
88
+ print("\n" + "="*80)
model.py ADDED
@@ -0,0 +1,280 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ import torch.nn as nn
3
+ import torch.nn.functional as F
4
+ from torch.utils.data import Dataset, DataLoader
5
+ import json
6
+ import numpy as np
7
+ from collections import Counter
8
+ import pickle
9
+ from typing import Dict, List, Tuple
10
+ import os
11
+
12
+ # ============================================================================
13
+ # TOKENIZER
14
+ # ============================================================================
15
+
16
+ class CustomTokenizer:
17
+ def __init__(self, vocab_dict):
18
+ self.vocab_dict = vocab_dict
19
+ self.idx_to_word = {idx: word for word, idx in vocab_dict.items()}
20
+ self.pad_token_id = vocab_dict['<PAD>']
21
+ self.unk_token_id = vocab_dict['<UNK>']
22
+ self.vocab_size = len(vocab_dict)
23
+
24
+ def encode(self, text, max_length=128):
25
+ text = text.lower()
26
+ # Simple tokenization
27
+ text = text.replace(',', ' ,').replace('.', ' .')
28
+ text = text.replace('(', ' ( ').replace(')', ' ) ')
29
+ words = text.split()
30
+
31
+ token_ids = [self.vocab_dict.get(w, self.unk_token_id) for w in words]
32
+
33
+ if len(token_ids) < max_length:
34
+ token_ids += [self.pad_token_id] * (max_length - len(token_ids))
35
+ else:
36
+ token_ids = token_ids[:max_length]
37
+
38
+ return token_ids
39
+
40
+ def decode(self, token_ids, skip_special_tokens=True):
41
+ words = []
42
+ special_tokens = ['<PAD>', '<UNK>', '<START>', '<END>']
43
+
44
+ for idx in token_ids:
45
+ word = self.idx_to_word.get(int(idx), '<UNK>')
46
+ if skip_special_tokens and word in special_tokens:
47
+ continue
48
+ words.append(word)
49
+
50
+ return ' '.join(words)
51
+
52
+
53
+ def build_vocabulary(dataset_path, vocab_size=5000):
54
+ """Build vocabulary from dataset"""
55
+ print("Building vocabulary...")
56
+ with open(dataset_path) as f:
57
+ data = json.load(f)
58
+
59
+ word_counts = Counter()
60
+ for sample in data['samples']:
61
+ text = sample['text'].lower()
62
+ text = text.replace(',', ' ,').replace('.', ' .')
63
+ words = text.split()
64
+ word_counts.update(words)
65
+
66
+ special_tokens = ['<PAD>', '<UNK>', '<START>', '<END>']
67
+ top_words = [word for word, _ in word_counts.most_common(vocab_size - len(special_tokens))]
68
+ vocabulary = special_tokens + top_words
69
+
70
+ vocab_dict = {word: idx for idx, word in enumerate(vocabulary)}
71
+
72
+ print(f"Vocabulary size: {len(vocab_dict)}")
73
+ return vocab_dict
74
+
75
+
76
+ # ============================================================================
77
+ # MODEL COMPONENTS
78
+ # ============================================================================
79
+
80
+ class MixtureOfExperts(nn.Module):
81
+ """MoE for handling multi-dimensional futures"""
82
+
83
+ def __init__(self, d_model, n_experts=8, expert_dim=256, dropout=0.1):
84
+ super().__init__()
85
+ self.n_experts = n_experts
86
+
87
+ # Experts (simple FFNs)
88
+ self.experts = nn.ModuleList([
89
+ nn.Sequential(
90
+ nn.Linear(d_model, expert_dim),
91
+ nn.GELU(),
92
+ nn.Dropout(dropout),
93
+ nn.Linear(expert_dim, d_model),
94
+ nn.Dropout(dropout)
95
+ ) for _ in range(n_experts)
96
+ ])
97
+
98
+ # Gating network
99
+ self.gate = nn.Linear(d_model, n_experts)
100
+
101
+ def forward(self, x):
102
+ # x: (batch, seq, d_model)
103
+ batch_size, seq_len, d_model = x.shape
104
+
105
+ # Compute gates
106
+ gate_logits = self.gate(x) # (batch, seq, n_experts)
107
+ gate_weights = F.softmax(gate_logits, dim=-1)
108
+
109
+ # Apply experts
110
+ expert_outputs = torch.stack([expert(x) for expert in self.experts], dim=2)
111
+ # (batch, seq, n_experts, d_model)
112
+
113
+ # Weighted combination
114
+ gate_weights_expanded = gate_weights.unsqueeze(-1) # (batch, seq, n_experts, 1)
115
+ output = (expert_outputs * gate_weights_expanded).sum(dim=2) # (batch, seq, d_model)
116
+
117
+ # Gate statistics for loss
118
+ gate_entropy = -(gate_weights * torch.log(gate_weights + 1e-10)).sum(dim=-1).mean()
119
+ gate_std = gate_weights.std(dim=-1).mean()
120
+
121
+ return output, gate_entropy, gate_std
122
+
123
+
124
+ class TrajectorySSM(nn.Module):
125
+ """State Space Model for temporal trajectories"""
126
+
127
+ def __init__(self, d_model, state_dim=64):
128
+ super().__init__()
129
+ self.state_dim = state_dim
130
+
131
+ # State matrices
132
+ self.A = nn.Parameter(torch.randn(state_dim, state_dim) * 0.01)
133
+ self.B = nn.Parameter(torch.randn(state_dim, d_model) * 0.01)
134
+ self.C = nn.Parameter(torch.randn(d_model, state_dim) * 0.01)
135
+ self.D = nn.Parameter(torch.randn(d_model, d_model) * 0.01)
136
+
137
+ # Learnable initialization
138
+ self.h0 = nn.Parameter(torch.zeros(1, state_dim))
139
+
140
+ def forward(self, x):
141
+ # x: (batch, seq, d_model)
142
+ batch_size, seq_len, d_model = x.shape
143
+
144
+ # Initialize state
145
+ h = self.h0.expand(batch_size, -1) # (batch, state_dim)
146
+
147
+ outputs = []
148
+ for t in range(seq_len):
149
+ x_t = x[:, t, :] # (batch, d_model)
150
+
151
+ # Update state: h_t = Ah_{t-1} + Bx_t
152
+ h = torch.matmul(h, self.A.t()) + torch.matmul(x_t, self.B.t())
153
+
154
+ # Output: y_t = Ch_t + Dx_t
155
+ y = torch.matmul(h, self.C.t()) + torch.matmul(x_t, self.D.t())
156
+ outputs.append(y)
157
+
158
+ output = torch.stack(outputs, dim=1) # (batch, seq, d_model)
159
+ return output, h
160
+
161
+
162
+ class FiLMConditioning(nn.Module):
163
+ """Feature-wise Linear Modulation for axis conditioning"""
164
+
165
+ def __init__(self, d_model, n_axes=12):
166
+ super().__init__()
167
+ self.gamma = nn.Linear(n_axes, d_model)
168
+ self.beta = nn.Linear(n_axes, d_model)
169
+
170
+ def forward(self, x, axis_weights):
171
+ # x: (batch, seq, d_model)
172
+ # axis_weights: (batch, n_axes)
173
+
174
+ gamma = self.gamma(axis_weights).unsqueeze(1) # (batch, 1, d_model)
175
+ beta = self.beta(axis_weights).unsqueeze(1)
176
+
177
+ return gamma * x + beta
178
+
179
+
180
+ # ============================================================================
181
+ # MAIN MODEL
182
+ # ============================================================================
183
+
184
+ class FuturesModel(nn.Module):
185
+ """Complete MoE + SSM + FiLM model for futures learning"""
186
+
187
+ def __init__(
188
+ self,
189
+ vocab_size,
190
+ n_axes=12,
191
+ d_model=256,
192
+ n_head=8,
193
+ n_layers=4,
194
+ n_experts=8,
195
+ dropout=0.1
196
+ ):
197
+ super().__init__()
198
+
199
+ self.d_model = d_model
200
+ self.n_axes = n_axes
201
+
202
+ # Embeddings
203
+ self.token_emb = nn.Embedding(vocab_size, d_model)
204
+ self.pos_emb = nn.Embedding(128, d_model)
205
+
206
+ # Transformer layers
207
+ self.layers = nn.ModuleList([
208
+ nn.ModuleDict({
209
+ 'attn': nn.MultiheadAttention(d_model, n_head, dropout=dropout, batch_first=True),
210
+ 'moe': MixtureOfExperts(d_model, n_experts=n_experts, dropout=dropout),
211
+ 'ssm': TrajectorySSM(d_model),
212
+ 'film': FiLMConditioning(d_model, n_axes),
213
+ 'norm1': nn.LayerNorm(d_model),
214
+ 'norm2': nn.LayerNorm(d_model),
215
+ 'norm3': nn.LayerNorm(d_model),
216
+ }) for _ in range(n_layers)
217
+ ])
218
+
219
+ # Output heads
220
+ self.axis_head = nn.Sequential(
221
+ nn.Linear(d_model, d_model),
222
+ nn.GELU(),
223
+ nn.Dropout(dropout),
224
+ nn.Linear(d_model, n_axes)
225
+ )
226
+
227
+ self.lm_head = nn.Linear(d_model, vocab_size)
228
+
229
+ self.dropout = nn.Dropout(dropout)
230
+
231
+ def forward(self, tokens, axis_weights=None):
232
+ batch_size, seq_len = tokens.shape
233
+
234
+ # Embeddings
235
+ x = self.token_emb(tokens)
236
+ pos = torch.arange(seq_len, device=tokens.device).unsqueeze(0).expand(batch_size, -1)
237
+ x = x + self.pos_emb(pos)
238
+ x = self.dropout(x)
239
+
240
+ # Track statistics
241
+ gate_entropies = []
242
+ gate_stds = []
243
+
244
+ # Transformer layers with MoE, SSM, FiLM
245
+ for layer in self.layers:
246
+ # Self-attention
247
+ attn_out, _ = layer['attn'](x, x, x)
248
+ x = layer['norm1'](x + attn_out)
249
+
250
+ # MoE
251
+ moe_out, gate_entropy, gate_std = layer['moe'](x)
252
+ gate_entropies.append(gate_entropy)
253
+ gate_stds.append(gate_std)
254
+ x = layer['norm2'](x + moe_out)
255
+
256
+ # SSM (for temporal modeling)
257
+ ssm_out, _ = layer['ssm'](x)
258
+ x = x + ssm_out
259
+
260
+ # FiLM conditioning (if axis weights provided)
261
+ if axis_weights is not None:
262
+ x = layer['film'](x, axis_weights)
263
+
264
+ x = layer['norm3'](x)
265
+
266
+ # Mean pooling for axis classification
267
+ mask = (tokens != 0).float().unsqueeze(-1)
268
+ x_masked = x * mask
269
+ x_pooled = x_masked.sum(dim=1) / mask.sum(dim=1).clamp(min=1)
270
+
271
+ # Outputs
272
+ axis_logits = self.axis_head(x_pooled) # (batch, n_axes) - for regression
273
+ lm_logits = self.lm_head(x) # (batch, seq, vocab_size)
274
+
275
+ stats = {
276
+ 'gate_entropy': torch.stack(gate_entropies).mean(),
277
+ 'gate_std': torch.stack(gate_stds).mean()
278
+ }
279
+
280
+ return axis_logits, lm_logits, stats
push_to_hub.py ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ from huggingface_hub import HfApi, HfFolder
3
+
4
+ def push_to_huggingface_hub(
5
+ repo_name,
6
+ username,
7
+ folder_path='app',
8
+ token=None
9
+ ):
10
+ """
11
+ Pushes the contents of a folder to a new Hugging Face Hub repository.
12
+
13
+ Args:
14
+ repo_name (str): The name of the repository to create on the Hub.
15
+ username (str): Your Hugging Face Hub username.
16
+ folder_path (str, optional): The local folder to upload. Defaults to 'app'.
17
+ token (str, optional): Your Hugging Face Hub token. If not provided,
18
+ it will be read from the environment or a login.
19
+ """
20
+ if token:
21
+ HfFolder.save_token(token)
22
+ print("Hugging Face token saved.")
23
+
24
+ api = HfApi()
25
+ repo_id = f"{username}/{repo_name}"
26
+
27
+ # 1. Create the repository on the Hub
28
+ print(f"Creating repository: {repo_id}")
29
+ try:
30
+ api.create_repo(repo_id=repo_id, exist_ok=True, repo_type="model")
31
+ print(f"Repository '{repo_id}' created or already exists.")
32
+ except Exception as e:
33
+ print(f"❌ Error creating repository: {e}")
34
+ return
35
+
36
+ # 2. Upload the entire folder
37
+ print(f"Uploading contents of '{folder_path}' to '{repo_id}'...")
38
+ try:
39
+ api.upload_folder(
40
+ folder_path=folder_path,
41
+ repo_id=repo_id,
42
+ repo_type="model",
43
+ )
44
+ print("\n✅ Successfully uploaded files to the Hugging Face Hub!")
45
+ print(f"Model available at: https://huggingface.co/{repo_id}")
46
+ except Exception as e:
47
+ print(f"\n❌ An error occurred during upload: {e}")
48
+ print("Please ensure your token has 'write' permissions.")
49
+
50
+ if __name__ == "__main__":
51
+ print("="*80)
52
+ print("Pushing Model to Hugging Face Hub")
53
+ print("="*80)
54
+
55
+ # --- User Configuration ---
56
+ # Replace with your details
57
+ HF_USERNAME = "jules-agent" # <-- IMPORTANT: SET YOUR HF USERNAME
58
+ HF_REPO_NAME = "futures-prediction-model" # <-- Choose a name for your model repo
59
+
60
+ # The token is read from the environment variable for security
61
+ HF_TOKEN = os.getenv("HF_TOKEN")
62
+
63
+ if HF_USERNAME == "your-username" or not HF_TOKEN:
64
+ print("\n⚠️ Please configure your Hugging Face username and token in this script.")
65
+ print(" - Set HF_USERNAME to your username.")
66
+ print(" - Set the HF_TOKEN environment variable with your write token.")
67
+ else:
68
+ push_to_huggingface_hub(
69
+ repo_name=HF_REPO_NAME,
70
+ username=HF_USERNAME,
71
+ token=HF_TOKEN
72
+ )
73
+
74
+ print("\n" + "="*80)
push_to_space.py ADDED
@@ -0,0 +1,90 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ from huggingface_hub import HfApi, login
3
+
4
+ def deploy_to_huggingface_space(
5
+ repo_name,
6
+ username,
7
+ folder_path='app',
8
+ token=None,
9
+ app_file="gradio_app.py"
10
+ ):
11
+ """
12
+ Pushes the contents of a folder to a new Hugging Face Space.
13
+
14
+ Args:
15
+ repo_name (str): The name of the Space repository to create.
16
+ username (str): Your Hugging Face Hub username.
17
+ folder_path (str, optional): The local folder to upload. Defaults to 'app'.
18
+ token (str, optional): Your Hugging Face Hub write token.
19
+ app_file (str, optional): The main application file. Defaults to "gradio_app.py".
20
+ """
21
+ if not token:
22
+ print("❌ Hugging Face token not found. Please set the HF_TOKEN environment variable.")
23
+ return
24
+
25
+ # 1. Log in to Hugging Face
26
+ print("Logging in to Hugging Face Hub...")
27
+ try:
28
+ login(token=token, add_to_git_credential=False)
29
+ print("✅ Login successful.")
30
+ except Exception as e:
31
+ print(f"❌ Login failed: {e}")
32
+ return
33
+
34
+ api = HfApi()
35
+ repo_id = f"{username}/{repo_name}"
36
+
37
+ # 2. Create the Space repository on the Hub
38
+ print(f"Creating Space repository: {repo_id}")
39
+ try:
40
+ api.create_repo(
41
+ repo_id=repo_id,
42
+ repo_type="space",
43
+ space_sdk="gradio",
44
+ exist_ok=True,
45
+ )
46
+ print(f"✅ Space repository '{repo_id}' created or already exists.")
47
+ except Exception as e:
48
+ print(f"❌ Error creating repository: {e}")
49
+ return
50
+
51
+ # 3. Upload the entire application folder to the Space
52
+ print(f"Uploading contents of '{folder_path}' to '{repo_id}'...")
53
+ try:
54
+ # This will upload all files from the 'app' directory to the root of the Space repo
55
+ api.upload_folder(
56
+ folder_path=folder_path,
57
+ repo_id=repo_id,
58
+ repo_type="space",
59
+ )
60
+ print("\n✅ Successfully uploaded files to the Hugging Face Space!")
61
+ print(f"Interactive demo available at: https://huggingface.co/spaces/{repo_id}")
62
+ except Exception as e:
63
+ print(f"\n❌ An error occurred during upload: {e}")
64
+ print("Please ensure your token has 'write' permissions.")
65
+
66
+ if __name__ == "__main__":
67
+ print("="*80)
68
+ print("Deploying Gradio App to Hugging Face Spaces")
69
+ print("="*80)
70
+
71
+ # --- User Configuration ---
72
+ HF_USERNAME = "LOOFYYLO"
73
+ HF_SPACE_NAME = "interactive-futures-model"
74
+
75
+ # Securely get the token from an environment variable
76
+ HF_TOKEN = os.getenv("HF_TOKEN")
77
+
78
+ if HF_USERNAME == "your-username" or not HF_TOKEN:
79
+ print("\n⚠️ Please configure your Hugging Face username and token.")
80
+ print(" - Set HF_USERNAME in this script.")
81
+ print(" - Set the HF_TOKEN environment variable with your write token.")
82
+ else:
83
+ deploy_to_huggingface_space(
84
+ repo_name=HF_SPACE_NAME,
85
+ username=HF_USERNAME,
86
+ token=HF_TOKEN,
87
+ folder_path='app' # The folder containing our app files
88
+ )
89
+
90
+ print("\n" + "="*80)
requirements.txt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ torch
2
+ numpy
3
+ gradio