p80x commited on
Commit
a356aee
·
1 Parent(s): 3bd8cfb

Deploy hf-save: Go API backend and multi-platform CLI installer

Browse files
Files changed (5) hide show
  1. Dockerfile +42 -0
  2. README.md +84 -3
  3. cli/main.go +550 -0
  4. go.mod +3 -0
  5. main.go +464 -0
Dockerfile ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Build Stage
2
+ FROM golang:1.21-alpine AS builder
3
+
4
+ WORKDIR /app
5
+
6
+ # Copy the server and cli source code
7
+ COPY main.go go.mod ./
8
+ COPY cli/ cli/
9
+
10
+ # Create directory to store precompiled binaries
11
+ RUN mkdir -p bin
12
+
13
+ # Cross-compile CLI for various target OS and architectures
14
+ RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-s -w" -o bin/linux-amd64 cli/main.go
15
+ RUN CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags="-s -w" -o bin/linux-arm64 cli/main.go
16
+ RUN CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -ldflags="-s -w" -o bin/windows-amd64 cli/main.go
17
+ RUN CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -ldflags="-s -w" -o bin/darwin-amd64 cli/main.go
18
+ RUN CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -ldflags="-s -w" -o bin/darwin-arm64 cli/main.go
19
+
20
+ # Build the main server binary statically linked
21
+ RUN CGO_ENABLED=0 go build -ldflags="-s -w" -o server main.go
22
+
23
+ # Production Stage
24
+ FROM scratch
25
+
26
+ WORKDIR /app
27
+
28
+ # Copy the server binary and precompiled CLI binaries
29
+ COPY --from=builder /app/server /app/server
30
+ COPY --from=builder /app/bin /app/bin
31
+
32
+ # Environment variables (Can be overridden in HF Spaces configuration)
33
+ ENV DATA_DIR=/data
34
+ ENV BIN_DIR=/app/bin
35
+ ENV PORT=7860
36
+
37
+ # Expose default HF Spaces port
38
+ EXPOSE 7860
39
+
40
+ # Command to run backend
41
+ CMD ["/app/server"]
42
+
README.md CHANGED
@@ -1,10 +1,91 @@
1
  ---
2
- title: Store
3
- emoji: 😻
4
  colorFrom: blue
5
  colorTo: indigo
6
  sdk: docker
 
7
  pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: hf-save
3
+ emoji: 💾
4
  colorFrom: blue
5
  colorTo: indigo
6
  sdk: docker
7
+ app_port: 7860
8
  pinned: false
9
  ---
10
 
11
+ # hf-save
12
+
13
+ An ultra-fast, zero-overhead developer tool to save work artifacts from ephemeral GPU instances (RunPod, Vast.ai, Lambda Labs) and restore them later.
14
+
15
+ Uses a **Hugging Face Docker Space** with a **Dataset Storage Mount** at `/data` as the backend and a **Single-binary Go CLI** on the client side.
16
+
17
+
18
+ ---
19
+
20
+ ## Deploying the Backend Space
21
+
22
+ 1. Create a new Space on [Hugging Face](https://huggingface.co/spaces).
23
+ 2. Set the SDK to **Docker** (Blank template).
24
+ 3. **Set up Persistent Storage / Dataset Mount**:
25
+ * Mount a dataset repository at `/data` inside your Space configuration settings.
26
+ 4. **Set environment variable**:
27
+ * In your Space's settings, add a new Secret variable: `HF_SAVE_API_KEY` (e.g. `my-super-secret-key-123`). This will authenticate client-side uploads.
28
+ 5. Clone your space repository locally, copy the repository files (`main.go`, `cli/`, `Dockerfile`, `go.mod`), commit and push to the Space repo. Hugging Face will automatically compile the server and build the CLI binaries for Linux, macOS, and Windows.
29
+
30
+ ---
31
+
32
+ ## Client Installation
33
+
34
+ To install the client binary on any GPU instance or development machine, run the appropriate command. Replace `your-space-name.hf.space` with your actual Hugging Face Space hostname.
35
+
36
+ ### Linux:
37
+ ```bash
38
+ curl -fsSL "https://your-space-name.hf.space/init?platform=linux&token=YOUR_API_KEY" | bash
39
+ ```
40
+
41
+ ### Windows (PowerShell):
42
+ ```powershell
43
+ irm "https://your-space-name.hf.space/init?platform=windows&token=YOUR_API_KEY" | iex
44
+ ```
45
+
46
+ ### macOS:
47
+ ```bash
48
+ curl -fsSL "https://your-space-name.hf.space/init?platform=mac&token=YOUR_API_KEY" | bash
49
+ ```
50
+
51
+ *Note: If you omit the `&token=YOUR_API_KEY` parameter from the URL, the installer will interactively prompt you to enter the API Key during setup.*
52
+
53
+ ---
54
+
55
+ ## CLI Usage Guide
56
+
57
+ Once installed, use the CLI commands to backup and restore files.
58
+
59
+ ### 1. Saving Directories
60
+ To backup folders at the end of a session:
61
+ ```bash
62
+ # Save a single directory
63
+ hf-save outputs
64
+
65
+ # Save multiple directories
66
+ hf-save outputs logs checkpoints
67
+ ```
68
+ This scans files, filters out binary/bloat directories (like `.git`, `__pycache__`, `venv`), and streams them directly to `/data/{YYYY-MM-DD}/{HH-MM-SS}_{name}` on your backend.
69
+
70
+ ### 2. Listing Backups
71
+ List all backup dates:
72
+ ```bash
73
+ hf-list
74
+ ```
75
+
76
+ List individual snapshots for a specific date:
77
+ ```bash
78
+ hf-list 2026-06-15
79
+ ```
80
+
81
+ ### 3. Restoring Backups
82
+ To restore the **latest** backup of a specific folder name automatically:
83
+ ```bash
84
+ hf-mount outputs
85
+ ```
86
+
87
+ To restore a **specific** timestamped backup run:
88
+ ```bash
89
+ hf-mount 2026-06-15 16-35-20_outputs
90
+ ```
91
+ This downloads the files and reconstructs the folders inside your current working directory.
cli/main.go ADDED
@@ -0,0 +1,550 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package main
2
+
3
+ import (
4
+ "bytes"
5
+ "encoding/json"
6
+ "fmt"
7
+ "io"
8
+ "mime/multipart"
9
+ "net/http"
10
+ "os"
11
+ "path/filepath"
12
+ "strings"
13
+ "time"
14
+ )
15
+
16
+
17
+ type Config struct {
18
+ ServerURL string `json:"server_url"`
19
+ APIKey string `json:"api_key"`
20
+ }
21
+
22
+ func main() {
23
+ if len(os.Args) < 2 {
24
+ printUsageAndExit()
25
+ }
26
+
27
+ command := os.Args[1]
28
+
29
+ // Help check
30
+ if command == "-h" || command == "--help" || command == "help" {
31
+ printUsageAndExit()
32
+ }
33
+
34
+ switch command {
35
+ case "save":
36
+ // User can run either: hf-save <dirs>... OR hf-save save <dirs>...
37
+ // We'll support both, but the primary expectation is: hf-save outputs logs
38
+ if len(os.Args) < 3 {
39
+ fmt.Println("Error: No directories specified to save.")
40
+ os.Exit(1)
41
+ }
42
+ runSave(os.Args[2:])
43
+ case "list":
44
+ // user runs hf-list or hf-save list
45
+ date := ""
46
+ if len(os.Args) > 2 {
47
+ date = os.Args[2]
48
+ }
49
+ runList(date)
50
+ case "mount":
51
+ // hf-mount outputs OR hf-mount 2026-06-15 16-35-20_outputs
52
+ if len(os.Args) < 3 {
53
+ fmt.Println("Error: Please specify what to restore.")
54
+ os.Exit(1)
55
+ }
56
+ runMount(os.Args[2:])
57
+ default:
58
+ // If command is not recognized, treat all arguments as directories to save
59
+ // This enables direct usage like: hf-save outputs logs
60
+ runSave(os.Args[1:])
61
+ }
62
+ }
63
+
64
+ func printUsageAndExit() {
65
+ fmt.Println(`hf-save: Extreme fast backup/restore tool for ephemeral GPU instances
66
+
67
+ Usage:
68
+ hf-save <dir1> [dir2]... Save directories to the backup server
69
+ hf-list List available backup dates
70
+ hf-list <date> List all backups on a specific date (e.g. hf-list 2026-06-15)
71
+ hf-mount <dir_name> Restore latest backup containing directory name
72
+ hf-mount <date> <run> Restore specific backup run (e.g. hf-mount 2026-06-15 16-35-20_outputs)
73
+
74
+ Aliases / Subcommands:
75
+ hf-save save <dir>...
76
+ hf-save list [date]
77
+ hf-save mount <args>...
78
+ `)
79
+ os.Exit(0)
80
+ }
81
+
82
+ func loadConfig() Config {
83
+ home, err := os.UserHomeDir()
84
+ if err != nil {
85
+ fmt.Printf("Error determining user home directory: %v\n", err)
86
+ os.Exit(1)
87
+ }
88
+
89
+ configPath := filepath.Join(home, ".config", "hf-save", "config.json")
90
+ file, err := os.Open(configPath)
91
+ if err != nil {
92
+ fmt.Printf("Error: Configuration file not found at %s. Please run the init script first.\n", configPath)
93
+ os.Exit(1)
94
+ }
95
+ defer file.Close()
96
+
97
+ var cfg Config
98
+ if err := json.NewDecoder(file).Decode(&cfg); err != nil {
99
+ fmt.Printf("Error parsing configuration: %v\n", err)
100
+ os.Exit(1)
101
+ }
102
+
103
+ cfg.ServerURL = strings.TrimSuffix(cfg.ServerURL, "/")
104
+ return cfg
105
+ }
106
+
107
+ type uploadFile struct {
108
+ relPath string
109
+ absPath string
110
+ }
111
+
112
+ func ensureServerAwake(serverURL string) {
113
+ client := &http.Client{Timeout: 5 * time.Second}
114
+ fmt.Print("Checking connection to backup server... ")
115
+
116
+ for i := 0; i < 30; i++ {
117
+ resp, err := client.Get(serverURL + "/health")
118
+ if err == nil {
119
+ if resp.StatusCode == http.StatusOK {
120
+ resp.Body.Close()
121
+ fmt.Println("Connected.")
122
+ return
123
+ }
124
+ resp.Body.Close()
125
+ }
126
+
127
+ if i == 0 {
128
+ fmt.Println("\nServer is asleep or starting up. Waking it up (this may take 1-2 minutes)...")
129
+ }
130
+
131
+ fmt.Printf("Waiting for server to wake up... (Attempt %d/30)\r", i+1)
132
+ time.Sleep(5 * time.Second)
133
+ }
134
+
135
+ fmt.Println("\nWarning: Could not verify server is awake. Attempting operation anyway...")
136
+ }
137
+
138
+ type progressReader struct {
139
+ r io.Reader
140
+ total int64
141
+ current int64
142
+ }
143
+
144
+ func (pr *progressReader) Read(p []byte) (n int, err error) {
145
+ n, err = pr.r.Read(p)
146
+ if n > 0 {
147
+ pr.current += int64(n)
148
+ percent := float64(pr.current) / float64(pr.total) * 100
149
+ mbCurrent := float64(pr.current) / (1024 * 1024)
150
+ mbTotal := float64(pr.total) / (1024 * 1024)
151
+ fmt.Printf("Uploading: %.1f%% (%.1f/%.1f MB)\r", percent, mbCurrent, mbTotal)
152
+ }
153
+ return
154
+ }
155
+
156
+ type writeCounter struct {
157
+ total int64
158
+ current int64
159
+ name string
160
+ }
161
+
162
+ func (wc *writeCounter) Write(p []byte) (int, error) {
163
+ n := len(p)
164
+ wc.current += int64(n)
165
+ percent := float64(wc.current) / float64(wc.total) * 100
166
+ mbCurrent := float64(wc.current) / (1024 * 1024)
167
+ mbTotal := float64(wc.total) / (1024 * 1024)
168
+ fmt.Printf("Downloading %s: %.1f%% (%.1f/%.1f MB)\r", wc.name, percent, mbCurrent, mbTotal)
169
+ return n, nil
170
+ }
171
+
172
+ var quoteEscaper = strings.NewReplacer("\\", "\\\\", "\"", "\\\"")
173
+
174
+ func escapeQuotes(s string) string {
175
+ return quoteEscaper.Replace(s)
176
+ }
177
+
178
+ func calculateUploadSize(prefix string, files []uploadFile, boundary string) (int64, error) {
179
+ var totalSize int64
180
+
181
+ // 1. Prefix field size
182
+ var prefixBuf bytes.Buffer
183
+ w := multipart.NewWriter(&prefixBuf)
184
+ w.SetBoundary(boundary)
185
+ _ = w.WriteField("prefix", prefix)
186
+ totalSize += int64(prefixBuf.Len())
187
+
188
+ // 2. Files size
189
+ for _, file := range files {
190
+ var fileHeaderBuf bytes.Buffer
191
+ fw := multipart.NewWriter(&fileHeaderBuf)
192
+ fw.SetBoundary(boundary)
193
+ _, err := fw.CreateFormFile("files", file.relPath)
194
+ if err != nil {
195
+ return 0, err
196
+ }
197
+ totalSize += int64(fileHeaderBuf.Len())
198
+
199
+ info, err := os.Stat(file.absPath)
200
+ if err != nil {
201
+ return 0, err
202
+ }
203
+ totalSize += info.Size()
204
+ totalSize += 2 // trailing \r\n after file part data
205
+ }
206
+
207
+ // 3. Closing boundary size (\r\n--boundary--\r\n)
208
+ totalSize += int64(len("\r\n--" + boundary + "--\r\n"))
209
+
210
+ return totalSize, nil
211
+ }
212
+
213
+ // bytes helper import for size calculation
214
+ type bytesBufferWriter struct {
215
+ bytes.Buffer
216
+ }
217
+
218
+ func runSave(targets []string) {
219
+ cfg := loadConfig()
220
+ ensureServerAwake(cfg.ServerURL)
221
+
222
+ for _, target := range targets {
223
+ saveSingleTarget(cfg, target)
224
+ }
225
+ }
226
+
227
+ func saveSingleTarget(cfg Config, target string) {
228
+ cleanDir := filepath.Clean(target)
229
+ info, err := os.Stat(cleanDir)
230
+ if err != nil {
231
+ fmt.Printf("Error: Target '%s' not found, skipping.\n", target)
232
+ return
233
+ }
234
+
235
+ var filesToUpload []uploadFile
236
+
237
+ if !info.IsDir() {
238
+ filesToUpload = append(filesToUpload, uploadFile{
239
+ relPath: filepath.Base(cleanDir),
240
+ absPath: cleanDir,
241
+ })
242
+ } else {
243
+ err = filepath.Walk(cleanDir, func(path string, fileInfo os.FileInfo, walkErr error) error {
244
+ if walkErr != nil {
245
+ return walkErr
246
+ }
247
+ if fileInfo.IsDir() {
248
+ return nil
249
+ }
250
+
251
+ parts := strings.Split(path, string(filepath.Separator))
252
+ for _, part := range parts {
253
+ if part == ".git" || part == "node_modules" || part == "__pycache__" || part == ".venv" || part == "venv" {
254
+ return nil
255
+ }
256
+ }
257
+
258
+ rel, err := filepath.Rel(filepath.Dir(cleanDir), path)
259
+ if err == nil {
260
+ filesToUpload = append(filesToUpload, uploadFile{
261
+ relPath: filepath.ToSlash(rel),
262
+ absPath: path,
263
+ })
264
+ }
265
+ return nil
266
+ })
267
+ if err != nil {
268
+ fmt.Printf("Error scanning target %s: %v\n", target, err)
269
+ return
270
+ }
271
+ }
272
+
273
+ if len(filesToUpload) == 0 {
274
+ fmt.Printf("No files found to save in target %s.\n", target)
275
+ return
276
+ }
277
+
278
+ dateStr := time.Now().Format("2006-01-02")
279
+ timeStr := time.Now().Format("15-04-05")
280
+ folderSuffix := filepath.Base(cleanDir)
281
+ prefix := fmt.Sprintf("%s/%s_%s", dateStr, timeStr, folderSuffix)
282
+
283
+ fmt.Printf("\n--- Saving target: %s (%d files) ---\n", target, len(filesToUpload))
284
+ fmt.Printf("Uploading snapshot: %s\n", prefix)
285
+
286
+ // Create a dummy multipart writer to generate a consistent boundary string
287
+ dummyWriter := multipart.NewWriter(nil)
288
+ boundary := dummyWriter.Boundary()
289
+
290
+ totalSize, err := calculateUploadSize(prefix, filesToUpload, boundary)
291
+ if err != nil {
292
+ fmt.Printf("Failed to calculate upload size: %v\n", err)
293
+ return
294
+ }
295
+
296
+ pipeReader, pipeWriter := io.Pipe()
297
+ writer := multipart.NewWriter(pipeWriter)
298
+ writer.SetBoundary(boundary)
299
+
300
+ go func() {
301
+ defer pipeWriter.Close()
302
+ defer writer.Close()
303
+
304
+ err := writer.WriteField("prefix", prefix)
305
+ if err != nil {
306
+ return
307
+ }
308
+
309
+ for _, file := range filesToUpload {
310
+ fileWriter, err := writer.CreateFormFile("files", file.relPath)
311
+ if err != nil {
312
+ return
313
+ }
314
+
315
+ f, err := os.Open(file.absPath)
316
+ if err != nil {
317
+ return
318
+ }
319
+
320
+ _, err = io.Copy(fileWriter, f)
321
+ f.Close()
322
+ if err != nil {
323
+ return
324
+ }
325
+ }
326
+ }()
327
+
328
+ reqURL := fmt.Sprintf("%s/upload", cfg.ServerURL)
329
+ pr := &progressReader{
330
+ r: pipeReader,
331
+ total: totalSize,
332
+ }
333
+
334
+ req, err := http.NewRequest(http.MethodPost, reqURL, pr)
335
+ if err != nil {
336
+ fmt.Printf("Error creating upload request: %v\n", err)
337
+ return
338
+ }
339
+
340
+ req.Header.Set("Content-Type", writer.FormDataContentType())
341
+ req.ContentLength = totalSize
342
+ if cfg.APIKey != "" {
343
+ req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", cfg.APIKey))
344
+ }
345
+
346
+ client := &http.Client{}
347
+ resp, err := client.Do(req)
348
+ if err != nil {
349
+ fmt.Printf("\nUpload failed: %v\n", err)
350
+ return
351
+ }
352
+ defer resp.Body.Close()
353
+ fmt.Println() // print newline after progress percentage
354
+
355
+ if resp.StatusCode != http.StatusOK {
356
+ body, _ := io.ReadAll(resp.Body)
357
+ fmt.Printf("Upload failed with status %s: %s\n", resp.Status, string(body))
358
+ return
359
+ }
360
+
361
+ var result map[string]interface{}
362
+ if err := json.NewDecoder(resp.Body).Decode(&result); err == nil {
363
+ fmt.Printf("Successfully saved %s! (%v files)\n", target, result["file_count"])
364
+ } else {
365
+ fmt.Printf("Successfully saved %s!\n", target)
366
+ }
367
+ }
368
+
369
+ func runList(date string) {
370
+ cfg := loadConfig()
371
+ ensureServerAwake(cfg.ServerURL)
372
+
373
+ reqURL := fmt.Sprintf("%s/list", cfg.ServerURL)
374
+ if date != "" {
375
+ reqURL += "?date=" + date
376
+ }
377
+
378
+ resp, err := http.Get(reqURL)
379
+ if err != nil {
380
+ fmt.Printf("Request failed: %v\n", err)
381
+ os.Exit(1)
382
+ }
383
+ defer resp.Body.Close()
384
+
385
+ if resp.StatusCode != http.StatusOK {
386
+ fmt.Printf("Request failed: %s\n", resp.Status)
387
+ os.Exit(1)
388
+ }
389
+
390
+ var results []string
391
+ if err := json.NewDecoder(resp.Body).Decode(&results); err != nil {
392
+ fmt.Printf("Failed to decode response: %v\n", err)
393
+ os.Exit(1)
394
+ }
395
+
396
+ if len(results) == 0 {
397
+ fmt.Println("No backups found.")
398
+ return
399
+ }
400
+
401
+ for _, item := range results {
402
+ fmt.Println(item)
403
+ }
404
+ }
405
+
406
+ func runMount(args []string) {
407
+ cfg := loadConfig()
408
+ ensureServerAwake(cfg.ServerURL)
409
+
410
+ var snapshot string
411
+
412
+ if len(args) == 1 {
413
+ folderName := args[0]
414
+ dates, err := fetchList(cfg.ServerURL, "")
415
+ if err != nil {
416
+ fmt.Printf("Error: Failed to fetch date indexes: %v\n", err)
417
+ os.Exit(1)
418
+ }
419
+
420
+ found := false
421
+ for i := len(dates) - 1; i >= 0; i-- {
422
+ runs, err := fetchList(cfg.ServerURL, dates[i])
423
+ if err != nil {
424
+ continue
425
+ }
426
+
427
+ for j := len(runs) - 1; j >= 0; j-- {
428
+ if strings.HasSuffix(runs[j], "_"+folderName) {
429
+ snapshot = dates[i] + "/" + runs[j]
430
+ found = true
431
+ break
432
+ }
433
+ }
434
+ if found {
435
+ break
436
+ }
437
+ }
438
+
439
+ if !found {
440
+ fmt.Printf("Error: No backups found matching folder name: %s\n", folderName)
441
+ os.Exit(1)
442
+ }
443
+
444
+ } else if len(args) == 2 {
445
+ snapshot = args[0] + "/" + args[1]
446
+ } else {
447
+ fmt.Println("Error: Invalid arguments for mount command.")
448
+ os.Exit(1)
449
+ }
450
+
451
+ fmt.Printf("Mounting snapshot: %s\n", snapshot)
452
+
453
+ manifestURL := fmt.Sprintf("%s/download?snapshot=%s", cfg.ServerURL, snapshot)
454
+ resp, err := http.Get(manifestURL)
455
+ if err != nil {
456
+ fmt.Printf("Failed to download snapshot manifest: %v\n", err)
457
+ os.Exit(1)
458
+ }
459
+ defer resp.Body.Close()
460
+
461
+ if resp.StatusCode != http.StatusOK {
462
+ body, _ := io.ReadAll(resp.Body)
463
+ fmt.Printf("Failed to fetch manifest: %s - %s\n", resp.Status, string(body))
464
+ os.Exit(1)
465
+ }
466
+
467
+ var files []string
468
+ if err := json.NewDecoder(resp.Body).Decode(&files); err != nil {
469
+ fmt.Printf("Failed to decode snapshot list: %v\n", err)
470
+ os.Exit(1)
471
+ }
472
+
473
+ if len(files) == 0 {
474
+ fmt.Println("Snapshot is empty.")
475
+ return
476
+ }
477
+
478
+ fmt.Printf("Restoring %d files...\n", len(files))
479
+
480
+ for _, file := range files {
481
+ targetLocalPath := filepath.FromSlash(file)
482
+ localDir := filepath.Dir(targetLocalPath)
483
+ if localDir != "." {
484
+ if err := os.MkdirAll(localDir, 0755); err != nil {
485
+ fmt.Printf("Failed to create folder %s: %v\n", localDir, err)
486
+ os.Exit(1)
487
+ }
488
+ }
489
+
490
+ fileURL := fmt.Sprintf("%s/download?file=%s/%s", cfg.ServerURL, snapshot, file)
491
+ err = downloadFile(fileURL, targetLocalPath, filepath.Base(targetLocalPath))
492
+ if err != nil {
493
+ fmt.Printf("Error downloading file %s: %v\n", file, err)
494
+ os.Exit(1)
495
+ }
496
+ }
497
+
498
+ fmt.Println("Restore completed successfully!")
499
+ }
500
+
501
+ func fetchList(serverURL, date string) ([]string, error) {
502
+ reqURL := fmt.Sprintf("%s/list", serverURL)
503
+ if date != "" {
504
+ reqURL += "?date=" + date
505
+ }
506
+
507
+ resp, err := http.Get(reqURL)
508
+ if err != nil {
509
+ return nil, err
510
+ }
511
+ defer resp.Body.Close()
512
+
513
+ if resp.StatusCode != http.StatusOK {
514
+ return nil, fmt.Errorf("unexpected status %s", resp.Status)
515
+ }
516
+
517
+ var res []string
518
+ err = json.NewDecoder(resp.Body).Decode(&res)
519
+ return res, err
520
+ }
521
+
522
+ func downloadFile(url, dest string, fileName string) error {
523
+ resp, err := http.Get(url)
524
+ if err != nil {
525
+ return err
526
+ }
527
+ defer resp.Body.Close()
528
+
529
+ if resp.StatusCode != http.StatusOK {
530
+ body, _ := io.ReadAll(resp.Body)
531
+ return fmt.Errorf("server returned %s - %s", resp.Status, string(body))
532
+ }
533
+
534
+ out, err := os.Create(dest)
535
+ if err != nil {
536
+ return err
537
+ }
538
+ defer out.Close()
539
+
540
+ if resp.ContentLength > 5*1024*1024 {
541
+ counter := &writeCounter{total: resp.ContentLength, name: fileName}
542
+ _, err = io.Copy(out, io.TeeReader(resp.Body, counter))
543
+ fmt.Println()
544
+ } else {
545
+ fmt.Printf("Downloading %s...\n", fileName)
546
+ _, err = io.Copy(out, resp.Body)
547
+ }
548
+ return err
549
+ }
550
+
go.mod ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ module hf-save
2
+
3
+ go 1.26.4
main.go ADDED
@@ -0,0 +1,464 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package main
2
+
3
+ import (
4
+ "encoding/json"
5
+ "fmt"
6
+ "io"
7
+ "log"
8
+ "net/http"
9
+ "os"
10
+ "path/filepath"
11
+ "strings"
12
+ "time"
13
+ )
14
+
15
+ var (
16
+ dataDir = "/data"
17
+ binDir = "/app/bin"
18
+ apiKey = ""
19
+ port = "7860" // Hugging Face Spaces default port
20
+ )
21
+
22
+ func init() {
23
+ if val, ok := os.LookupEnv("DATA_DIR"); ok {
24
+ dataDir = val
25
+ }
26
+ if val, ok := os.LookupEnv("BIN_DIR"); ok {
27
+ binDir = val
28
+ }
29
+ if val, ok := os.LookupEnv("HF_SAVE_API_KEY"); ok {
30
+ apiKey = val
31
+ } else {
32
+ log.Println("WARNING: HF_SAVE_API_KEY environment variable is not set. Write access will not be authenticated.")
33
+ }
34
+ if val, ok := os.LookupEnv("PORT"); ok {
35
+ port = val
36
+ }
37
+
38
+ // Ensure data directory exists
39
+ if err := os.MkdirAll(dataDir, 0755); err != nil {
40
+ log.Fatalf("Failed to create data directory %s: %v", dataDir, err)
41
+ }
42
+ }
43
+
44
+ func main() {
45
+ // API key middleware for write operations
46
+ authRequired := func(next http.HandlerFunc) http.HandlerFunc {
47
+ return func(w http.ResponseWriter, r *http.Request) {
48
+ if apiKey == "" {
49
+ next(w, r)
50
+ return
51
+ }
52
+ authHeader := r.Header.Get("Authorization")
53
+ if !strings.HasPrefix(authHeader, "Bearer ") {
54
+ http.Error(w, "Unauthorized: Missing or invalid Authorization header", http.StatusUnauthorized)
55
+ return
56
+ }
57
+ token := strings.TrimPrefix(authHeader, "Bearer ")
58
+ if token != apiKey {
59
+ http.Error(w, "Unauthorized: Invalid API key", http.StatusUnauthorized)
60
+ return
61
+ }
62
+ next(w, r)
63
+ }
64
+ }
65
+
66
+ http.HandleFunc("/init", handleInit)
67
+ http.HandleFunc("/bin/", handleServeBinaries)
68
+ http.HandleFunc("/upload", authRequired(handleUpload))
69
+ http.HandleFunc("/list", handleList)
70
+ http.HandleFunc("/download", handleDownload)
71
+ http.HandleFunc("/health", handleHealth)
72
+
73
+ log.Printf("Server starting on port %s...", port)
74
+ log.Printf("Data directory: %s", dataDir)
75
+ log.Printf("Binaries directory: %s", binDir)
76
+ if err := http.ListenAndServe(":"+port, nil); err != nil {
77
+ log.Fatalf("Server failed: %v", err)
78
+ }
79
+ }
80
+
81
+ func handleHealth(w http.ResponseWriter, r *http.Request) {
82
+ w.WriteHeader(http.StatusOK)
83
+ w.Write([]byte("OK"))
84
+ }
85
+
86
+ // Serves the install script for various platforms
87
+ func handleInit(w http.ResponseWriter, r *http.Request) {
88
+ platform := r.URL.Query().Get("platform")
89
+ host := r.Host
90
+ scheme := "https"
91
+ if r.TLS == nil && !strings.Contains(host, "hf.space") {
92
+ scheme = "http"
93
+ }
94
+ serverURL := fmt.Sprintf("%s://%s", scheme, host)
95
+
96
+ // If token isn't passed, CLI will prompt the user to input the pre-shared key during setup
97
+ token := r.URL.Query().Get("token")
98
+
99
+ w.Header().Set("Content-Type", "text/plain")
100
+
101
+ switch strings.ToLower(platform) {
102
+ case "windows":
103
+ // PowerShell script
104
+ script := fmt.Sprintf(`$ServerUrl = "%s"
105
+ $Token = "%s"
106
+ $BinDir = "$HOME\.local\bin"
107
+ if (!(Test-Path $BinDir)) { New-Item -ItemType Directory -Force -Path $BinDir }
108
+ $ExePath = "$BinDir\hf-save.exe"
109
+ Write-Host "Downloading hf-save CLI..." -ForegroundColor Cyan
110
+ Invoke-WebRequest -Uri "$ServerUrl/bin/windows-amd64" -OutFile $ExePath
111
+ if (!(($env:Path -split ';') -contains $BinDir)) {
112
+ [System.Environment]::SetEnvironmentVariable("Path", $env:Path + ";$BinDir", "User")
113
+ $env:Path += ";$BinDir"
114
+ Write-Host "Added $BinDir to PATH. You may need to restart your terminal." -ForegroundColor Yellow
115
+ }
116
+ $ConfigDir = "$HOME\.config\hf-save"
117
+ if (!(Test-Path $ConfigDir)) { New-Item -ItemType Directory -Force -Path $ConfigDir }
118
+ if ($Token -eq "") {
119
+ $Token = Read-Host "Enter your HF_SAVE_API_KEY (Pre-shared API Key)"
120
+ }
121
+ $Config = @{
122
+ server_url = $ServerUrl
123
+ api_key = $Token
124
+ } | ConvertTo-Json
125
+ $Config | Out-File -FilePath "$ConfigDir\config.json" -Encoding utf8
126
+ Write-Host "Installation successful! Try running: hf-save --help" -ForegroundColor Green
127
+ `, serverURL, token)
128
+ w.Write([]byte(script))
129
+
130
+ case "mac", "darwin":
131
+ // Bash/Zsh script for macOS
132
+ script := fmt.Sprintf(`#!/bin/bash
133
+ set -e
134
+ SERVER_URL="%s"
135
+ TOKEN="%s"
136
+ BIN_DIR="$HOME/.local/bin"
137
+ mkdir -p "$BIN_DIR"
138
+ # Determine CPU architecture
139
+ ARCH="amd64"
140
+ if [[ "$(uname -m)" == "arm64" ]]; then
141
+ ARCH="arm64"
142
+ fi
143
+ echo "Downloading hf-save CLI for macOS ($ARCH)..."
144
+ curl -fsSL "$SERVER_URL/bin/darwin-$ARCH" -o "$BIN_DIR/hf-save"
145
+ chmod +x "$BIN_DIR/hf-save"
146
+
147
+ # Add to path helper
148
+ if [[ ":$PATH:" != *":$BIN_DIR:"* ]]; then
149
+ echo "Adding $BIN_DIR to PATH in shell profile..."
150
+ if [[ "$SHELL" == */zsh ]]; then
151
+ echo 'export PATH="$HOME/.local/bin:$PATH"' >> "$HOME/.zshrc"
152
+ else
153
+ echo 'export PATH="$HOME/.local/bin:$PATH"' >> "$HOME/.bashrc"
154
+ fi
155
+ export PATH="$BIN_DIR:$PATH"
156
+ fi
157
+
158
+ CONFIG_DIR="$HOME/.config/hf-save"
159
+ mkdir -p "$CONFIG_DIR"
160
+ if [ -z "$TOKEN" ]; then
161
+ read -p "Enter your HF_SAVE_API_KEY (Pre-shared API Key): " TOKEN
162
+ fi
163
+ cat << EOF > "$CONFIG_DIR/config.json"
164
+ {
165
+ "server_url": "$SERVER_URL",
166
+ "api_key": "$TOKEN"
167
+ }
168
+ EOF
169
+ echo "Installation successful! Try running: hf-save --help (Or reload your shell profile)"
170
+ `, serverURL, token)
171
+ w.Write([]byte(script))
172
+
173
+ default: // Default to linux
174
+ script := fmt.Sprintf(`#!/bin/bash
175
+ set -e
176
+ SERVER_URL="%s"
177
+ TOKEN="%s"
178
+ BIN_DIR="$HOME/.local/bin"
179
+ mkdir -p "$BIN_DIR"
180
+ # Determine CPU architecture
181
+ ARCH="amd64"
182
+ if [[ "$(uname -m)" == "aarch64" ]]; then
183
+ ARCH="arm64"
184
+ fi
185
+ echo "Downloading hf-save CLI for Linux ($ARCH)..."
186
+ curl -fsSL "$SERVER_URL/bin/linux-$ARCH" -o "$BIN_DIR/hf-save"
187
+ chmod +x "$BIN_DIR/hf-save"
188
+
189
+ # Add to path helper
190
+ if [[ ":$PATH:" != *":$BIN_DIR:"* ]]; then
191
+ echo "Adding $BIN_DIR to PATH in shell profile..."
192
+ echo 'export PATH="$HOME/.local/bin:$PATH"' >> "$HOME/.bashrc"
193
+ export PATH="$BIN_DIR:$PATH"
194
+ fi
195
+
196
+ CONFIG_DIR="$HOME/.config/hf-save"
197
+ mkdir -p "$CONFIG_DIR"
198
+ if [ -z "$TOKEN" ]; then
199
+ read -p "Enter your HF_SAVE_API_KEY (Pre-shared API Key): " TOKEN
200
+ fi
201
+ cat << EOF > "$CONFIG_DIR/config.json"
202
+ {
203
+ "server_url": "$SERVER_URL",
204
+ "api_key": "$TOKEN"
205
+ }
206
+ EOF
207
+ echo "Installation successful! Try running: hf-save --help (Or run: export PATH=\$HOME/.local/bin:\$PATH)"
208
+ `, serverURL, token)
209
+ w.Write([]byte(script))
210
+ }
211
+ }
212
+
213
+ // Serves compiled CLI binaries
214
+ func handleServeBinaries(w http.ResponseWriter, r *http.Request) {
215
+ filename := strings.TrimPrefix(r.URL.Path, "/bin/")
216
+ if filename == "" {
217
+ http.Error(w, "Not Found", http.StatusNotFound)
218
+ return
219
+ }
220
+
221
+ // Sanitize filename
222
+ filename = filepath.Base(filename)
223
+ binaryPath := filepath.Join(binDir, filename)
224
+
225
+ if _, err := os.Stat(binaryPath); os.IsNotExist(err) {
226
+ // Fallback for names containing extensions like .exe
227
+ if strings.HasSuffix(filename, ".exe") {
228
+ binaryPath = filepath.Join(binDir, "windows-amd64")
229
+ } else if strings.Contains(filename, "linux") {
230
+ binaryPath = filepath.Join(binDir, "linux-amd64")
231
+ } else if strings.Contains(filename, "mac") || strings.Contains(filename, "darwin") {
232
+ binaryPath = filepath.Join(binDir, "darwin-amd64")
233
+ }
234
+ }
235
+
236
+ if _, err := os.Stat(binaryPath); os.IsNotExist(err) {
237
+ http.Error(w, "Binary not found on server", http.StatusNotFound)
238
+ return
239
+ }
240
+
241
+ w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%s", filename))
242
+ w.Header().Set("Content-Type", "application/octet-stream")
243
+ http.ServeFile(w, r, binaryPath)
244
+ }
245
+
246
+ // Handles multipart file uploads (Streams files straight to directory structure)
247
+ func handleUpload(w http.ResponseWriter, r *http.Request) {
248
+ if r.Method != http.MethodPost {
249
+ http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
250
+ return
251
+ }
252
+
253
+ mr, err := r.MultipartReader()
254
+ if err != nil {
255
+ http.Error(w, fmt.Sprintf("Failed to read multipart data: %v", err), http.StatusBadRequest)
256
+ return
257
+ }
258
+
259
+ var currentPrefix string
260
+ fileCount := 0
261
+
262
+ for {
263
+ part, err := mr.NextPart()
264
+ if err == io.EOF {
265
+ break
266
+ }
267
+ if err != nil {
268
+ http.Error(w, fmt.Sprintf("Error parsing part: %v", err), http.StatusInternalServerError)
269
+ return
270
+ }
271
+
272
+ formName := part.FormName()
273
+
274
+ if formName == "prefix" {
275
+ prefixVal, err := io.ReadAll(part)
276
+ if err != nil {
277
+ http.Error(w, "Failed to read prefix", http.StatusBadRequest)
278
+ return
279
+ }
280
+ currentPrefix = string(prefixVal)
281
+ part.Close()
282
+ continue
283
+ }
284
+
285
+ if formName == "files" {
286
+ if currentPrefix == "" {
287
+ // Fallback to today's date if prefix wasn't provided first
288
+ currentPrefix = time.Now().Format("2006-01-02/15-04-05_upload")
289
+ }
290
+
291
+ // Clean prefix
292
+ cleanPrefix := filepath.Clean(currentPrefix)
293
+ // Prevent directory traversal attacks
294
+ if strings.Contains(cleanPrefix, "..") || strings.HasPrefix(cleanPrefix, "/") {
295
+ http.Error(w, "Invalid prefix path", http.StatusBadRequest)
296
+ return
297
+ }
298
+
299
+ // The file's relative path is stored in the header's filename
300
+ relPath := part.FileName()
301
+ if relPath == "" {
302
+ part.Close()
303
+ continue
304
+ }
305
+
306
+ // Clean and resolve local target path
307
+ cleanRelPath := filepath.Clean(relPath)
308
+ if strings.Contains(cleanRelPath, "..") || strings.HasPrefix(cleanRelPath, "/") {
309
+ part.Close()
310
+ continue
311
+ }
312
+
313
+ targetPath := filepath.Join(dataDir, cleanPrefix, cleanRelPath)
314
+ targetDir := filepath.Dir(targetPath)
315
+
316
+ if err := os.MkdirAll(targetDir, 0755); err != nil {
317
+ http.Error(w, fmt.Sprintf("Failed to create target directory: %v", err), http.StatusInternalServerError)
318
+ return
319
+ }
320
+
321
+ dst, err := os.Create(targetPath)
322
+ if err != nil {
323
+ http.Error(w, fmt.Sprintf("Failed to create file: %v", err), http.StatusInternalServerError)
324
+ return
325
+ }
326
+
327
+ if _, err := io.Copy(dst, part); err != nil {
328
+ dst.Close()
329
+ http.Error(w, fmt.Sprintf("Failed to save file: %v", err), http.StatusInternalServerError)
330
+ return
331
+ }
332
+ dst.Close()
333
+ part.Close()
334
+ fileCount++
335
+ }
336
+ }
337
+
338
+ w.Header().Set("Content-Type", "application/json")
339
+ json.NewEncoder(w).Encode(map[string]interface{}{
340
+ "success": true,
341
+ "file_count": fileCount,
342
+ "prefix": currentPrefix,
343
+ })
344
+ }
345
+
346
+ // Lists dates or runs within a date
347
+ func handleList(w http.ResponseWriter, r *http.Request) {
348
+ dateParam := r.URL.Query().Get("date")
349
+
350
+ var results []string
351
+
352
+ if dateParam == "" {
353
+ // List top-level date directories
354
+ entries, err := os.ReadDir(dataDir)
355
+ if err != nil {
356
+ http.Error(w, fmt.Sprintf("Failed to read data directory: %v", err), http.StatusInternalServerError)
357
+ return
358
+ }
359
+ for _, entry := range entries {
360
+ if entry.IsDir() {
361
+ // Basic check for YYYY-MM-DD pattern
362
+ name := entry.Name()
363
+ if len(name) == 10 && name[4] == '-' && name[7] == '-' {
364
+ results = append(results, name)
365
+ }
366
+ }
367
+ }
368
+ } else {
369
+ // List upload runs inside the specified date
370
+ cleanDate := filepath.Clean(dateParam)
371
+ if strings.Contains(cleanDate, "..") || strings.HasPrefix(cleanDate, "/") {
372
+ http.Error(w, "Invalid date format", http.StatusBadRequest)
373
+ return
374
+ }
375
+
376
+ targetPath := filepath.Join(dataDir, cleanDate)
377
+ entries, err := os.ReadDir(targetPath)
378
+ if err != nil {
379
+ if os.IsNotExist(err) {
380
+ w.Header().Set("Content-Type", "application/json")
381
+ json.NewEncoder(w).Encode([]string{})
382
+ return
383
+ }
384
+ http.Error(w, fmt.Sprintf("Failed to read date directory: %v", err), http.StatusInternalServerError)
385
+ return
386
+ }
387
+
388
+ for _, entry := range entries {
389
+ if entry.IsDir() {
390
+ results = append(results, entry.Name())
391
+ }
392
+ }
393
+ }
394
+
395
+ w.Header().Set("Content-Type", "application/json")
396
+ json.NewEncoder(w).Encode(results)
397
+ }
398
+
399
+ // Download file stream or metadata list
400
+ func handleDownload(w http.ResponseWriter, r *http.Request) {
401
+ filePath := r.URL.Query().Get("file")
402
+ snapshotPath := r.URL.Query().Get("snapshot")
403
+
404
+ if filePath != "" {
405
+ // Stream a specific file
406
+ cleanPath := filepath.Clean(filePath)
407
+ if strings.Contains(cleanPath, "..") || strings.HasPrefix(cleanPath, "/") {
408
+ http.Error(w, "Invalid file path", http.StatusBadRequest)
409
+ return
410
+ }
411
+
412
+ fullPath := filepath.Join(dataDir, cleanPath)
413
+ if _, err := os.Stat(fullPath); os.IsNotExist(err) {
414
+ http.Error(w, "File not found", http.StatusNotFound)
415
+ return
416
+ }
417
+
418
+ w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%s", filepath.Base(fullPath)))
419
+ w.Header().Set("Content-Type", "application/octet-stream")
420
+ http.ServeFile(w, r, fullPath)
421
+ return
422
+ }
423
+
424
+ if snapshotPath != "" {
425
+ // List all files within a snapshot to download
426
+ cleanSnapshot := filepath.Clean(snapshotPath)
427
+ if strings.Contains(cleanSnapshot, "..") || strings.HasPrefix(cleanSnapshot, "/") {
428
+ http.Error(w, "Invalid snapshot path", http.StatusBadRequest)
429
+ return
430
+ }
431
+
432
+ fullDir := filepath.Join(dataDir, cleanSnapshot)
433
+ var files []string
434
+
435
+ err := filepath.Walk(fullDir, func(path string, info os.FileInfo, err error) error {
436
+ if err != nil {
437
+ return err
438
+ }
439
+ if !info.IsDir() {
440
+ rel, err := filepath.Rel(fullDir, path)
441
+ if err == nil {
442
+ // Convert windows backslashes to slash for web transport compatibility
443
+ files = append(files, filepath.ToSlash(rel))
444
+ }
445
+ }
446
+ return nil
447
+ })
448
+
449
+ if err != nil {
450
+ if os.IsNotExist(err) {
451
+ http.Error(w, "Snapshot not found", http.StatusNotFound)
452
+ return
453
+ }
454
+ http.Error(w, fmt.Sprintf("Failed to scan snapshot: %v", err), http.StatusInternalServerError)
455
+ return
456
+ }
457
+
458
+ w.Header().Set("Content-Type", "application/json")
459
+ json.NewEncoder(w).Encode(files)
460
+ return
461
+ }
462
+
463
+ http.Error(w, "Missing query parameter 'file' or 'snapshot'", http.StatusBadRequest)
464
+ }