Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+trackio/assets/trackio_logo_old.png filter=lfs diff=lfs merge=lfs -text

trackio/CHANGELOG.md

@@ -0,0 +1,187 @@
+# trackio
+
+## 0.20.0
+
+### Features
+
+- [#450](https://github.com/gradio-app/trackio/pull/450) [`b0571ef`](https://github.com/gradio-app/trackio/commit/b0571ef6207a1ce346696f858ad2b7b584dd194f) - Use Svelte source for Gradio components directly in Trackio dashboard.  Thanks @abidlabs!
+
+## 0.19.0
+
+### Features
+
+- [#445](https://github.com/gradio-app/trackio/pull/445) [`cef4a58`](https://github.com/gradio-app/trackio/commit/cef4a583cb76f4091fc6c0e5783124ee84f8e243) - Add remote HF Space support to CLI.  Thanks @abidlabs!
+- [#444](https://github.com/gradio-app/trackio/pull/444) [`358f2a9`](https://github.com/gradio-app/trackio/commit/358f2a9ca238ee8b90b5a8c96220da287e0698fb) - Fix alerts placeholder flashing on reports page.  Thanks @abidlabs!
+
+## 0.18.0
+
+### Features
+
+- [#435](https://github.com/gradio-app/trackio/pull/435) [`4a47112`](https://github.com/gradio-app/trackio/commit/4a471128e18a39e45fad48a67fd711c5ae9e4aed) - feat: allow hiding section header accordions.  Thanks @Saba9!
+- [#439](https://github.com/gradio-app/trackio/pull/439) [`18e9650`](https://github.com/gradio-app/trackio/commit/18e96503d5a3a7cf926e92782d457e23c19942bd) - Add alerts with webhooks, CLI, and documentation.  Thanks @abidlabs!
+- [#438](https://github.com/gradio-app/trackio/pull/438) [`0875ccd`](https://github.com/gradio-app/trackio/commit/0875ccd3d8a41b1376f64030f21cfe8cdcc73b05) - Add "share this view" functionality.  Thanks @qgallouedec!
+- [#409](https://github.com/gradio-app/trackio/pull/409) [`9282403`](https://github.com/gradio-app/trackio/commit/9282403d8896d48679b0f888208a7ba5bdd4271a) - Add Apple Silicon GPU and system monitoring support.  Thanks @znation!
+- [#434](https://github.com/gradio-app/trackio/pull/434) [`4193223`](https://github.com/gradio-app/trackio/commit/41932230a3a2e1c16405dba08ecba5a42f11d1a8) - fix: table slider crash.  Thanks @Saba9!
+
+### Fixes
+
+- [#441](https://github.com/gradio-app/trackio/pull/441) [`3a2d11d`](https://github.com/gradio-app/trackio/commit/3a2d11dab0b4b37c925abc30ef84b0e2910321ee) - preserve x-axis step when toggling run checkboxes.  Thanks @Saba9!
+
+## 0.17.0
+
+### Features
+
+- [#428](https://github.com/gradio-app/trackio/pull/428) [`f7dd1ce`](https://github.com/gradio-app/trackio/commit/f7dd1ce2dc8a1936f9983467fcbcf93bfef01e09) - feat: add ability to rename runs.  Thanks @Saba9!
+- [#437](https://github.com/gradio-app/trackio/pull/437) [`2727c0b`](https://github.com/gradio-app/trackio/commit/2727c0b0755f48f7f186162ea45185c98f6b5516) - Add markdown reports across Trackio.  Thanks @abidlabs!
+- [#427](https://github.com/gradio-app/trackio/pull/427) [`5aeb9ed`](https://github.com/gradio-app/trackio/commit/5aeb9edcfd2068d309d9d64f172dcbcc327be1ab) - Make Trackio logging much more robust.  Thanks @abidlabs!
+
+## 0.16.1
+
+### Features
+
+- [#431](https://github.com/gradio-app/trackio/pull/431) [`c7ce55b`](https://github.com/gradio-app/trackio/commit/c7ce55b14dd5eb0c2165fb15df17dd60721c9325) - Lazy load the UI when trackio is imported.  Thanks @abidlabs!
+
+## 0.16.0
+
+### Features
+
+- [#426](https://github.com/gradio-app/trackio/pull/426) [`ead4dc8`](https://github.com/gradio-app/trackio/commit/ead4dc8e74ee2d8e47d61bca0a7668456acf49be) - Fix redundant double rendering of group checkboxes.  Thanks @abidlabs!
+- [#413](https://github.com/gradio-app/trackio/pull/413) [`39c4750`](https://github.com/gradio-app/trackio/commit/39c4750951d554ba6eb4d58847c6bb444b2891a8) - Check `dist-packages` when checking for source installation.  Thanks @sergiopaniego!
+- [#423](https://github.com/gradio-app/trackio/pull/423) [`2e52ab3`](https://github.com/gradio-app/trackio/commit/2e52ab303e3041718a6a56fbf84d0848aca9ad67) - Fix legend outline visibility issue.  Thanks @Raghunath-Balaji!
+- [#407](https://github.com/gradio-app/trackio/pull/407) [`c8a384d`](https://github.com/gradio-app/trackio/commit/c8a384ddfe5a295cecf862a26178d40e48acb424) - Fix pytests that were failling locally on MacOS.  Thanks @abidlabs!
+- [#405](https://github.com/gradio-app/trackio/pull/405) [`35aae4e`](https://github.com/gradio-app/trackio/commit/35aae4e3aa3e2b2888887528478b9dc6a9808bda) - Add conditional padding for HF Space dashboard when not in iframe.  Thanks @znation!
+
+## 0.15.0
+
+### Features
+
+- [#397](https://github.com/gradio-app/trackio/pull/397) [`6b38ad0`](https://github.com/gradio-app/trackio/commit/6b38ad02e5d73a0df49c4eede7e91331282ece04) - Adds `--host` cli option support.  Thanks @abidlabs!
+- [#396](https://github.com/gradio-app/trackio/pull/396) [`4a4d1ab`](https://github.com/gradio-app/trackio/commit/4a4d1ab85e63d923132a3fa7afa5d90e16431bec) - Fix run selection issue.  Thanks @abidlabs!
+- [#394](https://github.com/gradio-app/trackio/pull/394) [`c47a3a3`](https://github.com/gradio-app/trackio/commit/c47a3a31f8c4b83bce1aa7fc22eeba3d9021ad3d) - Add wandb-compatible API for trackio.  Thanks @abidlabs!
+- [#378](https://github.com/gradio-app/trackio/pull/378) [`b02046a`](https://github.com/gradio-app/trackio/commit/b02046a5b0dad7c9854e099a87f884afba4aecb2) - Add JSON export button for line plots and upgrade gradio dependency.  Thanks @JamshedAli18!
+
+## 0.14.2
+
+### Features
+
+- [#386](https://github.com/gradio-app/trackio/pull/386) [`f9452cd`](https://github.com/gradio-app/trackio/commit/f9452cdb8f0819368f3610f7ac0ed08957305275) - Fixing some issues related to deployed Trackio Spaces.  Thanks @abidlabs!
+
+## 0.14.1
+
+### Features
+
+- [#382](https://github.com/gradio-app/trackio/pull/382) [`44fe9bb`](https://github.com/gradio-app/trackio/commit/44fe9bb264fb2aafb0ec302ff15227c045819a2c) - Fix app file path when Trackio is not installed from source.  Thanks @abidlabs!
+- [#380](https://github.com/gradio-app/trackio/pull/380) [`c3f4cff`](https://github.com/gradio-app/trackio/commit/c3f4cff74bc5676e812773d8571454894fcdc7cc) - Add CLI commands for querying projects, runs, and metrics.  Thanks @abidlabs!
+
+## 0.14.0
+
+### Features
+
+- [#377](https://github.com/gradio-app/trackio/pull/377) [`5c5015b`](https://github.com/gradio-app/trackio/commit/5c5015b68c85c5de51111dad983f735c27b9a05f) - fixed wrapping issue in Runs table.  Thanks @gaganchapa!
+- [#374](https://github.com/gradio-app/trackio/pull/374) [`388e26b`](https://github.com/gradio-app/trackio/commit/388e26b9e9f24cd7ad203affe9b709be885b3d24) - Save Optimized Parquet files.  Thanks @lhoestq!
+- [#371](https://github.com/gradio-app/trackio/pull/371) [`fbace9c`](https://github.com/gradio-app/trackio/commit/fbace9cd7732c166f34d268f54b05bb06846cc5d) - Add GPU metrics logging.  Thanks @kashif!
+- [#367](https://github.com/gradio-app/trackio/pull/367) [`862840c`](https://github.com/gradio-app/trackio/commit/862840c13e30fc960cbee5b9eac4d3c25beba9de) - Add option to only show latest run, and fix the double logo issue.  Thanks @abidlabs!
+
+## 0.13.1
+
+### Features
+
+- [#369](https://github.com/gradio-app/trackio/pull/369) [`767e9fe`](https://github.com/gradio-app/trackio/commit/767e9fe095d7c6ed102016caf927c1517fb8618c) - tiny pr removing unnecessary code.  Thanks @abidlabs!
+
+## 0.13.0
+
+### Features
+
+- [#358](https://github.com/gradio-app/trackio/pull/358) [`073715d`](https://github.com/gradio-app/trackio/commit/073715d1caf8282f68890117f09c3ac301205312) - Improvements to `trackio.sync()`.  Thanks @abidlabs!
+
+## 0.12.0
+
+### Features
+
+- [#357](https://github.com/gradio-app/trackio/pull/357) [`02ba815`](https://github.com/gradio-app/trackio/commit/02ba815358060f1966052de051a5bdb09702920e) - Redesign media and tables to show up on separate page.  Thanks @abidlabs!
+- [#359](https://github.com/gradio-app/trackio/pull/359) [`08fe9c9`](https://github.com/gradio-app/trackio/commit/08fe9c9ddd7fe99ee811555fdfb62df9ab88e939) - docs: Improve docstrings.  Thanks @qgallouedec!
+
+## 0.11.0
+
+### Features
+
+- [#355](https://github.com/gradio-app/trackio/pull/355) [`ea51f49`](https://github.com/gradio-app/trackio/commit/ea51f4954922f21be76ef828700420fe9a912c4b) - Color code run checkboxes and match with plot lines.  Thanks @abidlabs!
+- [#353](https://github.com/gradio-app/trackio/pull/353) [`8abe691`](https://github.com/gradio-app/trackio/commit/8abe6919aeefe21fc7a23af814883efbb037c21f) - Remove show_api from demo.launch.  Thanks @sergiopaniego!
+- [#351](https://github.com/gradio-app/trackio/pull/351) [`8a8957e`](https://github.com/gradio-app/trackio/commit/8a8957e530dd7908d1fef7f2df030303f808101f) - Add `trackio.save()`.  Thanks @abidlabs!
+
+## 0.10.0
+
+### Features
+
+- [#305](https://github.com/gradio-app/trackio/pull/305) [`e64883a`](https://github.com/gradio-app/trackio/commit/e64883a51f7b8b93f7d48b8afe55acdb62238b71) - bump to gradio 6.0, make `trackio` compatible, and fix related issues.  Thanks @abidlabs!
+
+## 0.9.1
+
+### Features
+
+- [#344](https://github.com/gradio-app/trackio/pull/344) [`7e01024`](https://github.com/gradio-app/trackio/commit/7e010241d9a34794e0ce0dc19c1a6f0cf94ba856) - Avoid redundant calls to /whoami-v2.  Thanks @Wauplin!
+
+## 0.9.0
+
+### Features
+
+- [#343](https://github.com/gradio-app/trackio/pull/343) [`51bea30`](https://github.com/gradio-app/trackio/commit/51bea30f2877adff8e6497466d3a799400a0a049) - Sync offline projects to Hugging Face spaces.  Thanks @candemircan!
+- [#341](https://github.com/gradio-app/trackio/pull/341) [`4fd841f`](https://github.com/gradio-app/trackio/commit/4fd841fa190e15071b02f6fba7683ef4f393a654) - Adds a basic UI test to `trackio`.  Thanks @abidlabs!
+- [#339](https://github.com/gradio-app/trackio/pull/339) [`011d91b`](https://github.com/gradio-app/trackio/commit/011d91bb6ae266516fd250a349285670a8049d05) - Allow customzing the trackio color palette.  Thanks @abidlabs!
+
+## 0.8.1
+
+### Features
+
+- [#336](https://github.com/gradio-app/trackio/pull/336) [`5f9f51d`](https://github.com/gradio-app/trackio/commit/5f9f51dac8677f240d7c42c3e3b2660a22aee138) - Support a list of `Trackio.Image` in a `trackio.Table` cell.  Thanks @abidlabs!
+
+## 0.8.0
+
+### Features
+
+- [#331](https://github.com/gradio-app/trackio/pull/331) [`2c02d0f`](https://github.com/gradio-app/trackio/commit/2c02d0fd0a5824160528782402bb0dd4083396d5) - Truncate table string values that are greater than 250 characters (configuirable via env variable).  Thanks @abidlabs!
+- [#324](https://github.com/gradio-app/trackio/pull/324) [`50b2122`](https://github.com/gradio-app/trackio/commit/50b2122e7965ac82a72e6cb3b7d048bc10a2a6b1) - Add log y-axis functionality to UI.  Thanks @abidlabs!
+- [#326](https://github.com/gradio-app/trackio/pull/326) [`61dc1f4`](https://github.com/gradio-app/trackio/commit/61dc1f40af2f545f8e70395ddf0dbb8aee6b60d5) - Fix: improve table rendering for metrics in Trackio Dashboard.  Thanks @vigneshwaran!
+- [#328](https://github.com/gradio-app/trackio/pull/328) [`6857cbb`](https://github.com/gradio-app/trackio/commit/6857cbbe557a59a4642f210ec42566d108294e63) - Support trackio.Table with trackio.Image columns.  Thanks @abidlabs!
+- [#323](https://github.com/gradio-app/trackio/pull/323) [`6857cbb`](https://github.com/gradio-app/trackio/commit/6857cbbe557a59a4642f210ec42566d108294e63) - add Trackio client implementations in Go, Rust, and JS.  Thanks @vaibhav-research!
+
+## 0.7.0
+
+### Features
+
+- [#277](https://github.com/gradio-app/trackio/pull/277) [`db35601`](https://github.com/gradio-app/trackio/commit/db35601b9c023423c4654c9909b8ab73e58737de) - fix: make grouped runs view reflect live updates.  Thanks @Saba9!
+- [#320](https://github.com/gradio-app/trackio/pull/320) [`24ae739`](https://github.com/gradio-app/trackio/commit/24ae73969b09fb3126acd2f91647cdfbf8cf72a1) - Add additional query parms for xmin, xmax, and smoothing.  Thanks @abidlabs!
+- [#270](https://github.com/gradio-app/trackio/pull/270) [`cd1dfc3`](https://github.com/gradio-app/trackio/commit/cd1dfc3dc641b4499ac6d4a1b066fa8e2b52c57b) - feature: add support for logging audio.  Thanks @Saba9!
+
+## 0.6.0
+
+### Features
+
+- [#309](https://github.com/gradio-app/trackio/pull/309) [`1df2353`](https://github.com/gradio-app/trackio/commit/1df23534d6c01938c8db9c0f584ffa23e8d6021d) - Add histogram support with wandb-compatible API.  Thanks @abidlabs!
+- [#315](https://github.com/gradio-app/trackio/pull/315) [`76ba060`](https://github.com/gradio-app/trackio/commit/76ba06055dc43ca8f03b79f3e72d761949bd19a8) - Add guards to avoid silent fails.  Thanks @Xmaster6y!
+- [#313](https://github.com/gradio-app/trackio/pull/313) [`a606b3e`](https://github.com/gradio-app/trackio/commit/a606b3e1c5edf3d4cf9f31bd50605226a5a1c5d0) - No longer prevent certain keys from being used. Instead, dunderify them to prevent collisions with internal usage.  Thanks @abidlabs!
+- [#317](https://github.com/gradio-app/trackio/pull/317) [`27370a5`](https://github.com/gradio-app/trackio/commit/27370a595d0dbdf7eebbe7159d2ba778f039da44) - quick fixes for trackio.histogram.  Thanks @abidlabs!
+- [#312](https://github.com/gradio-app/trackio/pull/312) [`aa0f3bf`](https://github.com/gradio-app/trackio/commit/aa0f3bf372e7a0dd592a38af699c998363830eeb) - Fix video logging by adding TRACKIO_DIR to allowed_paths.  Thanks @abidlabs!
+
+## 0.5.3
+
+### Features
+
+- [#300](https://github.com/gradio-app/trackio/pull/300) [`5e4cacf`](https://github.com/gradio-app/trackio/commit/5e4cacf2e7ce527b4ce60de3a5bc05d2c02c77fb) - Adds more environment variables to allow customization of Trackio dashboard.  Thanks @abidlabs!
+
+## 0.5.2
+
+### Features
+
+- [#293](https://github.com/gradio-app/trackio/pull/293) [`64afc28`](https://github.com/gradio-app/trackio/commit/64afc28d3ea1dfd821472dc6bf0b8ed35a9b74be) - Ensures that the TRACKIO_DIR environment variable is respected.  Thanks @abidlabs!
+- [#287](https://github.com/gradio-app/trackio/pull/287) [`cd3e929`](https://github.com/gradio-app/trackio/commit/cd3e9294320949e6b8b829239069a43d5d7ff4c1) - fix(sqlite): unify .sqlite extension, allow export when DBs exist, clean WAL sidecars on import.  Thanks @vaibhav-research!
+
+### Fixes
+
+- [#291](https://github.com/gradio-app/trackio/pull/291) [`3b5adc3`](https://github.com/gradio-app/trackio/commit/3b5adc3d1f452dbab7a714d235f4974782f93730) - Fix the wheel build.  Thanks @pngwn!
+
+## 0.5.1
+
+### Fixes
+
+- [#278](https://github.com/gradio-app/trackio/pull/278) [`314c054`](https://github.com/gradio-app/trackio/commit/314c05438007ddfea3383e06fd19143e27468e2d) - Fix row orientation of metrics plots.  Thanks @abidlabs!

trackio/__init__.py

@@ -0,0 +1,713 @@
+import atexit
+import glob
+import json
+import logging
+import os
+import shutil
+import warnings
+import webbrowser
+from pathlib import Path
+from typing import Any
+
+import huggingface_hub
+from gradio.themes import ThemeClass
+from gradio.utils import TupleNoPrint
+from gradio_client import Client, handle_file
+from huggingface_hub import SpaceStorage
+from huggingface_hub.errors import LocalTokenNotFoundError
+
+from trackio import context_vars, deploy, utils
+from trackio.alerts import AlertLevel
+from trackio.api import Api
+from trackio.apple_gpu import apple_gpu_available
+from trackio.apple_gpu import log_apple_gpu as _log_apple_gpu
+from trackio.deploy import sync
+from trackio.frontend_server import mount_frontend
+from trackio.gpu import gpu_available
+from trackio.gpu import log_gpu as _log_nvidia_gpu
+from trackio.histogram import Histogram
+from trackio.imports import import_csv, import_tf_events
+from trackio.markdown import Markdown
+from trackio.media import (
+    TrackioAudio,
+    TrackioImage,
+    TrackioVideo,
+    get_project_media_path,
+)
+from trackio.run import Run
+from trackio.server import make_trackio_server
+from trackio.sqlite_storage import SQLiteStorage
+from trackio.table import Table
+from trackio.typehints import UploadEntry
+from trackio.utils import TRACKIO_DIR, TRACKIO_LOGO_DIR
+
+logging.getLogger("httpx").setLevel(logging.WARNING)
+
+warnings.filterwarnings(
+    "ignore",
+    message="Empty session being created. Install gradio\\[oauth\\]",
+    category=UserWarning,
+    module="gradio.helpers",
+)
+
+__version__ = json.loads(Path(__file__).parent.joinpath("package.json").read_text())[
+    "version"
+]
+
+__all__ = [
+    "init",
+    "log",
+    "log_system",
+    "log_gpu",
+    "log_apple_gpu",
+    "finish",
+    "alert",
+    "AlertLevel",
+    "show",
+    "sync",
+    "delete_project",
+    "import_csv",
+    "import_tf_events",
+    "save",
+    "Image",
+    "Video",
+    "Audio",
+    "Table",
+    "Histogram",
+    "Markdown",
+    "Api",
+]
+
+Image = TrackioImage
+Video = TrackioVideo
+Audio = TrackioAudio
+
+
+config = {}
+
+_atexit_registered = False
+
+
+def _cleanup_current_run():
+    run = context_vars.current_run.get()
+    if run is not None:
+        try:
+            run.finish()
+        except Exception:
+            pass
+
+
+def init(
+    project: str,
+    name: str | None = None,
+    group: str | None = None,
+    space_id: str | None = None,
+    space_storage: SpaceStorage | None = None,
+    dataset_id: str | None = None,
+    config: dict | None = None,
+    resume: str = "never",
+    settings: Any = None,
+    private: bool | None = None,
+    embed: bool = True,
+    auto_log_gpu: bool | None = None,
+    gpu_log_interval: float = 10.0,
+    webhook_url: str | None = None,
+    webhook_min_level: AlertLevel | str | None = None,
+) -> Run:
+    """
+    Creates a new Trackio project and returns a [`Run`] object.
+
+    Args:
+        project (`str`):
+            The name of the project (can be an existing project to continue tracking or
+            a new project to start tracking from scratch).
+        name (`str`, *optional*):
+            The name of the run (if not provided, a default name will be generated).
+        group (`str`, *optional*):
+            The name of the group which this run belongs to in order to help organize
+            related runs together. You can toggle the entire group's visibilitiy in the
+            dashboard.
+        space_id (`str`, *optional*):
+            If provided, the project will be logged to a Hugging Face Space instead of
+            a local directory. Should be a complete Space name like
+            `"username/reponame"` or `"orgname/reponame"`, or just `"reponame"` in which
+            case the Space will be created in the currently-logged-in Hugging Face
+            user's namespace. If the Space does not exist, it will be created. If the
+            Space already exists, the project will be logged to it.
+        space_storage ([`~huggingface_hub.SpaceStorage`], *optional*):
+            Choice of persistent storage tier.
+        dataset_id (`str`, *optional*):
+            If a `space_id` is provided, a persistent Hugging Face Dataset will be
+            created and the metrics will be synced to it every 5 minutes. Specify a
+            Dataset with name like `"username/datasetname"` or `"orgname/datasetname"`,
+            or `"datasetname"` (uses currently-logged-in Hugging Face user's namespace),
+            or `None` (uses the same name as the Space but with the `"_dataset"`
+            suffix). If the Dataset does not exist, it will be created. If the Dataset
+            already exists, the project will be appended to it.
+        config (`dict`, *optional*):
+            A dictionary of configuration options. Provided for compatibility with
+            `wandb.init()`.
+        resume (`str`, *optional*, defaults to `"never"`):
+            Controls how to handle resuming a run. Can be one of:
+
+            - `"must"`: Must resume the run with the given name, raises error if run
+              doesn't exist
+            - `"allow"`: Resume the run if it exists, otherwise create a new run
+            - `"never"`: Never resume a run, always create a new one
+        private (`bool`, *optional*):
+            Whether to make the Space private. If None (default), the repo will be
+            public unless the organization's default is private. This value is ignored
+            if the repo already exists.
+        settings (`Any`, *optional*):
+            Not used. Provided for compatibility with `wandb.init()`.
+        embed (`bool`, *optional*, defaults to `True`):
+            If running inside a Jupyter/Colab notebook, whether the dashboard should
+            automatically be embedded in the cell when trackio.init() is called. For
+            local runs, this launches a local Gradio app and embeds it. For Space runs,
+            this embeds the Space URL. In Colab, the local dashboard will be accessible
+            via a public share URL (default Gradio behavior).
+        auto_log_gpu (`bool` or `None`, *optional*, defaults to `None`):
+            Controls automatic GPU metrics logging. If `None` (default), GPU logging
+            is automatically enabled when `nvidia-ml-py` is installed and an NVIDIA
+            GPU or Apple M series is detected. Set to `True` to force enable or
+            `False` to disable.
+        gpu_log_interval (`float`, *optional*, defaults to `10.0`):
+            The interval in seconds between automatic GPU metric logs.
+            Only used when `auto_log_gpu=True`.
+        webhook_url (`str`, *optional*):
+            A webhook URL to POST alert payloads to when `trackio.alert()` is
+            called. Supports Slack and Discord webhook URLs natively (payloads
+            are formatted automatically). Can also be set via the
+            `TRACKIO_WEBHOOK_URL` environment variable. Individual alerts can
+            override this URL by passing `webhook_url` to `trackio.alert()`.
+        webhook_min_level (`AlertLevel` or `str`, *optional*):
+            Minimum alert level that should trigger webhook delivery.
+            For example, `AlertLevel.WARN` sends only `WARN` and `ERROR`
+            alerts to the webhook destination. Can also be set via
+            `TRACKIO_WEBHOOK_MIN_LEVEL`.
+    Returns:
+        `Run`: A [`Run`] object that can be used to log metrics and finish the run.
+    """
+    if settings is not None:
+        warnings.warn(
+            "* Warning: settings is not used. Provided for compatibility with wandb.init(). Please create an issue at: https://github.com/gradio-app/trackio/issues if you need a specific feature implemented."
+        )
+
+    if space_id is None and dataset_id is not None:
+        raise ValueError("Must provide a `space_id` when `dataset_id` is provided.")
+    try:
+        space_id, dataset_id = utils.preprocess_space_and_dataset_ids(
+            space_id, dataset_id
+        )
+    except LocalTokenNotFoundError as e:
+        raise LocalTokenNotFoundError(
+            f"You must be logged in to Hugging Face locally when `space_id` is provided to deploy to a Space. {e}"
+        ) from e
+
+    url = context_vars.current_server.get()
+
+    if space_id is not None:
+        if url is None:
+            url = space_id
+            context_vars.current_server.set(url)
+            context_vars.current_space_id.set(space_id)
+
+    _should_embed_local = False
+
+    if (
+        context_vars.current_project.get() is None
+        or context_vars.current_project.get() != project
+    ):
+        print(f"* Trackio project initialized: {project}")
+
+        if dataset_id is not None:
+            os.environ["TRACKIO_DATASET_ID"] = dataset_id
+            print(
+                f"* Trackio metrics will be synced to Hugging Face Dataset: {dataset_id}"
+            )
+        if space_id is None:
+            print(f"* Trackio metrics logged to: {TRACKIO_DIR}")
+            _should_embed_local = embed and utils.is_in_notebook()
+            if not _should_embed_local:
+                utils.print_dashboard_instructions(project)
+        else:
+            deploy.create_space_if_not_exists(
+                space_id, space_storage, dataset_id, private
+            )
+            user_name, space_name = space_id.split("/")
+            space_url = deploy.SPACE_HOST_URL.format(
+                user_name=user_name, space_name=space_name
+            )
+            print(f"* View dashboard by going to: {space_url}")
+            if utils.is_in_notebook() and embed:
+                utils.embed_url_in_notebook(space_url)
+    context_vars.current_project.set(project)
+
+    if resume == "must":
+        if name is None:
+            raise ValueError("Must provide a run name when resume='must'")
+        if name not in SQLiteStorage.get_runs(project):
+            raise ValueError(f"Run '{name}' does not exist in project '{project}'")
+        resumed = True
+    elif resume == "allow":
+        resumed = name is not None and name in SQLiteStorage.get_runs(project)
+    elif resume == "never":
+        if name is not None and name in SQLiteStorage.get_runs(project):
+            warnings.warn(
+                f"* Warning: resume='never' but a run '{name}' already exists in "
+                f"project '{project}'. Generating a new name and instead. If you want "
+                "to resume this run, call init() with resume='must' or resume='allow'."
+            )
+            name = None
+        resumed = False
+    else:
+        raise ValueError("resume must be one of: 'must', 'allow', or 'never'")
+
+    if auto_log_gpu is None:
+        nvidia_available = gpu_available()
+        apple_available = apple_gpu_available()
+        auto_log_gpu = nvidia_available or apple_available
+        if nvidia_available:
+            print("* NVIDIA GPU detected, enabling automatic GPU metrics logging")
+        elif apple_available:
+            print("* Apple Silicon detected, enabling automatic system metrics logging")
+
+    run = Run(
+        url=url,
+        project=project,
+        client=None,
+        name=name,
+        group=group,
+        config=config,
+        space_id=space_id,
+        auto_log_gpu=auto_log_gpu,
+        gpu_log_interval=gpu_log_interval,
+        webhook_url=webhook_url,
+        webhook_min_level=webhook_min_level,
+    )
+
+    if space_id is not None:
+        SQLiteStorage.set_project_metadata(project, "space_id", space_id)
+        if SQLiteStorage.has_pending_data(project):
+            run._has_local_buffer = True
+
+    global _atexit_registered
+    if not _atexit_registered:
+        atexit.register(_cleanup_current_run)
+        _atexit_registered = True
+
+    if resumed:
+        print(f"* Resumed existing run: {run.name}")
+    else:
+        print(f"* Created new run: {run.name}")
+
+    context_vars.current_run.set(run)
+    globals()["config"] = run.config
+
+    if _should_embed_local:
+        show(project=project, open_browser=False, block_thread=False)
+
+    return run
+
+
+def log(metrics: dict, step: int | None = None) -> None:
+    """
+    Logs metrics to the current run.
+
+    Args:
+        metrics (`dict`):
+            A dictionary of metrics to log.
+        step (`int`, *optional*):
+            The step number. If not provided, the step will be incremented
+            automatically.
+    """
+    run = context_vars.current_run.get()
+    if run is None:
+        raise RuntimeError("Call trackio.init() before trackio.log().")
+    run.log(
+        metrics=metrics,
+        step=step,
+    )
+
+
+def log_system(metrics: dict) -> None:
+    """
+    Logs system metrics (GPU, etc.) to the current run using timestamps instead of steps.
+
+    Args:
+        metrics (`dict`):
+            A dictionary of system metrics to log.
+    """
+    run = context_vars.current_run.get()
+    if run is None:
+        raise RuntimeError("Call trackio.init() before trackio.log_system().")
+    run.log_system(metrics=metrics)
+
+
+def log_gpu(run: Run | None = None, device: int | None = None) -> dict:
+    """
+    Log GPU metrics to the current or specified run as system metrics.
+    Automatically detects whether an NVIDIA or Apple GPU is available and calls
+    the appropriate logging method.
+
+    Args:
+        run: Optional Run instance. If None, uses current run from context.
+        device: CUDA device index to collect metrics from (NVIDIA GPUs only).
+                If None, collects from all GPUs visible to this process.
+                This parameter is ignored for Apple GPUs.
+
+    Returns:
+        dict: The GPU metrics that were logged.
+
+    Example:
+        ```python
+        import trackio
+
+        run = trackio.init(project="my-project")
+        trackio.log({"loss": 0.5})
+        trackio.log_gpu()
+        trackio.log_gpu(device=0)
+        ```
+    """
+    if run is None:
+        run = context_vars.current_run.get()
+        if run is None:
+            raise RuntimeError("Call trackio.init() before trackio.log_gpu().")
+
+    if gpu_available():
+        return _log_nvidia_gpu(run=run, device=device)
+    elif apple_gpu_available():
+        return _log_apple_gpu(run=run)
+    else:
+        warnings.warn(
+            "No GPU detected. Install nvidia-ml-py for NVIDIA GPU support "
+            "or psutil for Apple Silicon support."
+        )
+        return {}
+
+
+def finish():
+    """
+    Finishes the current run.
+    """
+    run = context_vars.current_run.get()
+    if run is None:
+        raise RuntimeError("Call trackio.init() before trackio.finish().")
+    run.finish()
+
+
+def alert(
+    title: str,
+    text: str | None = None,
+    level: AlertLevel = AlertLevel.WARN,
+    webhook_url: str | None = None,
+) -> None:
+    """
+    Fires an alert immediately on the current run. The alert is printed to the
+    terminal, stored in the database, and displayed in the dashboard. If a
+    webhook URL is configured (via `trackio.init()`, the `TRACKIO_WEBHOOK_URL`
+    environment variable, or the `webhook_url` parameter here), the alert is
+    also POSTed to that URL.
+
+    Args:
+        title (`str`):
+            A short title for the alert.
+        text (`str`, *optional*):
+            A longer description with details about the alert.
+        level (`AlertLevel`, *optional*, defaults to `AlertLevel.WARN`):
+            The severity level. One of `AlertLevel.INFO`, `AlertLevel.WARN`,
+            or `AlertLevel.ERROR`.
+        webhook_url (`str`, *optional*):
+            A webhook URL to send this specific alert to. Overrides any
+            URL set in `trackio.init()` or the `TRACKIO_WEBHOOK_URL`
+            environment variable. Supports Slack and Discord webhook
+            URLs natively.
+    """
+    run = context_vars.current_run.get()
+    if run is None:
+        raise RuntimeError("Call trackio.init() before trackio.alert().")
+    run.alert(title=title, text=text, level=level, webhook_url=webhook_url)
+
+
+def delete_project(project: str, force: bool = False) -> bool:
+    """
+    Deletes a project by removing its local SQLite database.
+
+    Args:
+        project (`str`):
+            The name of the project to delete.
+        force (`bool`, *optional*, defaults to `False`):
+            If `True`, deletes the project without prompting for confirmation.
+            If `False`, prompts the user to confirm before deleting.
+
+    Returns:
+        `bool`: `True` if the project was deleted, `False` otherwise.
+    """
+    db_path = SQLiteStorage.get_project_db_path(project)
+
+    if not db_path.exists():
+        print(f"* Project '{project}' does not exist.")
+        return False
+
+    if not force:
+        response = input(
+            f"Are you sure you want to delete project '{project}'? "
+            f"This will permanently delete all runs and metrics. (y/N): "
+        )
+        if response.lower() not in ["y", "yes"]:
+            print("* Deletion cancelled.")
+            return False
+
+    try:
+        db_path.unlink()
+
+        for suffix in ("-wal", "-shm"):
+            sidecar = Path(str(db_path) + suffix)
+            if sidecar.exists():
+                sidecar.unlink()
+
+        print(f"* Project '{project}' has been deleted.")
+        return True
+    except Exception as e:
+        print(f"* Error deleting project '{project}': {e}")
+        return False
+
+
+def save(
+    glob_str: str | Path,
+    project: str | None = None,
+) -> str:
+    """
+    Saves files to a project (not linked to a specific run). If Trackio is running
+    locally, the file(s) will be copied to the project's files directory. If Trackio is
+    running in a Space, the file(s) will be uploaded to the Space's files directory.
+
+    Args:
+        glob_str (`str` or `Path`):
+            The file path or glob pattern to save. Can be a single file or a pattern
+            matching multiple files (e.g., `"*.py"`, `"models/**/*.pth"`).
+        project (`str`, *optional*):
+            The name of the project to save files to. If not provided, uses the current
+            project from `trackio.init()`. If no project is initialized, raises an
+            error.
+
+    Returns:
+        `str`: The path where the file(s) were saved (project's files directory).
+
+    Example:
+        ```python
+        import trackio
+
+        trackio.init(project="my-project")
+        trackio.save("config.yaml")
+        trackio.save("models/*.pth")
+        ```
+    """
+    if project is None:
+        project = context_vars.current_project.get()
+        if project is None:
+            raise RuntimeError(
+                "No project specified. Either call trackio.init() first or provide a "
+                "project parameter to trackio.save()."
+            )
+
+    glob_str = Path(glob_str)
+    base_path = Path.cwd().resolve()
+
+    matched_files = []
+    if glob_str.is_file():
+        matched_files = [glob_str.resolve()]
+    else:
+        pattern = str(glob_str)
+        if not glob_str.is_absolute():
+            pattern = str((Path.cwd() / glob_str).resolve())
+        matched_files = [
+            Path(f).resolve()
+            for f in glob.glob(pattern, recursive=True)
+            if Path(f).is_file()
+        ]
+
+    if not matched_files:
+        raise ValueError(f"No files found matching pattern: {glob_str}")
+
+    current_run = context_vars.current_run.get()
+    is_local = (
+        current_run._is_local
+        if current_run is not None
+        else (context_vars.current_space_id.get() is None)
+    )
+
+    if is_local:
+        for file_path in matched_files:
+            try:
+                relative_to_base = file_path.relative_to(base_path)
+            except ValueError:
+                relative_to_base = Path(file_path.name)
+
+            if current_run is not None:
+                current_run._queue_upload(
+                    file_path,
+                    step=None,
+                    relative_path=str(relative_to_base.parent),
+                    use_run_name=False,
+                )
+            else:
+                media_path = get_project_media_path(
+                    project=project,
+                    run=None,
+                    step=None,
+                    relative_path=str(relative_to_base),
+                )
+                shutil.copy(str(file_path), str(media_path))
+    else:
+        url = context_vars.current_server.get()
+
+        upload_entries = []
+        for file_path in matched_files:
+            try:
+                relative_to_base = file_path.relative_to(base_path)
+            except ValueError:
+                relative_to_base = Path(file_path.name)
+
+            if current_run is not None:
+                current_run._queue_upload(
+                    file_path,
+                    step=None,
+                    relative_path=str(relative_to_base.parent),
+                    use_run_name=False,
+                )
+            else:
+                upload_entry: UploadEntry = {
+                    "project": project,
+                    "run": None,
+                    "step": None,
+                    "relative_path": str(relative_to_base),
+                    "uploaded_file": handle_file(file_path),
+                }
+                upload_entries.append(upload_entry)
+
+        if upload_entries:
+            if url is None:
+                raise RuntimeError(
+                    "No server available. Call trackio.init() before trackio.save() to start the server."
+                )
+
+            try:
+                client = Client(url, verbose=False, httpx_kwargs={"timeout": 90})
+                client.predict(
+                    api_name="/bulk_upload_media",
+                    uploads=upload_entries,
+                    hf_token=huggingface_hub.utils.get_token(),
+                )
+            except Exception as e:
+                warnings.warn(
+                    f"Failed to upload files: {e}. "
+                    "Files may not be available in the dashboard."
+                )
+
+    return str(utils.MEDIA_DIR / project / "files")
+
+
+def show(
+    project: str | None = None,
+    *,
+    theme: str | ThemeClass | None = None,
+    mcp_server: bool | None = None,
+    footer: bool = True,
+    color_palette: list[str] | None = None,
+    open_browser: bool = True,
+    block_thread: bool | None = None,
+    host: str | None = None,
+):
+    """
+    Launches the Trackio dashboard.
+
+    Args:
+        project (`str`, *optional*):
+            The name of the project whose runs to show. If not provided, all projects
+            will be shown and the user can select one.
+        theme (`str` or `ThemeClass`, *optional*):
+            A Gradio Theme to use for the dashboard instead of the default Gradio theme,
+            can be a built-in theme (e.g. `'soft'`, `'citrus'`), a theme from the Hub
+            (e.g. `"gstaff/xkcd"`), or a custom Theme class. If not provided, the
+            `TRACKIO_THEME` environment variable will be used, or if that is not set,
+            the default Gradio theme will be used.
+        mcp_server (`bool`, *optional*):
+            If `True`, the Trackio dashboard will be set up as an MCP server and certain
+            functions will be added as MCP tools. If `None` (default behavior), then the
+            `GRADIO_MCP_SERVER` environment variable will be used to determine if the
+            MCP server should be enabled (which is `"True"` on Hugging Face Spaces).
+        footer (`bool`, *optional*, defaults to `True`):
+            Whether to show the Gradio footer. When `False`, the footer will be hidden.
+            This can also be controlled via the `footer` query parameter in the URL.
+        color_palette (`list[str]`, *optional*):
+            A list of hex color codes to use for plot lines. If not provided, the
+            `TRACKIO_COLOR_PALETTE` environment variable will be used (comma-separated
+            hex codes), or if that is not set, the default color palette will be used.
+            Example: `['#FF0000', '#00FF00', '#0000FF']`
+        open_browser (`bool`, *optional*, defaults to `True`):
+            If `True` and not in a notebook, a new browser tab will be opened with the
+            dashboard. If `False`, the browser will not be opened.
+        block_thread (`bool`, *optional*):
+            If `True`, the main thread will be blocked until the dashboard is closed.
+            If `None` (default behavior), then the main thread will not be blocked if the
+            dashboard is launched in a notebook, otherwise the main thread will be blocked.
+        host (`str`, *optional*):
+            The host to bind the server to. If not provided, defaults to `'127.0.0.1'`
+            (localhost only). Set to `'0.0.0.0'` to allow remote access.
+
+        Returns:
+            `app`: The Gradio app object corresponding to the dashboard launched by Trackio.
+            `url`: The local URL of the dashboard.
+            `share_url`: The public share URL of the dashboard.
+            `full_url`: The full URL of the dashboard including the write token (will use the public share URL if launched publicly, otherwise the local URL).
+    """
+    if color_palette is not None:
+        os.environ["TRACKIO_COLOR_PALETTE"] = ",".join(color_palette)
+
+    theme = theme or os.environ.get("TRACKIO_THEME")
+
+    _mcp_server = (
+        mcp_server
+        if mcp_server is not None
+        else os.environ.get("GRADIO_MCP_SERVER", "False") == "True"
+    )
+
+    server = make_trackio_server()
+    mount_frontend(server)
+
+    _, url, share_url = server.launch(
+        quiet=True,
+        inline=False,
+        prevent_thread_lock=True,
+        favicon_path=TRACKIO_LOGO_DIR / "trackio_logo_light.png",
+        allowed_paths=[TRACKIO_LOGO_DIR, TRACKIO_DIR],
+        mcp_server=_mcp_server,
+        theme=theme,
+        server_name=host,
+    )
+
+    base_url = share_url + "/" if share_url else url
+    dashboard_url = base_url.rstrip("/") + "/"
+    if project:
+        dashboard_url += f"?project={project}"
+    full_url = utils.get_full_url(
+        base_url.rstrip("/"),
+        project=project,
+        write_token=server.write_token,
+        footer=footer,
+    )
+
+    if not utils.is_in_notebook():
+        print(f"* Trackio UI launched at: {dashboard_url}")
+        print(f"* Gradio API available at: {base_url}")
+        if open_browser:
+            webbrowser.open(dashboard_url)
+        block_thread = block_thread if block_thread is not None else True
+    else:
+        utils.embed_url_in_notebook(dashboard_url)
+        block_thread = block_thread if block_thread is not None else False
+
+    if block_thread:
+        utils.block_main_thread_until_keyboard_interrupt()
+    return TupleNoPrint((server, url, share_url, full_url))

trackio/alerts.py

@@ -0,0 +1,185 @@
+import json
+import logging
+import ssl
+import urllib.error
+import urllib.request
+from enum import Enum
+
+try:
+    import certifi
+
+    _SSL_CONTEXT = ssl.create_default_context(cafile=certifi.where())
+except ImportError:
+    _SSL_CONTEXT = None
+
+logger = logging.getLogger(__name__)
+
+
+class AlertLevel(str, Enum):
+    INFO = "info"
+    WARN = "warn"
+    ERROR = "error"
+
+
+ALERT_LEVEL_ORDER = {
+    AlertLevel.INFO: 0,
+    AlertLevel.WARN: 1,
+    AlertLevel.ERROR: 2,
+}
+
+ALERT_COLORS = {
+    AlertLevel.INFO: "\033[94m",
+    AlertLevel.WARN: "\033[93m",
+    AlertLevel.ERROR: "\033[91m",
+}
+RESET_COLOR = "\033[0m"
+
+LEVEL_EMOJI = {
+    AlertLevel.INFO: "ℹ️",
+    AlertLevel.WARN: "⚠️",
+    AlertLevel.ERROR: "🚨",
+}
+
+
+def format_alert_terminal(
+    level: AlertLevel, title: str, text: str | None, step: int | None
+) -> str:
+    color = ALERT_COLORS.get(level, "")
+    step_str = f" (step {step})" if step is not None else ""
+    if text:
+        return f"{color}[TRACKIO {level.value.upper()}]{RESET_COLOR} {title}: {text}{step_str}"
+    return f"{color}[TRACKIO {level.value.upper()}]{RESET_COLOR} {title}{step_str}"
+
+
+def _is_slack_url(url: str) -> bool:
+    return "hooks.slack.com" in url
+
+
+def _is_discord_url(url: str) -> bool:
+    return "discord.com/api/webhooks" in url or "discordapp.com/api/webhooks" in url
+
+
+def _build_slack_payload(
+    level: AlertLevel,
+    title: str,
+    text: str | None,
+    project: str,
+    run: str,
+    step: int | None,
+) -> dict:
+    emoji = LEVEL_EMOJI.get(level, "")
+    step_str = f"  •  Step {step}" if step is not None else ""
+    header = f"{emoji} *[{level.value.upper()}] {title}*"
+    context = f"Project: {project}  •  Run: {run}{step_str}"
+    blocks = [
+        {"type": "section", "text": {"type": "mrkdwn", "text": header}},
+    ]
+    if text:
+        blocks.append({"type": "section", "text": {"type": "mrkdwn", "text": text}})
+    blocks.append(
+        {"type": "context", "elements": [{"type": "mrkdwn", "text": context}]}
+    )
+    return {"blocks": blocks}
+
+
+def _build_discord_payload(
+    level: AlertLevel,
+    title: str,
+    text: str | None,
+    project: str,
+    run: str,
+    step: int | None,
+) -> dict:
+    color_map = {
+        AlertLevel.INFO: 3447003,
+        AlertLevel.WARN: 16776960,
+        AlertLevel.ERROR: 15158332,
+    }
+    emoji = LEVEL_EMOJI.get(level, "")
+    step_str = f"  •  Step {step}" if step is not None else ""
+    embed = {
+        "title": f"{emoji} [{level.value.upper()}] {title}",
+        "color": color_map.get(level, 0),
+        "footer": {"text": f"Project: {project}  •  Run: {run}{step_str}"},
+    }
+    if text:
+        embed["description"] = text
+    return {"embeds": [embed]}
+
+
+def _build_generic_payload(
+    level: AlertLevel,
+    title: str,
+    text: str | None,
+    project: str,
+    run: str,
+    step: int | None,
+    timestamp: str | None,
+) -> dict:
+    return {
+        "level": level.value,
+        "title": title,
+        "text": text,
+        "project": project,
+        "run": run,
+        "step": step,
+        "timestamp": timestamp,
+    }
+
+
+def parse_alert_level(level: AlertLevel | str) -> AlertLevel:
+    if isinstance(level, AlertLevel):
+        return level
+    normalized = level.lower().strip()
+    try:
+        return AlertLevel(normalized)
+    except ValueError as e:
+        allowed = ", ".join(lvl.value for lvl in AlertLevel)
+        raise ValueError(
+            f"Invalid alert level '{level}'. Expected one of: {allowed}."
+        ) from e
+
+
+def resolve_webhook_min_level(
+    webhook_min_level: AlertLevel | str | None,
+) -> AlertLevel | None:
+    if webhook_min_level is None:
+        return None
+    return parse_alert_level(webhook_min_level)
+
+
+def should_send_webhook(
+    level: AlertLevel, webhook_min_level: AlertLevel | None
+) -> bool:
+    if webhook_min_level is None:
+        return True
+    return ALERT_LEVEL_ORDER[level] >= ALERT_LEVEL_ORDER[webhook_min_level]
+
+
+def send_webhook(
+    url: str,
+    level: AlertLevel,
+    title: str,
+    text: str | None,
+    project: str,
+    run: str,
+    step: int | None,
+    timestamp: str | None = None,
+) -> None:
+    if _is_slack_url(url):
+        payload = _build_slack_payload(level, title, text, project, run, step)
+    elif _is_discord_url(url):
+        payload = _build_discord_payload(level, title, text, project, run, step)
+    else:
+        payload = _build_generic_payload(
+            level, title, text, project, run, step, timestamp
+        )
+
+    data = json.dumps(payload).encode("utf-8")
+    req = urllib.request.Request(
+        url, data=data, headers={"Content-Type": "application/json"}
+    )
+    try:
+        urllib.request.urlopen(req, timeout=10, context=_SSL_CONTEXT)
+    except Exception as e:
+        logger.warning(f"Failed to send webhook to {url}: {e}")

trackio/api.py

@@ -0,0 +1,87 @@
+from typing import Iterator
+
+from trackio.sqlite_storage import SQLiteStorage
+
+
+class Run:
+    def __init__(self, project: str, name: str):
+        self.project = project
+        self.name = name
+        self._config = None
+
+    @property
+    def id(self) -> str:
+        return self.name
+
+    @property
+    def config(self) -> dict | None:
+        if self._config is None:
+            self._config = SQLiteStorage.get_run_config(self.project, self.name)
+        return self._config
+
+    def alerts(self, level: str | None = None, since: str | None = None) -> list[dict]:
+        return SQLiteStorage.get_alerts(
+            self.project, run_name=self.name, level=level, since=since
+        )
+
+    def delete(self) -> bool:
+        return SQLiteStorage.delete_run(self.project, self.name)
+
+    def move(self, new_project: str) -> bool:
+        success = SQLiteStorage.move_run(self.project, self.name, new_project)
+        if success:
+            self.project = new_project
+        return success
+
+    def rename(self, new_name: str) -> "Run":
+        SQLiteStorage.rename_run(self.project, self.name, new_name)
+        self.name = new_name
+        return self
+
+    def __repr__(self) -> str:
+        return f"<Run {self.name} in project {self.project}>"
+
+
+class Runs:
+    def __init__(self, project: str):
+        self.project = project
+        self._runs = None
+
+    def _load_runs(self):
+        if self._runs is None:
+            run_names = SQLiteStorage.get_runs(self.project)
+            self._runs = [Run(self.project, name) for name in run_names]
+
+    def __iter__(self) -> Iterator[Run]:
+        self._load_runs()
+        return iter(self._runs)
+
+    def __getitem__(self, index: int) -> Run:
+        self._load_runs()
+        return self._runs[index]
+
+    def __len__(self) -> int:
+        self._load_runs()
+        return len(self._runs)
+
+    def __repr__(self) -> str:
+        self._load_runs()
+        return f"<Runs project={self.project} count={len(self._runs)}>"
+
+
+class Api:
+    def runs(self, project: str) -> Runs:
+        if not SQLiteStorage.get_project_db_path(project).exists():
+            raise ValueError(f"Project '{project}' does not exist")
+        return Runs(project)
+
+    def alerts(
+        self,
+        project: str,
+        run: str | None = None,
+        level: str | None = None,
+        since: str | None = None,
+    ) -> list[dict]:
+        if not SQLiteStorage.get_project_db_path(project).exists():
+            raise ValueError(f"Project '{project}' does not exist")
+        return SQLiteStorage.get_alerts(project, run_name=run, level=level, since=since)

trackio/apple_gpu.py

@@ -0,0 +1,253 @@
+import platform
+import subprocess
+import sys
+import threading
+import warnings
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    from trackio.run import Run
+
+psutil: Any = None
+PSUTIL_AVAILABLE = False
+_monitor_lock = threading.Lock()
+
+
+def _ensure_psutil():
+    global PSUTIL_AVAILABLE, psutil
+    if PSUTIL_AVAILABLE:
+        return psutil
+    try:
+        import psutil as _psutil
+
+        psutil = _psutil
+        PSUTIL_AVAILABLE = True
+        return psutil
+    except ImportError:
+        raise ImportError(
+            "psutil is required for Apple Silicon monitoring. "
+            "Install it with: pip install psutil"
+        )
+
+
+def is_apple_silicon() -> bool:
+    """Check if running on Apple Silicon (M1/M2/M3/M4)."""
+    if platform.system() != "Darwin":
+        return False
+
+    try:
+        result = subprocess.run(
+            ["sysctl", "-n", "machdep.cpu.brand_string"],
+            capture_output=True,
+            text=True,
+            timeout=1,
+        )
+        cpu_brand = result.stdout.strip()
+        return "Apple" in cpu_brand
+    except Exception:
+        return False
+
+
+def get_gpu_info() -> dict[str, Any]:
+    """Get Apple GPU information using ioreg."""
+    try:
+        result = subprocess.run(
+            ["ioreg", "-r", "-d", "1", "-w", "0", "-c", "IOAccelerator"],
+            capture_output=True,
+            text=True,
+            timeout=2,
+        )
+
+        if result.returncode == 0 and result.stdout:
+            lines = result.stdout.strip().split("\n")
+            for line in lines:
+                if "IOAccelerator" in line and "class" in line:
+                    return {"detected": True, "type": "Apple GPU"}
+        else:
+            print("Error collecting Apple GPU info. ioreg stdout was:", file=sys.stderr)
+            print(result.stdout, file=sys.stderr)
+            print("ioreg stderr was:", file=sys.stderr)
+            print(result.stderr, file=sys.stderr)
+
+        result = subprocess.run(
+            ["system_profiler", "SPDisplaysDataType"],
+            capture_output=True,
+            text=True,
+            timeout=3,
+        )
+
+        if result.returncode == 0 and "Apple" in result.stdout:
+            for line in result.stdout.split("\n"):
+                if "Chipset Model:" in line:
+                    model = line.split(":")[-1].strip()
+                    return {"detected": True, "type": model}
+
+    except Exception:
+        pass
+
+    return {"detected": False}
+
+
+def apple_gpu_available() -> bool:
+    """
+    Check if Apple GPU monitoring is available.
+
+    Returns True if running on Apple Silicon (M-series chips) and psutil is installed.
+    """
+    try:
+        _ensure_psutil()
+        return is_apple_silicon()
+    except ImportError:
+        return False
+    except Exception:
+        return False
+
+
+def collect_apple_metrics() -> dict:
+    """
+    Collect system metrics for Apple Silicon.
+
+    Returns:
+        Dictionary of system metrics including CPU, memory, and GPU info.
+    """
+    if not PSUTIL_AVAILABLE:
+        try:
+            _ensure_psutil()
+        except ImportError:
+            return {}
+
+    metrics = {}
+
+    try:
+        cpu_percent = psutil.cpu_percent(interval=0.1, percpu=False)
+        metrics["cpu/utilization"] = cpu_percent
+    except Exception:
+        pass
+
+    try:
+        cpu_percents = psutil.cpu_percent(interval=0.1, percpu=True)
+        for i, percent in enumerate(cpu_percents):
+            metrics[f"cpu/{i}/utilization"] = percent
+    except Exception:
+        pass
+
+    try:
+        cpu_freq = psutil.cpu_freq()
+        if cpu_freq:
+            metrics["cpu/frequency"] = cpu_freq.current
+            if cpu_freq.max > 0:
+                metrics["cpu/frequency_max"] = cpu_freq.max
+    except Exception:
+        pass
+
+    try:
+        mem = psutil.virtual_memory()
+        metrics["memory/used"] = mem.used / (1024**3)
+        metrics["memory/total"] = mem.total / (1024**3)
+        metrics["memory/available"] = mem.available / (1024**3)
+        metrics["memory/percent"] = mem.percent
+    except Exception:
+        pass
+
+    try:
+        swap = psutil.swap_memory()
+        metrics["swap/used"] = swap.used / (1024**3)
+        metrics["swap/total"] = swap.total / (1024**3)
+        metrics["swap/percent"] = swap.percent
+    except Exception:
+        pass
+
+    try:
+        sensors_temps = psutil.sensors_temperatures()
+        if sensors_temps:
+            for name, entries in sensors_temps.items():
+                for i, entry in enumerate(entries):
+                    label = entry.label or f"{name}_{i}"
+                    metrics[f"temp/{label}"] = entry.current
+    except Exception:
+        pass
+
+    gpu_info = get_gpu_info()
+    if gpu_info.get("detected"):
+        metrics["gpu/detected"] = 1
+        if "type" in gpu_info:
+            pass
+
+    return metrics
+
+
+class AppleGpuMonitor:
+    def __init__(self, run: "Run", interval: float = 10.0):
+        self._run = run
+        self._interval = interval
+        self._stop_flag = threading.Event()
+        self._thread: "threading.Thread | None" = None
+
+    def start(self):
+        if not is_apple_silicon():
+            warnings.warn(
+                "auto_log_gpu=True but not running on Apple Silicon. "
+                "Apple GPU logging disabled."
+            )
+            return
+
+        if not PSUTIL_AVAILABLE:
+            try:
+                _ensure_psutil()
+            except ImportError:
+                warnings.warn(
+                    "auto_log_gpu=True but psutil not installed. "
+                    "Install with: pip install psutil"
+                )
+                return
+
+        self._thread = threading.Thread(target=self._monitor_loop, daemon=True)
+        self._thread.start()
+
+    def stop(self):
+        self._stop_flag.set()
+        if self._thread is not None:
+            self._thread.join(timeout=2.0)
+
+    def _monitor_loop(self):
+        while not self._stop_flag.is_set():
+            try:
+                metrics = collect_apple_metrics()
+                if metrics:
+                    self._run.log_system(metrics)
+            except Exception:
+                pass
+
+            self._stop_flag.wait(timeout=self._interval)
+
+
+def log_apple_gpu(run: "Run | None" = None) -> dict:
+    """
+    Log Apple Silicon system metrics to the current or specified run.
+
+    Args:
+        run: Optional Run instance. If None, uses current run from context.
+
+    Returns:
+        dict: The system metrics that were logged.
+
+    Example:
+        ```python
+        import trackio
+
+        run = trackio.init(project="my-project")
+        trackio.log({"loss": 0.5})
+        trackio.log_apple_gpu()
+        ```
+    """
+    from trackio import context_vars
+
+    if run is None:
+        run = context_vars.current_run.get()
+        if run is None:
+            raise RuntimeError("Call trackio.init() before trackio.log_apple_gpu().")
+
+    metrics = collect_apple_metrics()
+    if metrics:
+        run.log_system(metrics)
+    return metrics

trackio/cli.py

@@ -0,0 +1,1231 @@
+import argparse
+import os
+
+from trackio import show, sync
+from trackio.cli_helpers import (
+    error_exit,
+    format_alerts,
+    format_json,
+    format_list,
+    format_metric_values,
+    format_project_summary,
+    format_run_summary,
+    format_snapshot,
+    format_system_metric_names,
+    format_system_metrics,
+)
+from trackio.markdown import Markdown
+from trackio.server import get_project_summary, get_run_summary
+from trackio.sqlite_storage import SQLiteStorage
+
+
+def _get_space(args):
+    return getattr(args, "space", None)
+
+
+def _get_remote(args):
+    from trackio.remote_client import RemoteClient
+
+    space = _get_space(args)
+    if not space:
+        return None
+    hf_token = getattr(args, "hf_token", None)
+    return RemoteClient(space, hf_token=hf_token)
+
+
+def _handle_status():
+    print("Reading local Trackio projects...\n")
+    projects = SQLiteStorage.get_projects()
+    if not projects:
+        print("No Trackio projects found.")
+        return
+
+    local_projects = []
+    synced_projects = []
+    unsynced_projects = []
+
+    for project in projects:
+        space_id = SQLiteStorage.get_space_id(project)
+        if space_id is None:
+            local_projects.append(project)
+        elif SQLiteStorage.has_pending_data(project):
+            unsynced_projects.append(project)
+        else:
+            synced_projects.append(project)
+
+    print("Finished reading Trackio projects")
+    if local_projects:
+        print(f"  * {len(local_projects)} local trackio project(s) [OK]")
+    if synced_projects:
+        print(f"  * {len(synced_projects)} trackio project(s) synced to Spaces [OK]")
+    if unsynced_projects:
+        print(
+            f"  * {len(unsynced_projects)} trackio project(s) with unsynced changes [WARNING]:"
+        )
+        for p in unsynced_projects:
+            print(f"    - {p}")
+
+    if unsynced_projects:
+        print(
+            f"\nRun `trackio sync --project {unsynced_projects[0]}` to sync. "
+            "Or run `trackio sync --all` to sync all unsynced changes."
+        )
+
+
+def _handle_sync(args):
+    from trackio.deploy import sync_incremental
+
+    if args.sync_all and args.project:
+        error_exit("Cannot use --all and --project together.")
+    if not args.sync_all and not args.project:
+        error_exit("Must provide either --project or --all.")
+
+    if args.sync_all:
+        projects = SQLiteStorage.get_projects()
+        synced_any = False
+        for project in projects:
+            space_id = SQLiteStorage.get_space_id(project)
+            if space_id and SQLiteStorage.has_pending_data(project):
+                sync_incremental(
+                    project, space_id, private=args.private, pending_only=True
+                )
+                synced_any = True
+        if not synced_any:
+            print("No projects with unsynced data found.")
+    else:
+        space_id = args.space_id
+        if space_id is None:
+            space_id = SQLiteStorage.get_space_id(args.project)
+        sync(
+            project=args.project,
+            space_id=space_id,
+            private=args.private,
+            force=args.force,
+        )
+
+
+def _extract_reports(
+    run: str, logs: list[dict], report_name: str | None = None
+) -> list[dict]:
+    reports = []
+    for log in logs:
+        timestamp = log.get("timestamp")
+        step = log.get("step")
+        for key, value in log.items():
+            if report_name is not None and key != report_name:
+                continue
+            if isinstance(value, dict) and value.get("_type") == Markdown.TYPE:
+                content = value.get("_value")
+                if isinstance(content, str):
+                    reports.append(
+                        {
+                            "run": run,
+                            "report": key,
+                            "step": step,
+                            "timestamp": timestamp,
+                            "content": content,
+                        }
+                    )
+    return reports
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Trackio CLI")
+    parser.add_argument(
+        "--space",
+        required=False,
+        help="HF Space ID (e.g. 'user/space') or Space URL to query remotely.",
+    )
+    parser.add_argument(
+        "--hf-token",
+        required=False,
+        help="HF token for accessing private Spaces.",
+    )
+    subparsers = parser.add_subparsers(dest="command")
+
+    ui_parser = subparsers.add_parser(
+        "show", help="Show the Trackio dashboard UI for a project"
+    )
+    ui_parser.add_argument(
+        "--project", required=False, help="Project name to show in the dashboard"
+    )
+    ui_parser.add_argument(
+        "--theme",
+        required=False,
+        default="default",
+        help="A Gradio Theme to use for the dashboard instead of the default, can be a built-in theme (e.g. 'soft', 'citrus'), or a theme from the Hub (e.g. 'gstaff/xkcd').",
+    )
+    ui_parser.add_argument(
+        "--mcp-server",
+        action="store_true",
+        help="Enable MCP server functionality. The Trackio dashboard will be set up as an MCP server and certain functions will be exposed as MCP tools.",
+    )
+    ui_parser.add_argument(
+        "--footer",
+        action="store_true",
+        default=True,
+        help="Show the Gradio footer. Use --no-footer to hide it.",
+    )
+    ui_parser.add_argument(
+        "--no-footer",
+        dest="footer",
+        action="store_false",
+        help="Hide the Gradio footer.",
+    )
+    ui_parser.add_argument(
+        "--color-palette",
+        required=False,
+        help="Comma-separated list of hex color codes for plot lines (e.g. '#FF0000,#00FF00,#0000FF'). If not provided, the TRACKIO_COLOR_PALETTE environment variable will be used, or the default palette if not set.",
+    )
+    ui_parser.add_argument(
+        "--host",
+        required=False,
+        help="Host to bind the server to (e.g. '0.0.0.0' for remote access). If not provided, defaults to '127.0.0.1' (localhost only).",
+    )
+
+    subparsers.add_parser(
+        "status",
+        help="Show the status of all local Trackio projects, including sync status.",
+    )
+
+    sync_parser = subparsers.add_parser(
+        "sync",
+        help="Sync a local project's database to a Hugging Face Space. If the Space does not exist, it will be created.",
+    )
+    sync_parser.add_argument(
+        "--project",
+        required=False,
+        help="The name of the local project.",
+    )
+    sync_parser.add_argument(
+        "--space-id",
+        required=False,
+        help="The Hugging Face Space ID where the project will be synced (e.g. username/space_id). If not provided, uses the previously-configured Space.",
+    )
+    sync_parser.add_argument(
+        "--all",
+        action="store_true",
+        dest="sync_all",
+        help="Sync all projects that have unsynced data to their configured Spaces.",
+    )
+    sync_parser.add_argument(
+        "--private",
+        action="store_true",
+        help="Make the Hugging Face Space private if creating a new Space. By default, the repo will be public unless the organization's default is private. This value is ignored if the repo already exists.",
+    )
+    sync_parser.add_argument(
+        "--force",
+        action="store_true",
+        help="Overwrite the existing database without prompting for confirmation.",
+    )
+
+    list_parser = subparsers.add_parser(
+        "list",
+        help="List projects, runs, or metrics",
+    )
+    list_subparsers = list_parser.add_subparsers(dest="list_type", required=True)
+
+    list_projects_parser = list_subparsers.add_parser(
+        "projects",
+        help="List all projects",
+    )
+    list_projects_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    list_runs_parser = list_subparsers.add_parser(
+        "runs",
+        help="List runs for a project",
+    )
+    list_runs_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    list_runs_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    list_metrics_parser = list_subparsers.add_parser(
+        "metrics",
+        help="List metrics for a run",
+    )
+    list_metrics_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    list_metrics_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    list_metrics_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    list_system_metrics_parser = list_subparsers.add_parser(
+        "system-metrics",
+        help="List system metrics for a run",
+    )
+    list_system_metrics_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    list_system_metrics_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    list_system_metrics_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    list_alerts_parser = list_subparsers.add_parser(
+        "alerts",
+        help="List alerts for a project or run",
+    )
+    list_alerts_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    list_alerts_parser.add_argument(
+        "--run",
+        required=False,
+        help="Run name (optional)",
+    )
+    list_alerts_parser.add_argument(
+        "--level",
+        required=False,
+        help="Filter by alert level (info, warn, error)",
+    )
+    list_alerts_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+    list_alerts_parser.add_argument(
+        "--since",
+        required=False,
+        help="Only show alerts after this ISO 8601 timestamp",
+    )
+
+    list_reports_parser = list_subparsers.add_parser(
+        "reports",
+        help="List markdown reports for a project or run",
+    )
+    list_reports_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    list_reports_parser.add_argument(
+        "--run",
+        required=False,
+        help="Run name (optional)",
+    )
+    list_reports_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_parser = subparsers.add_parser(
+        "get",
+        help="Get project, run, or metric information",
+    )
+    get_subparsers = get_parser.add_subparsers(dest="get_type", required=True)
+
+    get_project_parser = get_subparsers.add_parser(
+        "project",
+        help="Get project summary",
+    )
+    get_project_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_project_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_run_parser = get_subparsers.add_parser(
+        "run",
+        help="Get run summary",
+    )
+    get_run_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_run_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    get_run_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_metric_parser = get_subparsers.add_parser(
+        "metric",
+        help="Get metric values for a run",
+    )
+    get_metric_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_metric_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    get_metric_parser.add_argument(
+        "--metric",
+        required=True,
+        help="Metric name",
+    )
+    get_metric_parser.add_argument(
+        "--step",
+        type=int,
+        required=False,
+        help="Get metric at exactly this step",
+    )
+    get_metric_parser.add_argument(
+        "--around",
+        type=int,
+        required=False,
+        help="Get metrics around this step (use with --window)",
+    )
+    get_metric_parser.add_argument(
+        "--at-time",
+        required=False,
+        help="Get metrics around this ISO 8601 timestamp (use with --window)",
+    )
+    get_metric_parser.add_argument(
+        "--window",
+        type=int,
+        required=False,
+        default=10,
+        help="Window size: ±steps for --around, ±seconds for --at-time (default: 10)",
+    )
+    get_metric_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_snapshot_parser = get_subparsers.add_parser(
+        "snapshot",
+        help="Get all metrics at/around a step or timestamp",
+    )
+    get_snapshot_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_snapshot_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    get_snapshot_parser.add_argument(
+        "--step",
+        type=int,
+        required=False,
+        help="Get all metrics at exactly this step",
+    )
+    get_snapshot_parser.add_argument(
+        "--around",
+        type=int,
+        required=False,
+        help="Get all metrics around this step (use with --window)",
+    )
+    get_snapshot_parser.add_argument(
+        "--at-time",
+        required=False,
+        help="Get all metrics around this ISO 8601 timestamp (use with --window)",
+    )
+    get_snapshot_parser.add_argument(
+        "--window",
+        type=int,
+        required=False,
+        default=10,
+        help="Window size: ±steps for --around, ±seconds for --at-time (default: 10)",
+    )
+    get_snapshot_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_system_metric_parser = get_subparsers.add_parser(
+        "system-metric",
+        help="Get system metric values for a run",
+    )
+    get_system_metric_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_system_metric_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    get_system_metric_parser.add_argument(
+        "--metric",
+        required=False,
+        help="System metric name (optional, if not provided returns all system metrics)",
+    )
+    get_system_metric_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_alerts_parser = get_subparsers.add_parser(
+        "alerts",
+        help="Get alerts for a project or run",
+    )
+    get_alerts_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_alerts_parser.add_argument(
+        "--run",
+        required=False,
+        help="Run name (optional)",
+    )
+    get_alerts_parser.add_argument(
+        "--level",
+        required=False,
+        help="Filter by alert level (info, warn, error)",
+    )
+    get_alerts_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+    get_alerts_parser.add_argument(
+        "--since",
+        required=False,
+        help="Only show alerts after this ISO 8601 timestamp",
+    )
+
+    get_report_parser = get_subparsers.add_parser(
+        "report",
+        help="Get markdown report entries for a run",
+    )
+    get_report_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_report_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    get_report_parser.add_argument(
+        "--report",
+        required=True,
+        help="Report metric name",
+    )
+    get_report_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    skills_parser = subparsers.add_parser(
+        "skills",
+        help="Manage Trackio skills for AI coding assistants",
+    )
+    skills_subparsers = skills_parser.add_subparsers(
+        dest="skills_action", required=True
+    )
+    skills_add_parser = skills_subparsers.add_parser(
+        "add",
+        help="Download and install the Trackio skill for an AI assistant",
+    )
+    skills_add_parser.add_argument(
+        "--cursor",
+        action="store_true",
+        help="Install for Cursor",
+    )
+    skills_add_parser.add_argument(
+        "--claude",
+        action="store_true",
+        help="Install for Claude Code",
+    )
+    skills_add_parser.add_argument(
+        "--codex",
+        action="store_true",
+        help="Install for Codex",
+    )
+    skills_add_parser.add_argument(
+        "--opencode",
+        action="store_true",
+        help="Install for OpenCode",
+    )
+    skills_add_parser.add_argument(
+        "--global",
+        dest="global_",
+        action="store_true",
+        help="Install globally (user-level) instead of in the current project directory",
+    )
+    skills_add_parser.add_argument(
+        "--dest",
+        type=str,
+        required=False,
+        help="Install into a custom destination (path to skills directory)",
+    )
+    skills_add_parser.add_argument(
+        "--force",
+        action="store_true",
+        help="Overwrite existing skill if it already exists",
+    )
+
+    args, unknown_args = parser.parse_known_args()
+    if unknown_args:
+        trailing_global_parser = argparse.ArgumentParser(add_help=False)
+        trailing_global_parser.add_argument("--space", required=False)
+        trailing_global_parser.add_argument("--hf-token", required=False)
+        trailing_globals, remaining_unknown = trailing_global_parser.parse_known_args(
+            unknown_args
+        )
+        if remaining_unknown:
+            parser.error(f"unrecognized arguments: {' '.join(remaining_unknown)}")
+        if trailing_globals.space is not None:
+            args.space = trailing_globals.space
+        if trailing_globals.hf_token is not None:
+            args.hf_token = trailing_globals.hf_token
+
+    if args.command in ("show", "status", "sync", "skills") and _get_space(args):
+        error_exit(
+            f"The '{args.command}' command does not support --space (remote mode)."
+        )
+
+    if args.command == "show":
+        color_palette = None
+        if args.color_palette:
+            color_palette = [color.strip() for color in args.color_palette.split(",")]
+        show(
+            project=args.project,
+            theme=args.theme,
+            mcp_server=args.mcp_server,
+            footer=args.footer,
+            color_palette=color_palette,
+            host=args.host,
+        )
+    elif args.command == "status":
+        _handle_status()
+    elif args.command == "sync":
+        _handle_sync(args)
+    elif args.command == "list":
+        remote = _get_remote(args)
+        if args.list_type == "projects":
+            if remote:
+                projects = remote.predict(api_name="/get_all_projects")
+            else:
+                projects = SQLiteStorage.get_projects()
+            if args.json:
+                print(format_json({"projects": projects}))
+            else:
+                print(format_list(projects, "Projects"))
+        elif args.list_type == "runs":
+            if remote:
+                runs = remote.predict(args.project, api_name="/get_runs_for_project")
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+            if args.json:
+                print(format_json({"project": args.project, "runs": runs}))
+            else:
+                print(format_list(runs, f"Runs in '{args.project}'"))
+        elif args.list_type == "metrics":
+            if remote:
+                metrics = remote.predict(
+                    args.project, args.run, api_name="/get_metrics_for_run"
+                )
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+                if args.run not in runs:
+                    error_exit(
+                        f"Run '{args.run}' not found in project '{args.project}'."
+                    )
+                metrics = SQLiteStorage.get_all_metrics_for_run(args.project, args.run)
+            if args.json:
+                print(
+                    format_json(
+                        {"project": args.project, "run": args.run, "metrics": metrics}
+                    )
+                )
+            else:
+                print(
+                    format_list(
+                        metrics, f"Metrics for '{args.run}' in '{args.project}'"
+                    )
+                )
+        elif args.list_type == "system-metrics":
+            if remote:
+                system_metrics = remote.predict(
+                    args.project, args.run, api_name="/get_system_metrics_for_run"
+                )
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+                if args.run not in runs:
+                    error_exit(
+                        f"Run '{args.run}' not found in project '{args.project}'."
+                    )
+                system_metrics = SQLiteStorage.get_all_system_metrics_for_run(
+                    args.project, args.run
+                )
+            if args.json:
+                print(
+                    format_json(
+                        {
+                            "project": args.project,
+                            "run": args.run,
+                            "system_metrics": system_metrics,
+                        }
+                    )
+                )
+            else:
+                print(format_system_metric_names(system_metrics))
+        elif args.list_type == "alerts":
+            if remote:
+                alerts = remote.predict(
+                    args.project,
+                    args.run,
+                    args.level,
+                    args.since,
+                    api_name="/get_alerts",
+                )
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                alerts = SQLiteStorage.get_alerts(
+                    args.project,
+                    run_name=args.run,
+                    level=args.level,
+                    since=args.since,
+                )
+            if args.json:
+                print(
+                    format_json(
+                        {
+                            "project": args.project,
+                            "run": args.run,
+                            "level": args.level,
+                            "since": args.since,
+                            "alerts": alerts,
+                        }
+                    )
+                )
+            else:
+                print(format_alerts(alerts))
+        elif args.list_type == "reports":
+            if remote:
+                runs = remote.predict(args.project, api_name="/get_runs_for_project")
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+            if args.run and args.run not in runs:
+                error_exit(f"Run '{args.run}' not found in project '{args.project}'.")
+
+            target_runs = [args.run] if args.run else runs
+            all_reports = []
+            for run_name in target_runs:
+                if remote:
+                    logs = remote.predict(args.project, run_name, api_name="/get_logs")
+                else:
+                    logs = SQLiteStorage.get_logs(args.project, run_name)
+                all_reports.extend(_extract_reports(run_name, logs))
+
+            if args.json:
+                print(
+                    format_json(
+                        {
+                            "project": args.project,
+                            "run": args.run,
+                            "reports": all_reports,
+                        }
+                    )
+                )
+            else:
+                report_lines = [
+                    f"{entry['run']} | {entry['report']} | step={entry['step']} | {entry['timestamp']}"
+                    for entry in all_reports
+                ]
+                if args.run:
+                    print(
+                        format_list(
+                            report_lines,
+                            f"Reports for '{args.run}' in '{args.project}'",
+                        )
+                    )
+                else:
+                    print(format_list(report_lines, f"Reports in '{args.project}'"))
+    elif args.command == "get":
+        remote = _get_remote(args)
+        if args.get_type == "project":
+            if remote:
+                summary = remote.predict(args.project, api_name="/get_project_summary")
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                summary = get_project_summary(args.project)
+            if args.json:
+                print(format_json(summary))
+            else:
+                print(format_project_summary(summary))
+        elif args.get_type == "run":
+            if remote:
+                summary = remote.predict(
+                    args.project, args.run, api_name="/get_run_summary"
+                )
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+                if args.run not in runs:
+                    error_exit(
+                        f"Run '{args.run}' not found in project '{args.project}'."
+                    )
+                summary = get_run_summary(args.project, args.run)
+            if args.json:
+                print(format_json(summary))
+            else:
+                print(format_run_summary(summary))
+        elif args.get_type == "metric":
+            at_time = getattr(args, "at_time", None)
+            if remote:
+                values = remote.predict(
+                    args.project,
+                    args.run,
+                    args.metric,
+                    args.step,
+                    args.around,
+                    at_time,
+                    args.window,
+                    api_name="/get_metric_values",
+                )
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+                if args.run not in runs:
+                    error_exit(
+                        f"Run '{args.run}' not found in project '{args.project}'."
+                    )
+                metrics = SQLiteStorage.get_all_metrics_for_run(args.project, args.run)
+                if args.metric not in metrics:
+                    error_exit(
+                        f"Metric '{args.metric}' not found in run '{args.run}' of project '{args.project}'."
+                    )
+                values = SQLiteStorage.get_metric_values(
+                    args.project,
+                    args.run,
+                    args.metric,
+                    step=args.step,
+                    around_step=args.around,
+                    at_time=at_time,
+                    window=args.window,
+                )
+            if args.json:
+                print(
+                    format_json(
+                        {
+                            "project": args.project,
+                            "run": args.run,
+                            "metric": args.metric,
+                            "values": values,
+                        }
+                    )
+                )
+            else:
+                print(format_metric_values(values))
+        elif args.get_type == "snapshot":
+            if not args.step and not args.around and not getattr(args, "at_time", None):
+                error_exit(
+                    "Provide --step, --around (with --window), or --at-time (with --window)."
+                )
+            at_time = getattr(args, "at_time", None)
+            if remote:
+                snapshot = remote.predict(
+                    args.project,
+                    args.run,
+                    args.step,
+                    args.around,
+                    at_time,
+                    args.window,
+                    api_name="/get_snapshot",
+                )
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+                if args.run not in runs:
+                    error_exit(
+                        f"Run '{args.run}' not found in project '{args.project}'."
+                    )
+                snapshot = SQLiteStorage.get_snapshot(
+                    args.project,
+                    args.run,
+                    step=args.step,
+                    around_step=args.around,
+                    at_time=at_time,
+                    window=args.window,
+                )
+            if args.json:
+                result = {
+                    "project": args.project,
+                    "run": args.run,
+                    "metrics": snapshot,
+                }
+                if args.step is not None:
+                    result["step"] = args.step
+                if args.around is not None:
+                    result["around"] = args.around
+                    result["window"] = args.window
+                if at_time is not None:
+                    result["at_time"] = at_time
+                    result["window"] = args.window
+                print(format_json(result))
+            else:
+                print(format_snapshot(snapshot))
+        elif args.get_type == "system-metric":
+            if remote:
+                system_metrics = remote.predict(
+                    args.project, args.run, api_name="/get_system_logs"
+                )
+                if args.metric:
+                    all_system_metric_names = remote.predict(
+                        args.project,
+                        args.run,
+                        api_name="/get_system_metrics_for_run",
+                    )
+                    if args.metric not in all_system_metric_names:
+                        error_exit(
+                            f"System metric '{args.metric}' not found in run '{args.run}' of project '{args.project}'."
+                        )
+                    filtered_metrics = [
+                        {
+                            k: v
+                            for k, v in entry.items()
+                            if k == "timestamp" or k == args.metric
+                        }
+                        for entry in system_metrics
+                        if args.metric in entry
+                    ]
+                    if args.json:
+                        print(
+                            format_json(
+                                {
+                                    "project": args.project,
+                                    "run": args.run,
+                                    "metric": args.metric,
+                                    "values": filtered_metrics,
+                                }
+                            )
+                        )
+                    else:
+                        print(format_system_metrics(filtered_metrics))
+                else:
+                    if args.json:
+                        print(
+                            format_json(
+                                {
+                                    "project": args.project,
+                                    "run": args.run,
+                                    "system_metrics": system_metrics,
+                                }
+                            )
+                        )
+                    else:
+                        print(format_system_metrics(system_metrics))
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+                if args.run not in runs:
+                    error_exit(
+                        f"Run '{args.run}' not found in project '{args.project}'."
+                    )
+                if args.metric:
+                    system_metrics = SQLiteStorage.get_system_logs(
+                        args.project, args.run
+                    )
+                    all_system_metric_names = (
+                        SQLiteStorage.get_all_system_metrics_for_run(
+                            args.project, args.run
+                        )
+                    )
+                    if args.metric not in all_system_metric_names:
+                        error_exit(
+                            f"System metric '{args.metric}' not found in run '{args.run}' of project '{args.project}'."
+                        )
+                    filtered_metrics = [
+                        {
+                            k: v
+                            for k, v in entry.items()
+                            if k == "timestamp" or k == args.metric
+                        }
+                        for entry in system_metrics
+                        if args.metric in entry
+                    ]
+                    if args.json:
+                        print(
+                            format_json(
+                                {
+                                    "project": args.project,
+                                    "run": args.run,
+                                    "metric": args.metric,
+                                    "values": filtered_metrics,
+                                }
+                            )
+                        )
+                    else:
+                        print(format_system_metrics(filtered_metrics))
+                else:
+                    system_metrics = SQLiteStorage.get_system_logs(
+                        args.project, args.run
+                    )
+                    if args.json:
+                        print(
+                            format_json(
+                                {
+                                    "project": args.project,
+                                    "run": args.run,
+                                    "system_metrics": system_metrics,
+                                }
+                            )
+                        )
+                    else:
+                        print(format_system_metrics(system_metrics))
+        elif args.get_type == "alerts":
+            if remote:
+                alerts = remote.predict(
+                    args.project,
+                    args.run,
+                    args.level,
+                    args.since,
+                    api_name="/get_alerts",
+                )
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                alerts = SQLiteStorage.get_alerts(
+                    args.project,
+                    run_name=args.run,
+                    level=args.level,
+                    since=args.since,
+                )
+            if args.json:
+                print(
+                    format_json(
+                        {
+                            "project": args.project,
+                            "run": args.run,
+                            "level": args.level,
+                            "since": args.since,
+                            "alerts": alerts,
+                        }
+                    )
+                )
+            else:
+                print(format_alerts(alerts))
+        elif args.get_type == "report":
+            if remote:
+                logs = remote.predict(args.project, args.run, api_name="/get_logs")
+            else:
+                db_path = SQLiteStorage.get_project_db_path(args.project)
+                if not db_path.exists():
+                    error_exit(f"Project '{args.project}' not found.")
+                runs = SQLiteStorage.get_runs(args.project)
+                if args.run not in runs:
+                    error_exit(
+                        f"Run '{args.run}' not found in project '{args.project}'."
+                    )
+                logs = SQLiteStorage.get_logs(args.project, args.run)
+
+            reports = _extract_reports(args.run, logs, report_name=args.report)
+            if not reports:
+                error_exit(
+                    f"Report '{args.report}' not found in run '{args.run}' of project '{args.project}'."
+                )
+
+            if args.json:
+                print(
+                    format_json(
+                        {
+                            "project": args.project,
+                            "run": args.run,
+                            "report": args.report,
+                            "values": reports,
+                        }
+                    )
+                )
+            else:
+                output = []
+                for idx, entry in enumerate(reports, start=1):
+                    output.append(
+                        f"Entry {idx} | step={entry['step']} | timestamp={entry['timestamp']}"
+                    )
+                    output.append(entry["content"])
+                    if idx < len(reports):
+                        output.append("-" * 80)
+                print("\n".join(output))
+    elif args.command == "skills":
+        if args.skills_action == "add":
+            _handle_skills_add(args)
+    else:
+        parser.print_help()
+
+
+def _handle_skills_add(args):
+    import shutil
+    from pathlib import Path
+
+    try:
+        from huggingface_hub.cli.skills import (
+            CENTRAL_GLOBAL,
+            CENTRAL_LOCAL,
+            GLOBAL_TARGETS,
+            LOCAL_TARGETS,
+        )
+    except (ImportError, ModuleNotFoundError):
+        error_exit(
+            "The 'trackio skills' command requires huggingface_hub >= 1.4.0.\n"
+            "Please upgrade: pip install --upgrade huggingface_hub"
+        )
+
+    SKILL_ID = "trackio"
+    GITHUB_RAW = "https://raw.githubusercontent.com/gradio-app/trackio/main"
+    SKILL_PREFIX = ".agents/skills/trackio"
+    SKILL_FILES = [
+        "SKILL.md",
+        "alerts.md",
+        "logging_metrics.md",
+        "retrieving_metrics.md",
+    ]
+
+    if not (args.cursor or args.claude or args.codex or args.opencode or args.dest):
+        error_exit(
+            "Pick a destination via --cursor, --claude, --codex, --opencode, or --dest."
+        )
+
+    def download(url: str) -> str:
+        from huggingface_hub.utils import get_session
+
+        try:
+            response = get_session().get(url)
+            response.raise_for_status()
+        except Exception as e:
+            error_exit(
+                f"Failed to download {url}\n{e}\n\n"
+                "Make sure you have internet access. The skill files are fetched from "
+                "the Trackio GitHub repository."
+            )
+        return response.text
+
+    def remove_existing(path: Path, force: bool):
+        if not (path.exists() or path.is_symlink()):
+            return
+        if not force:
+            error_exit(
+                f"Skill already exists at {path}.\nRe-run with --force to overwrite."
+            )
+        if path.is_dir() and not path.is_symlink():
+            shutil.rmtree(path)
+        else:
+            path.unlink()
+
+    def install_to(skills_dir: Path, force: bool) -> Path:
+        skills_dir = skills_dir.expanduser().resolve()
+        skills_dir.mkdir(parents=True, exist_ok=True)
+        dest = skills_dir / SKILL_ID
+        remove_existing(dest, force)
+        dest.mkdir()
+        for fname in SKILL_FILES:
+            content = download(f"{GITHUB_RAW}/{SKILL_PREFIX}/{fname}")
+            (dest / fname).write_text(content, encoding="utf-8")
+        return dest
+
+    def create_symlink(
+        agent_skills_dir: Path, central_skill_path: Path, force: bool
+    ) -> Path:
+        agent_skills_dir = agent_skills_dir.expanduser().resolve()
+        agent_skills_dir.mkdir(parents=True, exist_ok=True)
+        link_path = agent_skills_dir / SKILL_ID
+        remove_existing(link_path, force)
+        link_path.symlink_to(os.path.relpath(central_skill_path, agent_skills_dir))
+        return link_path
+
+    global_targets = {**GLOBAL_TARGETS, "cursor": Path("~/.cursor/skills")}
+    local_targets = {**LOCAL_TARGETS, "cursor": Path(".cursor/skills")}
+    targets_dict = global_targets if args.global_ else local_targets
+
+    if args.dest:
+        if args.cursor or args.claude or args.codex or args.opencode or args.global_:
+            error_exit("--dest cannot be combined with agent flags or --global.")
+        skill_dest = install_to(Path(args.dest), args.force)
+        print(f"Installed '{SKILL_ID}' to {skill_dest}")
+        return
+
+    agent_targets = []
+    if args.cursor:
+        agent_targets.append(targets_dict["cursor"])
+    if args.claude:
+        agent_targets.append(targets_dict["claude"])
+    if args.codex:
+        agent_targets.append(targets_dict["codex"])
+    if args.opencode:
+        agent_targets.append(targets_dict["opencode"])
+
+    central_path = CENTRAL_GLOBAL if args.global_ else CENTRAL_LOCAL
+    central_skill_path = install_to(central_path, args.force)
+    print(f"Installed '{SKILL_ID}' to central location: {central_skill_path}")
+
+    for agent_target in agent_targets:
+        link_path = create_symlink(agent_target, central_skill_path, args.force)
+        print(f"Created symlink: {link_path}")
+
+
+if __name__ == "__main__":
+    main()

trackio/cli_helpers.py

@@ -0,0 +1,158 @@
+import json
+import sys
+from typing import Any
+
+
+def format_json(data: Any) -> str:
+    """Format data as JSON."""
+    return json.dumps(data, indent=2)
+
+
+def format_list(items: list[str], title: str | None = None) -> str:
+    """Format a list of items in human-readable format."""
+    if not items:
+        return f"No {title.lower() if title else 'items'} found."
+
+    output = []
+    if title:
+        output.append(f"{title}:")
+
+    for item in items:
+        output.append(f"  - {item}")
+
+    return "\n".join(output)
+
+
+def format_project_summary(summary: dict) -> str:
+    """Format project summary in human-readable format."""
+    output = [f"Project: {summary['project']}"]
+    output.append(f"Number of runs: {summary['num_runs']}")
+
+    if summary["runs"]:
+        output.append("\nRuns:")
+        for run in summary["runs"]:
+            output.append(f"  - {run}")
+    else:
+        output.append("\nNo runs found.")
+
+    if summary.get("last_activity"):
+        output.append(f"\nLast activity (max step): {summary['last_activity']}")
+
+    return "\n".join(output)
+
+
+def format_run_summary(summary: dict) -> str:
+    """Format run summary in human-readable format."""
+    output = [f"Project: {summary['project']}"]
+    output.append(f"Run: {summary['run']}")
+    output.append(f"Number of logs: {summary['num_logs']}")
+
+    if summary.get("last_step") is not None:
+        output.append(f"Last step: {summary['last_step']}")
+
+    if summary.get("metrics"):
+        output.append("\nMetrics:")
+        for metric in summary["metrics"]:
+            output.append(f"  - {metric}")
+    else:
+        output.append("\nNo metrics found.")
+
+    config = summary.get("config")
+    if config:
+        output.append("\nConfig:")
+        config_display = {k: v for k, v in config.items() if not k.startswith("_")}
+        if config_display:
+            for key, value in config_display.items():
+                output.append(f"  {key}: {value}")
+        else:
+            output.append("  (no config)")
+    else:
+        output.append("\nConfig: (no config)")
+
+    return "\n".join(output)
+
+
+def format_metric_values(values: list[dict]) -> str:
+    """Format metric values in human-readable format."""
+    if not values:
+        return "No metric values found."
+
+    output = [f"Found {len(values)} value(s):\n"]
+    output.append("Step | Timestamp | Value")
+    output.append("-" * 50)
+
+    for value in values:
+        step = value.get("step", "N/A")
+        timestamp = value.get("timestamp", "N/A")
+        val = value.get("value", "N/A")
+        output.append(f"{step} | {timestamp} | {val}")
+
+    return "\n".join(output)
+
+
+def format_system_metrics(metrics: list[dict]) -> str:
+    """Format system metrics in human-readable format."""
+    if not metrics:
+        return "No system metrics found."
+
+    output = [f"Found {len(metrics)} system metric entry/entries:\n"]
+
+    for i, entry in enumerate(metrics):
+        timestamp = entry.get("timestamp", "N/A")
+        output.append(f"\nEntry {i + 1} (Timestamp: {timestamp}):")
+        for key, value in entry.items():
+            if key != "timestamp":
+                output.append(f"  {key}: {value}")
+
+    return "\n".join(output)
+
+
+def format_system_metric_names(names: list[str]) -> str:
+    """Format system metric names in human-readable format."""
+    return format_list(names, "System Metrics")
+
+
+def format_snapshot(snapshot: dict[str, list[dict]]) -> str:
+    """Format a metrics snapshot in human-readable format."""
+    if not snapshot:
+        return "No metrics found in the specified range."
+
+    output = []
+    for metric_name, values in sorted(snapshot.items()):
+        output.append(f"\n{metric_name}:")
+        output.append("  Step | Timestamp | Value")
+        output.append("  " + "-" * 48)
+        for v in values:
+            step = v.get("step", "N/A")
+            ts = v.get("timestamp", "N/A")
+            val = v.get("value", "N/A")
+            output.append(f"  {step} | {ts} | {val}")
+
+    return "\n".join(output)
+
+
+def format_alerts(alerts: list[dict]) -> str:
+    """Format alerts in human-readable format."""
+    if not alerts:
+        return "No alerts found."
+
+    output = [f"Found {len(alerts)} alert(s):\n"]
+    output.append("Timestamp | Run | Level | Title | Text | Step")
+    output.append("-" * 80)
+
+    for a in alerts:
+        ts = a.get("timestamp", "N/A")
+        run = a.get("run", "N/A")
+        level = a.get("level", "N/A").upper()
+        title = a.get("title", "")
+        text = a.get("text", "") or ""
+        step = a.get("step", "N/A")
+        output.append(f"{ts} | {run} | {level} | {title} | {text} | {step}")
+
+    return "\n".join(output)
+
+
+def error_exit(message: str, code: int = 1) -> None:
+    """Print error message and exit."""
+    print(f"Error: {message}", file=sys.stderr)
+    sys.exit(code)

trackio/commit_scheduler.py

@@ -0,0 +1,310 @@
+# Originally copied from https://github.com/huggingface/huggingface_hub/blob/d0a948fc2a32ed6e557042a95ef3e4af97ec4a7c/src/huggingface_hub/_commit_scheduler.py
+
+import atexit
+import logging
+import time
+from concurrent.futures import Future
+from dataclasses import dataclass
+from pathlib import Path
+from threading import Lock, Thread
+from typing import Callable, Dict, List, Union
+
+from huggingface_hub.hf_api import (
+    DEFAULT_IGNORE_PATTERNS,
+    CommitInfo,
+    CommitOperationAdd,
+    HfApi,
+)
+from huggingface_hub.utils import filter_repo_objects
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass(frozen=True)
+class _FileToUpload:
+    """Temporary dataclass to store info about files to upload. Not meant to be used directly."""
+
+    local_path: Path
+    path_in_repo: str
+    size_limit: int
+    last_modified: float
+
+
+class CommitScheduler:
+    """
+    Scheduler to upload a local folder to the Hub at regular intervals (e.g. push to hub every 5 minutes).
+
+    The recommended way to use the scheduler is to use it as a context manager. This ensures that the scheduler is
+    properly stopped and the last commit is triggered when the script ends. The scheduler can also be stopped manually
+    with the `stop` method. Checkout the [upload guide](https://huggingface.co/docs/huggingface_hub/guides/upload#scheduled-uploads)
+    to learn more about how to use it.
+
+    Args:
+        repo_id (`str`):
+            The id of the repo to commit to.
+        folder_path (`str` or `Path`):
+            Path to the local folder to upload regularly.
+        every (`int` or `float`, *optional*):
+            The number of minutes between each commit. Defaults to 5 minutes.
+        path_in_repo (`str`, *optional*):
+            Relative path of the directory in the repo, for example: `"checkpoints/"`. Defaults to the root folder
+            of the repository.
+        repo_type (`str`, *optional*):
+            The type of the repo to commit to. Defaults to `model`.
+        revision (`str`, *optional*):
+            The revision of the repo to commit to. Defaults to `main`.
+        private (`bool`, *optional*):
+            Whether to make the repo private. If `None` (default), the repo will be public unless the organization's default is private. This value is ignored if the repo already exists.
+        token (`str`, *optional*):
+            The token to use to commit to the repo. Defaults to the token saved on the machine.
+        allow_patterns (`List[str]` or `str`, *optional*):
+            If provided, only files matching at least one pattern are uploaded.
+        ignore_patterns (`List[str]` or `str`, *optional*):
+            If provided, files matching any of the patterns are not uploaded.
+        squash_history (`bool`, *optional*):
+            Whether to squash the history of the repo after each commit. Defaults to `False`. Squashing commits is
+            useful to avoid degraded performances on the repo when it grows too large.
+        hf_api (`HfApi`, *optional*):
+            The [`HfApi`] client to use to commit to the Hub. Can be set with custom settings (user agent, token,...).
+        on_before_commit (`Callable[[], None]`, *optional*):
+            If specified, a function that will be called before the CommitScheduler lists files to create a commit.
+
+    Example:
+    ```py
+    >>> from pathlib import Path
+    >>> from huggingface_hub import CommitScheduler
+
+    # Scheduler uploads every 10 minutes
+    >>> csv_path = Path("watched_folder/data.csv")
+    >>> CommitScheduler(repo_id="test_scheduler", repo_type="dataset", folder_path=csv_path.parent, every=10)
+
+    >>> with csv_path.open("a") as f:
+    ...     f.write("first line")
+
+    # Some time later (...)
+    >>> with csv_path.open("a") as f:
+    ...     f.write("second line")
+    ```
+
+    Example using a context manager:
+    ```py
+    >>> from pathlib import Path
+    >>> from huggingface_hub import CommitScheduler
+
+    >>> with CommitScheduler(repo_id="test_scheduler", repo_type="dataset", folder_path="watched_folder", every=10) as scheduler:
+    ...     csv_path = Path("watched_folder/data.csv")
+    ...     with csv_path.open("a") as f:
+    ...         f.write("first line")
+    ...     (...)
+    ...     with csv_path.open("a") as f:
+    ...         f.write("second line")
+
+    # Scheduler is now stopped and last commit have been triggered
+    ```
+    """
+
+    def __init__(
+        self,
+        *,
+        repo_id: str,
+        folder_path: Union[str, Path],
+        every: Union[int, float] = 5,
+        path_in_repo: str | None = None,
+        repo_type: str | None = None,
+        revision: str | None = None,
+        private: bool | None = None,
+        token: str | None = None,
+        allow_patterns: list[str] | str | None = None,
+        ignore_patterns: list[str] | str | None = None,
+        squash_history: bool = False,
+        hf_api: HfApi | None = None,
+        on_before_commit: Callable[[], None] | None = None,
+    ) -> None:
+        self.api = hf_api or HfApi(token=token)
+        self.on_before_commit = on_before_commit
+
+        # Folder
+        self.folder_path = Path(folder_path).expanduser().resolve()
+        self.path_in_repo = path_in_repo or ""
+        self.allow_patterns = allow_patterns
+
+        if ignore_patterns is None:
+            ignore_patterns = []
+        elif isinstance(ignore_patterns, str):
+            ignore_patterns = [ignore_patterns]
+        self.ignore_patterns = ignore_patterns + DEFAULT_IGNORE_PATTERNS
+
+        if self.folder_path.is_file():
+            raise ValueError(
+                f"'folder_path' must be a directory, not a file: '{self.folder_path}'."
+            )
+        self.folder_path.mkdir(parents=True, exist_ok=True)
+
+        # Repository
+        repo_url = self.api.create_repo(
+            repo_id=repo_id, private=private, repo_type=repo_type, exist_ok=True
+        )
+        self.repo_id = repo_url.repo_id
+        self.repo_type = repo_type
+        self.revision = revision
+        self.token = token
+
+        self.last_uploaded: Dict[Path, float] = {}
+        self.last_push_time: float | None = None
+
+        if not every > 0:
+            raise ValueError(f"'every' must be a positive integer, not '{every}'.")
+        self.lock = Lock()
+        self.every = every
+        self.squash_history = squash_history
+
+        logger.info(
+            f"Scheduled job to push '{self.folder_path}' to '{self.repo_id}' every {self.every} minutes."
+        )
+        self._scheduler_thread = Thread(target=self._run_scheduler, daemon=True)
+        self._scheduler_thread.start()
+        atexit.register(self._push_to_hub)
+
+        self.__stopped = False
+
+    def stop(self) -> None:
+        """Stop the scheduler.
+
+        A stopped scheduler cannot be restarted. Mostly for tests purposes.
+        """
+        self.__stopped = True
+
+    def __enter__(self) -> "CommitScheduler":
+        return self
+
+    def __exit__(self, exc_type, exc_value, traceback) -> None:
+        # Upload last changes before exiting
+        self.trigger().result()
+        self.stop()
+        return
+
+    def _run_scheduler(self) -> None:
+        """Dumb thread waiting between each scheduled push to Hub."""
+        while True:
+            self.last_future = self.trigger()
+            time.sleep(self.every * 60)
+            if self.__stopped:
+                break
+
+    def trigger(self) -> Future:
+        """Trigger a `push_to_hub` and return a future.
+
+        This method is automatically called every `every` minutes. You can also call it manually to trigger a commit
+        immediately, without waiting for the next scheduled commit.
+        """
+        return self.api.run_as_future(self._push_to_hub)
+
+    def _push_to_hub(self) -> CommitInfo | None:
+        if self.__stopped:  # If stopped, already scheduled commits are ignored
+            return None
+
+        logger.info("(Background) scheduled commit triggered.")
+        try:
+            value = self.push_to_hub()
+            if self.squash_history:
+                logger.info("(Background) squashing repo history.")
+                self.api.super_squash_history(
+                    repo_id=self.repo_id, repo_type=self.repo_type, branch=self.revision
+                )
+            return value
+        except Exception as e:
+            logger.error(
+                f"Error while pushing to Hub: {e}"
+            )  # Depending on the setup, error might be silenced
+            raise
+
+    def push_to_hub(self) -> CommitInfo | None:
+        """
+        Push folder to the Hub and return the commit info.
+
+        <Tip warning={true}>
+
+        This method is not meant to be called directly. It is run in the background by the scheduler, respecting a
+        queue mechanism to avoid concurrent commits. Making a direct call to the method might lead to concurrency
+        issues.
+
+        </Tip>
+
+        The default behavior of `push_to_hub` is to assume an append-only folder. It lists all files in the folder and
+        uploads only changed files. If no changes are found, the method returns without committing anything. If you want
+        to change this behavior, you can inherit from [`CommitScheduler`] and override this method. This can be useful
+        for example to compress data together in a single file before committing. For more details and examples, check
+        out our [integration guide](https://huggingface.co/docs/huggingface_hub/main/en/guides/upload#scheduled-uploads).
+        """
+        # Check files to upload (with lock)
+        with self.lock:
+            if self.on_before_commit is not None:
+                self.on_before_commit()
+
+            logger.debug("Listing files to upload for scheduled commit.")
+
+            # List files from folder (taken from `_prepare_upload_folder_additions`)
+            relpath_to_abspath = {
+                path.relative_to(self.folder_path).as_posix(): path
+                for path in sorted(
+                    self.folder_path.glob("**/*")
+                )  # sorted to be deterministic
+                if path.is_file()
+            }
+            prefix = f"{self.path_in_repo.strip('/')}/" if self.path_in_repo else ""
+
+            # Filter with pattern + filter out unchanged files + retrieve current file size
+            files_to_upload: List[_FileToUpload] = []
+            for relpath in filter_repo_objects(
+                relpath_to_abspath.keys(),
+                allow_patterns=self.allow_patterns,
+                ignore_patterns=self.ignore_patterns,
+            ):
+                local_path = relpath_to_abspath[relpath]
+                stat = local_path.stat()
+                if (
+                    self.last_uploaded.get(local_path) is None
+                    or self.last_uploaded[local_path] != stat.st_mtime
+                ):
+                    files_to_upload.append(
+                        _FileToUpload(
+                            local_path=local_path,
+                            path_in_repo=prefix + relpath,
+                            size_limit=stat.st_size,
+                            last_modified=stat.st_mtime,
+                        )
+                    )
+
+        # Return if nothing to upload
+        if len(files_to_upload) == 0:
+            logger.debug("Dropping schedule commit: no changed file to upload.")
+            return None
+
+        # Convert `_FileToUpload` as `CommitOperationAdd` (=> compute file shas + limit to file size)
+        logger.debug("Removing unchanged files since previous scheduled commit.")
+        add_operations = [
+            CommitOperationAdd(
+                # TODO: Cap the file to its current size, even if the user append data to it while a scheduled commit is happening
+                # (requires an upstream fix for XET-535: `hf_xet` should support `BinaryIO` for upload)
+                path_or_fileobj=file_to_upload.local_path,
+                path_in_repo=file_to_upload.path_in_repo,
+            )
+            for file_to_upload in files_to_upload
+        ]
+
+        # Upload files (append mode expected - no need for lock)
+        logger.debug("Uploading files for scheduled commit.")
+        commit_info = self.api.create_commit(
+            repo_id=self.repo_id,
+            repo_type=self.repo_type,
+            operations=add_operations,
+            commit_message="Scheduled Commit",
+            revision=self.revision,
+        )
+
+        for file in files_to_upload:
+            self.last_uploaded[file.local_path] = file.last_modified
+
+        self.last_push_time = time.time()
+
+        return commit_info

trackio/context_vars.py

@@ -0,0 +1,18 @@
+import contextvars
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from trackio.run import Run
+
+current_run: contextvars.ContextVar["Run | None"] = contextvars.ContextVar(
+    "current_run", default=None
+)
+current_project: contextvars.ContextVar[str | None] = contextvars.ContextVar(
+    "current_project", default=None
+)
+current_server: contextvars.ContextVar[str | None] = contextvars.ContextVar(
+    "current_server", default=None
+)
+current_space_id: contextvars.ContextVar[str | None] = contextvars.ContextVar(
+    "current_space_id", default=None
+)

trackio/deploy.py

@@ -0,0 +1,466 @@
+import importlib.metadata
+import io
+import os
+import sys
+import threading
+import time
+from importlib.resources import files
+from pathlib import Path
+
+if sys.version_info >= (3, 11):
+    import tomllib
+else:
+    import tomli as tomllib
+
+import gradio
+import huggingface_hub
+from gradio_client import Client, handle_file
+from httpx import ReadTimeout
+from huggingface_hub.errors import HfHubHTTPError, RepositoryNotFoundError
+
+import trackio
+from trackio.sqlite_storage import SQLiteStorage
+from trackio.utils import get_or_create_project_hash, preprocess_space_and_dataset_ids
+
+SPACE_HOST_URL = "https://{user_name}-{space_name}.hf.space/"
+SPACE_URL = "https://huggingface.co/spaces/{space_id}"
+
+
+def _get_source_install_dependencies() -> str:
+    """Get trackio dependencies from pyproject.toml for source installs."""
+    trackio_path = files("trackio")
+    pyproject_path = Path(trackio_path).parent / "pyproject.toml"
+    with open(pyproject_path, "rb") as f:
+        pyproject = tomllib.load(f)
+    deps = pyproject["project"]["dependencies"]
+    spaces_deps = (
+        pyproject["project"].get("optional-dependencies", {}).get("spaces", [])
+    )
+    return "\n".join(deps + spaces_deps)
+
+
+def _is_trackio_installed_from_source() -> bool:
+    """Check if trackio is installed from source/editable install vs PyPI."""
+    try:
+        trackio_file = trackio.__file__
+        if "site-packages" not in trackio_file and "dist-packages" not in trackio_file:
+            return True
+
+        dist = importlib.metadata.distribution("trackio")
+        if dist.files:
+            files = list(dist.files)
+            has_pth = any(".pth" in str(f) for f in files)
+            if has_pth:
+                return True
+
+        return False
+    except (
+        AttributeError,
+        importlib.metadata.PackageNotFoundError,
+        importlib.metadata.MetadataError,
+        ValueError,
+        TypeError,
+    ):
+        return True
+
+
+def deploy_as_space(
+    space_id: str,
+    space_storage: huggingface_hub.SpaceStorage | None = None,
+    dataset_id: str | None = None,
+    private: bool | None = None,
+):
+    if (
+        os.getenv("SYSTEM") == "spaces"
+    ):  # in case a repo with this function is uploaded to spaces
+        return
+
+    trackio_path = files("trackio")
+
+    hf_api = huggingface_hub.HfApi()
+
+    try:
+        huggingface_hub.create_repo(
+            space_id,
+            private=private,
+            space_sdk="gradio",
+            space_storage=space_storage,
+            repo_type="space",
+            exist_ok=True,
+        )
+    except HfHubHTTPError as e:
+        if e.response.status_code in [401, 403]:  # unauthorized or forbidden
+            print("Need 'write' access token to create a Spaces repo.")
+            huggingface_hub.login(add_to_git_credential=False)
+            huggingface_hub.create_repo(
+                space_id,
+                private=private,
+                space_sdk="gradio",
+                space_storage=space_storage,
+                repo_type="space",
+                exist_ok=True,
+            )
+        else:
+            raise ValueError(f"Failed to create Space: {e}")
+
+    # We can assume pandas, gradio, and huggingface-hub are already installed in a Gradio Space.
+    # Make sure necessary dependencies are installed by creating a requirements.txt.
+    is_source_install = _is_trackio_installed_from_source()
+
+    with open(Path(trackio_path, "README.md"), "r") as f:
+        readme_content = f.read()
+        readme_content = readme_content.replace("{GRADIO_VERSION}", gradio.__version__)
+        readme_content = readme_content.replace("{APP_FILE}", "app.py")
+        readme_buffer = io.BytesIO(readme_content.encode("utf-8"))
+        hf_api.upload_file(
+            path_or_fileobj=readme_buffer,
+            path_in_repo="README.md",
+            repo_id=space_id,
+            repo_type="space",
+        )
+
+    if is_source_install:
+        requirements_content = _get_source_install_dependencies()
+    else:
+        requirements_content = f"trackio[spaces]=={trackio.__version__}"
+
+    requirements_buffer = io.BytesIO(requirements_content.encode("utf-8"))
+    hf_api.upload_file(
+        path_or_fileobj=requirements_buffer,
+        path_in_repo="requirements.txt",
+        repo_id=space_id,
+        repo_type="space",
+    )
+
+    huggingface_hub.utils.disable_progress_bars()
+
+    if is_source_install:
+        dist_index = (
+            Path(trackio.__file__).resolve().parent / "frontend" / "dist" / "index.html"
+        )
+        if not dist_index.is_file():
+            raise ValueError(
+                "The Trackio frontend build is missing. From the repository root run "
+                "`cd trackio/frontend && npm ci && npm run build`, then deploy again."
+            )
+        hf_api.upload_folder(
+            repo_id=space_id,
+            repo_type="space",
+            folder_path=trackio_path,
+            path_in_repo="trackio",
+            ignore_patterns=[
+                "README.md",
+                "frontend/node_modules/**",
+                "frontend/src/**",
+                "frontend/.gitignore",
+                "frontend/package.json",
+                "frontend/package-lock.json",
+                "frontend/vite.config.js",
+                "frontend/svelte.config.js",
+                "**/__pycache__/**",
+                "*.pyc",
+            ],
+        )
+
+    app_file_content = """import trackio
+trackio.show()"""
+    app_file_buffer = io.BytesIO(app_file_content.encode("utf-8"))
+    hf_api.upload_file(
+        path_or_fileobj=app_file_buffer,
+        path_in_repo="app.py",
+        repo_id=space_id,
+        repo_type="space",
+    )
+
+    if hf_token := huggingface_hub.utils.get_token():
+        huggingface_hub.add_space_secret(space_id, "HF_TOKEN", hf_token)
+    if dataset_id is not None:
+        huggingface_hub.add_space_variable(space_id, "TRACKIO_DATASET_ID", dataset_id)
+    if logo_light_url := os.environ.get("TRACKIO_LOGO_LIGHT_URL"):
+        huggingface_hub.add_space_variable(
+            space_id, "TRACKIO_LOGO_LIGHT_URL", logo_light_url
+        )
+    if logo_dark_url := os.environ.get("TRACKIO_LOGO_DARK_URL"):
+        huggingface_hub.add_space_variable(
+            space_id, "TRACKIO_LOGO_DARK_URL", logo_dark_url
+        )
+    if plot_order := os.environ.get("TRACKIO_PLOT_ORDER"):
+        huggingface_hub.add_space_variable(space_id, "TRACKIO_PLOT_ORDER", plot_order)
+    if theme := os.environ.get("TRACKIO_THEME"):
+        huggingface_hub.add_space_variable(space_id, "TRACKIO_THEME", theme)
+    huggingface_hub.add_space_variable(space_id, "GRADIO_MCP_SERVER", "True")
+
+
+def create_space_if_not_exists(
+    space_id: str,
+    space_storage: huggingface_hub.SpaceStorage | None = None,
+    dataset_id: str | None = None,
+    private: bool | None = None,
+) -> None:
+    """
+    Creates a new Hugging Face Space if it does not exist.
+
+    Args:
+        space_id (`str`):
+            The ID of the Space to create.
+        space_storage ([`~huggingface_hub.SpaceStorage`], *optional*):
+            Choice of persistent storage tier for the Space.
+        dataset_id (`str`, *optional*):
+            The ID of the Dataset to add to the Space as a space variable.
+        private (`bool`, *optional*):
+            Whether to make the Space private. If `None` (default), the repo will be
+            public unless the organization's default is private. This value is ignored
+            if the repo already exists.
+    """
+    if "/" not in space_id:
+        raise ValueError(
+            f"Invalid space ID: {space_id}. Must be in the format: username/reponame or orgname/reponame."
+        )
+    if dataset_id is not None and "/" not in dataset_id:
+        raise ValueError(
+            f"Invalid dataset ID: {dataset_id}. Must be in the format: username/datasetname or orgname/datasetname."
+        )
+    try:
+        huggingface_hub.repo_info(space_id, repo_type="space")
+        print(f"* Found existing space: {SPACE_URL.format(space_id=space_id)}")
+        return
+    except RepositoryNotFoundError:
+        pass
+    except HfHubHTTPError as e:
+        if e.response.status_code in [401, 403]:  # unauthorized or forbidden
+            print("Need 'write' access token to create a Spaces repo.")
+            huggingface_hub.login(add_to_git_credential=False)
+        else:
+            raise ValueError(f"Failed to create Space: {e}")
+
+    print(f"* Creating new space: {SPACE_URL.format(space_id=space_id)}")
+    deploy_as_space(space_id, space_storage, dataset_id, private)
+    print("* Waiting for Space to be ready...")
+    _wait_until_space_running(space_id)
+
+
+def _wait_until_space_running(space_id: str, timeout: int = 300) -> None:
+    hf_api = huggingface_hub.HfApi()
+    start = time.time()
+    delay = 2
+    while time.time() - start < timeout:
+        try:
+            info = hf_api.space_info(space_id)
+            if info.runtime and info.runtime.stage == "RUNNING":
+                return
+        except (huggingface_hub.utils.HfHubHTTPError, ReadTimeout):
+            pass
+        time.sleep(delay)
+        delay = min(delay * 1.5, 15)
+
+
+def wait_until_space_exists(
+    space_id: str,
+) -> None:
+    """
+    Blocks the current thread until the Space exists.
+
+    Args:
+        space_id (`str`):
+            The ID of the Space to wait for.
+
+    Raises:
+        `TimeoutError`: If waiting for the Space takes longer than expected.
+    """
+    hf_api = huggingface_hub.HfApi()
+    delay = 1
+    for _ in range(30):
+        try:
+            hf_api.space_info(space_id)
+            return
+        except (huggingface_hub.utils.HfHubHTTPError, ReadTimeout):
+            time.sleep(delay)
+            delay = min(delay * 2, 60)
+    raise TimeoutError("Waiting for space to exist took longer than expected")
+
+
+def upload_db_to_space(project: str, space_id: str, force: bool = False) -> None:
+    """
+    Uploads the database of a local Trackio project to a Hugging Face Space.
+
+    This uses the Gradio Client to upload since we do not want to trigger a new build of
+    the Space, which would happen if we used `huggingface_hub.upload_file`.
+
+    Args:
+        project (`str`):
+            The name of the project to upload.
+        space_id (`str`):
+            The ID of the Space to upload to.
+        force (`bool`, *optional*, defaults to `False`):
+            If `True`, overwrites the existing database without prompting. If `False`,
+            prompts for confirmation.
+    """
+    db_path = SQLiteStorage.get_project_db_path(project)
+    client = Client(space_id, verbose=False, httpx_kwargs={"timeout": 90})
+
+    if not force:
+        try:
+            existing_projects = client.predict(api_name="/get_all_projects")
+            if project in existing_projects:
+                response = input(
+                    f"Database for project '{project}' already exists on Space '{space_id}'. "
+                    f"Overwrite it? (y/N): "
+                )
+                if response.lower() not in ["y", "yes"]:
+                    print("* Upload cancelled.")
+                    return
+        except Exception as e:
+            print(f"* Warning: Could not check if project exists on Space: {e}")
+            print("* Proceeding with upload...")
+
+    client.predict(
+        api_name="/upload_db_to_space",
+        project=project,
+        uploaded_db=handle_file(db_path),
+        hf_token=huggingface_hub.utils.get_token(),
+    )
+
+
+SYNC_BATCH_SIZE = 500
+
+
+def sync_incremental(
+    project: str,
+    space_id: str,
+    private: bool | None = None,
+    pending_only: bool = False,
+) -> None:
+    """
+    Syncs a local Trackio project to a Space via the bulk_log API endpoints
+    instead of uploading the entire DB file. Supports incremental sync.
+
+    Args:
+        project: The name of the project to sync.
+        space_id: The HF Space ID to sync to.
+        private: Whether to make the Space private if creating.
+        pending_only: If True, only sync rows tagged with space_id (pending data).
+    """
+    print(
+        f"* Syncing project '{project}' to: {SPACE_URL.format(space_id=space_id)} (please wait...)"
+    )
+    create_space_if_not_exists(space_id, private=private)
+    wait_until_space_exists(space_id)
+
+    client = Client(space_id, verbose=False, httpx_kwargs={"timeout": 90})
+    hf_token = huggingface_hub.utils.get_token()
+
+    if pending_only:
+        pending_logs = SQLiteStorage.get_pending_logs(project)
+        if pending_logs:
+            logs = pending_logs["logs"]
+            for i in range(0, len(logs), SYNC_BATCH_SIZE):
+                batch = logs[i : i + SYNC_BATCH_SIZE]
+                print(
+                    f"  Syncing metrics: {min(i + SYNC_BATCH_SIZE, len(logs))}/{len(logs)}..."
+                )
+                client.predict(api_name="/bulk_log", logs=batch, hf_token=hf_token)
+            SQLiteStorage.clear_pending_logs(project, pending_logs["ids"])
+
+        pending_sys = SQLiteStorage.get_pending_system_logs(project)
+        if pending_sys:
+            logs = pending_sys["logs"]
+            for i in range(0, len(logs), SYNC_BATCH_SIZE):
+                batch = logs[i : i + SYNC_BATCH_SIZE]
+                print(
+                    f"  Syncing system metrics: {min(i + SYNC_BATCH_SIZE, len(logs))}/{len(logs)}..."
+                )
+                client.predict(
+                    api_name="/bulk_log_system", logs=batch, hf_token=hf_token
+                )
+            SQLiteStorage.clear_pending_system_logs(project, pending_sys["ids"])
+
+        pending_uploads = SQLiteStorage.get_pending_uploads(project)
+        if pending_uploads:
+            upload_entries = []
+            for u in pending_uploads["uploads"]:
+                fp = u["file_path"]
+                if os.path.exists(fp):
+                    upload_entries.append(
+                        {
+                            "project": u["project"],
+                            "run": u["run"],
+                            "step": u["step"],
+                            "relative_path": u["relative_path"],
+                            "uploaded_file": handle_file(fp),
+                        }
+                    )
+            if upload_entries:
+                print(f"  Syncing {len(upload_entries)} media files...")
+                client.predict(
+                    api_name="/bulk_upload_media",
+                    uploads=upload_entries,
+                    hf_token=hf_token,
+                )
+            SQLiteStorage.clear_pending_uploads(project, pending_uploads["ids"])
+    else:
+        all_logs = SQLiteStorage.get_all_logs_for_sync(project)
+        if all_logs:
+            for i in range(0, len(all_logs), SYNC_BATCH_SIZE):
+                batch = all_logs[i : i + SYNC_BATCH_SIZE]
+                print(
+                    f"  Syncing metrics: {min(i + SYNC_BATCH_SIZE, len(all_logs))}/{len(all_logs)}..."
+                )
+                client.predict(api_name="/bulk_log", logs=batch, hf_token=hf_token)
+
+        all_sys_logs = SQLiteStorage.get_all_system_logs_for_sync(project)
+        if all_sys_logs:
+            for i in range(0, len(all_sys_logs), SYNC_BATCH_SIZE):
+                batch = all_sys_logs[i : i + SYNC_BATCH_SIZE]
+                print(
+                    f"  Syncing system metrics: {min(i + SYNC_BATCH_SIZE, len(all_sys_logs))}/{len(all_sys_logs)}..."
+                )
+                client.predict(
+                    api_name="/bulk_log_system", logs=batch, hf_token=hf_token
+                )
+
+    SQLiteStorage.set_project_metadata(project, "space_id", space_id)
+    print(f"* Synced successfully to space: {SPACE_URL.format(space_id=space_id)}")
+
+
+def sync(
+    project: str,
+    space_id: str | None = None,
+    private: bool | None = None,
+    force: bool = False,
+    run_in_background: bool = False,
+) -> str:
+    """
+    Syncs a local Trackio project's database to a Hugging Face Space.
+    If the Space does not exist, it will be created.
+
+    Args:
+        project (`str`): The name of the project to upload.
+        space_id (`str`, *optional*): The ID of the Space to upload to (e.g., `"username/space_id"`).
+            If not provided, checks project metadata first, then generates a random space_id.
+        private (`bool`, *optional*):
+            Whether to make the Space private. If None (default), the repo will be
+            public unless the organization's default is private. This value is ignored
+            if the repo already exists.
+        force (`bool`, *optional*, defaults to `False`):
+            If `True`, overwrite the existing database without prompting for confirmation.
+            If `False`, prompt the user before overwriting an existing database.
+        run_in_background (`bool`, *optional*, defaults to `False`):
+            If `True`, the Space creation and database upload will be run in a background thread.
+            If `False`, all the steps will be run synchronously.
+    Returns:
+        `str`: The Space ID of the synced project.
+    """
+    if space_id is None:
+        space_id = SQLiteStorage.get_space_id(project)
+    if space_id is None:
+        space_id = f"{project}-{get_or_create_project_hash(project)}"
+    space_id, _ = preprocess_space_and_dataset_ids(space_id, None)
+
+    def _do_sync(space_id: str, private: bool | None = None):
+        sync_incremental(project, space_id, private=private, pending_only=False)
+
+    if run_in_background:
+        threading.Thread(target=_do_sync, args=(space_id, private)).start()
+    else:
+        _do_sync(space_id, private)
+    return space_id

trackio/dummy_commit_scheduler.py

@@ -0,0 +1,12 @@
+# A dummy object to fit the interface of huggingface_hub's CommitScheduler
+class DummyCommitSchedulerLock:
+    def __enter__(self):
+        return None
+
+    def __exit__(self, exception_type, exception_value, exception_traceback):
+        pass
+
+
+class DummyCommitScheduler:
+    def __init__(self):
+        self.lock = DummyCommitSchedulerLock()

trackio/frontend/dist/assets/index-0Wf8YNCR.css

@@ -0,0 +1 @@
+:root{--primary-50: #fff7ed;--primary-100: #ffedd5;--primary-200: #fed7aa;--primary-300: #fdba74;--primary-400: #fb923c;--primary-500: #f97316;--primary-600: #ea580c;--primary-700: #c2410c;--primary-800: #9a3412;--primary-900: #7c2d12;--primary-950: #6c2e12;--secondary-50: #eff6ff;--secondary-100: #dbeafe;--secondary-200: #bfdbfe;--secondary-300: #93c5fd;--secondary-400: #60a5fa;--secondary-500: #3b82f6;--secondary-600: #2563eb;--secondary-700: #1d4ed8;--secondary-800: #1e40af;--secondary-900: #1e3a8a;--secondary-950: #1d3660;--neutral-50: #f9fafb;--neutral-100: #f3f4f6;--neutral-200: #e5e7eb;--neutral-300: #d1d5db;--neutral-400: #9ca3af;--neutral-500: #6b7280;--neutral-600: #4b5563;--neutral-700: #374151;--neutral-800: #1f2937;--neutral-900: #111827;--neutral-950: #0b0f19;--size-0-5: 2px;--size-1: 4px;--size-2: 8px;--size-3: 12px;--size-4: 16px;--size-5: 20px;--size-6: 24px;--size-8: 32px;--size-14: 56px;--size-16: 64px;--size-28: 112px;--size-full: 100%;--spacing-xxs: 1px;--spacing-xs: 2px;--spacing-sm: 4px;--spacing-md: 6px;--spacing-lg: 8px;--spacing-xl: 10px;--spacing-xxl: 16px;--radius-xxs: 1px;--radius-xs: 2px;--radius-sm: 3px;--radius-md: 4px;--radius-lg: 5px;--radius-xl: 8px;--radius-xxl: 12px;--text-xxs: 9px;--text-xs: 10px;--text-sm: 12px;--text-md: 14px;--text-lg: 16px;--text-xl: 22px;--text-xxl: 26px;--line-sm: 1.4;--background-fill-primary: white;--background-fill-secondary: var(--neutral-50);--body-text-color: var(--neutral-900);--body-text-color-subdued: var(--neutral-600);--border-color-primary: var(--neutral-200);--color-accent: var(--primary-500);--color-accent-soft: var(--primary-50);--shadow-drop: rgba(0, 0, 0, .05) 0px 1px 2px 0px;--shadow-drop-lg: 0 1px 3px 0 rgb(0 0 0 / .1), 0 1px 2px -1px rgb(0 0 0 / .1);--shadow-inset: rgba(0, 0, 0, .05) 0px 2px 4px 0px inset;--shadow-spread: 3px;--block-title-text-color: var(--neutral-500);--block-title-text-size: var(--text-md);--block-title-text-weight: 400;--block-info-text-color: var(--body-text-color-subdued);--block-info-text-size: var(--text-sm);--input-background-fill: white;--input-background-fill-focus: var(--primary-500);--input-border-color: var(--border-color-primary);--input-border-color-focus: var(--primary-300);--input-border-width: 1px;--input-padding: var(--spacing-xl);--input-placeholder-color: var(--neutral-400);--input-radius: var(--radius-lg);--input-shadow: 0 0 0 var(--shadow-spread) transparent, var(--shadow-inset);--input-shadow-focus: 0 0 0 var(--shadow-spread) var(--primary-50), var(--shadow-inset);--input-text-size: var(--text-md);--checkbox-background-color: var(--background-fill-primary);--checkbox-background-color-focus: var(--checkbox-background-color);--checkbox-background-color-hover: var(--checkbox-background-color);--checkbox-background-color-selected: var(--primary-600);--checkbox-border-color: var(--neutral-300);--checkbox-border-color-focus: var(--primary-500);--checkbox-border-color-hover: var(--neutral-300);--checkbox-border-color-selected: var(--primary-600);--checkbox-border-radius: var(--radius-sm);--checkbox-border-width: var(--input-border-width);--checkbox-label-gap: var(--spacing-lg);--checkbox-label-padding: var(--spacing-md) calc(2 * var(--spacing-md));--checkbox-label-text-size: var(--text-md);--checkbox-shadow: var(--input-shadow);--checkbox-check: url("data:image/svg+xml,%3csvg viewBox='0 0 16 16' fill='white' xmlns='http://www.w3.org/2000/svg'%3e%3cpath d='M12.207 4.793a1 1 0 010 1.414l-5 5a1 1 0 01-1.414 0l-2-2a1 1 0 011.414-1.414L6.5 9.086l4.293-4.293a1 1 0 011.414 0z'/%3e%3c/svg%3e");--slider-color: var(--primary-500);--container-radius: var(--radius-lg);--layer-top: 9999}.navbar.svelte-d8j1hi{display:flex;align-items:stretch;border-bottom:1px solid var(--border-color-primary, #e5e7eb);background:var(--background-fill-primary, white);padding:0;flex-shrink:0;min-height:44px}.nav-spacer.svelte-d8j1hi{flex:1}.nav-tabs.svelte-d8j1hi{display:flex;gap:0;padding-right:16px}.nav-link.svelte-d8j1hi{padding:10px 16px;border:none;background:none;color:var(--body-text-color-subdued, #6b7280);font-size:var(--text-md, 14px);cursor:pointer;white-space:nowrap;border-bottom:2px solid transparent;transition:color .15s;font-weight:400}.nav-link.svelte-d8j1hi:hover{color:var(--body-text-color, #1f2937)}.nav-link.active.svelte-d8j1hi{color:var(--body-text-color, #1f2937);border-bottom-color:var(--body-text-color, #1f2937);font-weight:500}.checkbox-group.svelte-17gmtkf{display:flex;flex-direction:column}.checkbox-item.svelte-17gmtkf{display:flex;align-items:center;gap:8px;padding:3px 0;cursor:pointer;font-size:13px}.checkbox-item.svelte-17gmtkf input[type=checkbox]:where(.svelte-17gmtkf){-moz-appearance:none;appearance:none;-webkit-appearance:none;width:16px;height:16px;margin:0;border:1px solid var(--checkbox-border-color, #d1d5db);border-radius:var(--checkbox-border-radius, 4px);background-color:var(--checkbox-background-color, white);box-shadow:var(--checkbox-shadow);cursor:pointer;flex-shrink:0;transition:background-color .15s,border-color .15s}.checkbox-item.svelte-17gmtkf input[type=checkbox]:where(.svelte-17gmtkf):checked{background-image:var(--checkbox-check);background-color:var(--checkbox-background-color-selected, #f97316);border-color:var(--checkbox-border-color-selected, #f97316)}.checkbox-item.svelte-17gmtkf input[type=checkbox]:where(.svelte-17gmtkf):hover{border-color:var(--checkbox-border-color-hover, #d1d5db)}.color-dot.svelte-17gmtkf{width:10px;height:10px;border-radius:50%;flex-shrink:0}.run-name.svelte-17gmtkf{overflow:hidden;text-overflow:ellipsis;white-space:nowrap;color:var(--body-text-color, #1f2937)}.radio-group.svelte-1sinbnb{display:flex;flex-direction:column;max-height:300px;overflow-y:auto;margin-top:8px}.run-row.svelte-1sinbnb{display:flex;align-items:center;gap:8px;padding:3px 0;cursor:pointer;font-size:13px}.run-row.svelte-1sinbnb input[type=radio]:where(.svelte-1sinbnb){-moz-appearance:none;appearance:none;-webkit-appearance:none;width:16px;height:16px;margin:0;border:1px solid var(--checkbox-border-color, #d1d5db);border-radius:50%;background-color:var(--checkbox-background-color, white);cursor:pointer;flex-shrink:0;transition:border-color .15s,box-shadow .15s}.run-row.svelte-1sinbnb input[type=radio]:where(.svelte-1sinbnb):checked{border-color:var(--checkbox-border-color-selected, #f97316);box-shadow:inset 0 0 0 3px var(--checkbox-background-color-selected, #f97316)}.color-dot.svelte-1sinbnb{width:10px;height:10px;border-radius:50%;flex-shrink:0}.run-name.svelte-1sinbnb{overflow:hidden;text-overflow:ellipsis;white-space:nowrap;color:var(--body-text-color, #1f2937)}.dropdown-container.svelte-kgylqb{width:100%}.label.svelte-kgylqb{display:block;font-size:13px;font-weight:500;color:var(--body-text-color-subdued, #6b7280);margin-bottom:6px}.info.svelte-kgylqb{display:block;font-size:12px;color:var(--body-text-color-subdued, #9ca3af);margin-bottom:4px}.wrap.svelte-kgylqb{position:relative;border-radius:var(--input-radius, 8px);background:var(--input-background-fill, white);border:1px solid var(--border-color-primary, #e5e7eb);transition:border-color .15s,box-shadow .15s}.wrap.focused.svelte-kgylqb{border-color:var(--input-border-color-focus, #fdba74);box-shadow:0 0 0 2px var(--primary-50, #fff7ed)}.wrap-inner.svelte-kgylqb{display:flex;position:relative;align-items:center;padding:0 10px}.secondary-wrap.svelte-kgylqb{display:flex;flex:1;align-items:center}input.svelte-kgylqb{margin:0;outline:none;border:none;background:inherit;width:100%;color:var(--body-text-color, #1f2937);font-size:13px;font-family:inherit;padding:7px 0}input.svelte-kgylqb::placeholder{color:var(--input-placeholder-color, #9ca3af)}input[readonly].svelte-kgylqb{cursor:pointer}.icon-wrap.svelte-kgylqb{color:var(--body-text-color-subdued, #9ca3af);width:16px;flex-shrink:0;pointer-events:none}.options.svelte-kgylqb{position:fixed;z-index:var(--layer-top, 9999);margin:0;padding:4px 0;box-shadow:0 4px 12px #0000001f;border-radius:var(--input-radius, 8px);border:1px solid var(--border-color-primary, #e5e7eb);background:var(--background-fill-primary, white);min-width:fit-content;overflow:auto;color:var(--body-text-color, #1f2937);list-style:none}.item.svelte-kgylqb{display:flex;cursor:pointer;padding:6px 10px;font-size:13px;word-break:break-word}.item.svelte-kgylqb:hover,.item.active.svelte-kgylqb{background:var(--background-fill-secondary, #f9fafb)}.item.selected.svelte-kgylqb{font-weight:500}.check-mark.svelte-kgylqb{padding-right:6px;min-width:16px;font-size:12px}.check-mark.hide.svelte-kgylqb{visibility:hidden}.checkbox-container.svelte-oj84db{display:flex;align-items:center;gap:8px;cursor:pointer;margin:8px 0}.label-text.svelte-oj84db{color:var(--body-text-color, #1f2937);font-size:13px;line-height:1.4}input[type=checkbox].svelte-oj84db{--ring-color: transparent;position:relative;-moz-appearance:none;appearance:none;-webkit-appearance:none;width:16px;height:16px;box-shadow:var(--checkbox-shadow);border:1px solid var(--checkbox-border-color, #d1d5db);border-radius:var(--checkbox-border-radius, 4px);background-color:var(--checkbox-background-color, white);flex-shrink:0;cursor:pointer;transition:background-color .15s,border-color .15s}input[type=checkbox].svelte-oj84db:checked,input[type=checkbox].svelte-oj84db:checked:hover,input[type=checkbox].svelte-oj84db:checked:focus{background-image:var(--checkbox-check);background-color:var(--checkbox-background-color-selected, #f97316);border-color:var(--checkbox-border-color-selected, #f97316)}input[type=checkbox].svelte-oj84db:hover{border-color:var(--checkbox-border-color-hover, #d1d5db);background-color:var(--checkbox-background-color-hover, white)}input[type=checkbox].svelte-oj84db:focus{border-color:var(--checkbox-border-color-focus, #f97316);background-color:var(--checkbox-background-color-focus, white);outline:none}.slider-wrap.svelte-wei6ev{display:flex;flex-direction:column;width:100%}.head.svelte-wei6ev{margin-bottom:4px;display:flex;justify-content:space-between;align-items:center;width:100%}.label.svelte-wei6ev{flex:1;font-size:13px;font-weight:500;color:var(--body-text-color-subdued, #6b7280)}.info.svelte-wei6ev{display:block;font-size:12px;color:var(--body-text-color-subdued, #9ca3af);margin-bottom:4px}.slider-input-container.svelte-wei6ev{display:flex;align-items:center;gap:6px}input[type=range].svelte-wei6ev{-webkit-appearance:none;-moz-appearance:none;appearance:none;width:100%;cursor:pointer;outline:none;border-radius:var(--radius-xl, 12px);min-width:var(--size-28, 112px);background:transparent}input[type=range].svelte-wei6ev::-webkit-slider-runnable-track{height:6px;border-radius:var(--radius-xl, 12px);background:linear-gradient(to right,var(--slider-color, #f97316) var(--range_progress, 50%),var(--neutral-200, #e5e7eb) var(--range_progress, 50%))}input[type=range].svelte-wei6ev::-webkit-slider-thumb{-webkit-appearance:none;-moz-appearance:none;appearance:none;height:16px;width:16px;background-color:var(--slider-color, #f97316);border:2px solid var(--background-fill-primary, white);border-radius:50%;margin-top:-5px;box-shadow:0 0 0 1px var(--border-color-primary, rgba(0, 0, 0, .08)),0 1px 3px #0003}input[type=range].svelte-wei6ev::-moz-range-track{height:6px;background:var(--neutral-200, #e5e7eb);border-radius:var(--radius-xl, 12px)}input[type=range].svelte-wei6ev::-moz-range-thumb{-webkit-appearance:none;-moz-appearance:none;appearance:none;height:16px;width:16px;background-color:var(--slider-color, #f97316);border:2px solid var(--background-fill-primary, white);border-radius:50%;box-shadow:0 0 0 1px var(--border-color-primary, rgba(0, 0, 0, .08)),0 1px 3px #0003}input[type=range].svelte-wei6ev::-moz-range-progress{height:6px;background-color:var(--slider-color, #f97316);border-radius:var(--radius-xl, 12px)}.bound.svelte-wei6ev{font-size:11px;color:var(--body-text-color-subdued, #9ca3af);min-width:12px;text-align:center}.textbox-container.svelte-6yncpg{width:100%}.label.svelte-6yncpg{display:block;font-size:13px;font-weight:500;color:var(--body-text-color-subdued, #6b7280);margin-bottom:6px}.info.svelte-6yncpg{display:block;font-size:12px;color:var(--body-text-color-subdued, #9ca3af);margin-bottom:4px}.input-wrap.svelte-6yncpg{border-radius:var(--input-radius, 8px);background:var(--input-background-fill, white);border:1px solid var(--border-color-primary, #e5e7eb);transition:border-color .15s,box-shadow .15s}.input-wrap.svelte-6yncpg:focus-within{border-color:var(--input-border-color-focus, #fdba74);box-shadow:0 0 0 2px var(--primary-50, #fff7ed)}input.svelte-6yncpg{width:100%;padding:7px 10px;outline:none;border:none;background:transparent;color:var(--body-text-color, #1f2937);font-size:13px;font-family:inherit;border-radius:var(--input-radius, 8px)}input.svelte-6yncpg::placeholder{color:var(--input-placeholder-color, #9ca3af)}.sidebar.svelte-181dlmc{width:290px;min-width:290px;background:var(--background-fill-primary, white);border-right:1px solid var(--border-color-primary, #e5e7eb);display:flex;flex-direction:column;position:relative;overflow:hidden;transition:width .2s,min-width .2s}.sidebar.collapsed.svelte-181dlmc{width:40px;min-width:40px}.toggle-btn.svelte-181dlmc{position:absolute;top:12px;right:8px;z-index:10;border:none;background:none;color:var(--body-text-color-subdued, #9ca3af);cursor:pointer;padding:4px;display:flex;align-items:center;justify-content:center;border-radius:var(--radius-sm, 4px);transition:color .15s,background-color .15s}.toggle-btn.svelte-181dlmc:hover{color:var(--body-text-color, #1f2937);background-color:var(--background-fill-secondary, #f9fafb)}.sidebar-content.svelte-181dlmc{padding:16px;flex:1;min-height:0;display:flex;flex-direction:column}.sidebar-scroll.svelte-181dlmc{overflow-y:auto;flex:1;min-height:0}.oauth-footer.svelte-181dlmc{flex-shrink:0;margin-top:12px;padding-top:12px;border-top:1px solid var(--border-color-primary, #e5e7eb)}.oauth-line.svelte-181dlmc{margin:0;font-size:12px;line-height:1.4;color:var(--body-text-color-subdued, #6b7280)}.oauth-warn.svelte-181dlmc{color:var(--body-text-color, #92400e)}.hf-login-btn.svelte-181dlmc{display:inline-flex;align-items:center;justify-content:center;gap:8px;width:100%;padding:8px 12px;font-size:13px;font-weight:600;color:#fff;background:#141c2e;border-radius:var(--radius-lg, 8px);text-decoration:none;border:none;cursor:pointer;box-sizing:border-box}.hf-login-btn.svelte-181dlmc:hover{background:#283042}.hf-logo.svelte-181dlmc{width:20px;height:20px;flex-shrink:0}.oauth-hint.svelte-181dlmc{margin:8px 0 0;font-size:11px;line-height:1.35;color:var(--body-text-color-subdued, #9ca3af)}.oauth-signed-in.svelte-181dlmc{margin:0;font-size:12px;color:var(--body-text-color-subdued, #6b7280)}.oauth-logout.svelte-181dlmc{font-size:12px;color:var(--body-text-color-subdued, #9ca3af);text-decoration:none;cursor:pointer}.oauth-logout.svelte-181dlmc:hover{text-decoration:underline;color:var(--body-text-color, #1f2937)}.logo-section.svelte-181dlmc{margin-bottom:20px}.logo.svelte-181dlmc{width:80%;max-width:200px}.section.svelte-181dlmc{margin-bottom:18px}.section-label.svelte-181dlmc{font-size:13px;font-weight:500;color:var(--body-text-color-subdued, #6b7280)}.locked-project.svelte-181dlmc{margin-top:4px;font-size:13px;font-weight:500;color:var(--body-text-color, #1f2937);padding:8px 10px;border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-md, 6px);background:var(--background-fill-secondary, #f9fafb)}.runs-header.svelte-181dlmc{display:flex;align-items:center;justify-content:space-between;margin-bottom:6px}.latest-toggle.svelte-181dlmc{display:flex;align-items:center;gap:6px;font-size:12px;color:var(--body-text-color-subdued, #6b7280);cursor:pointer}.latest-toggle.svelte-181dlmc input[type=checkbox]:where(.svelte-181dlmc){-moz-appearance:none;appearance:none;-webkit-appearance:none;width:16px;height:16px;margin:0;border:1px solid var(--checkbox-border-color, #d1d5db);border-radius:var(--checkbox-border-radius, 4px);background-color:var(--checkbox-background-color, white);box-shadow:var(--checkbox-shadow);cursor:pointer;flex-shrink:0;transition:background-color .15s,border-color .15s}.latest-toggle.svelte-181dlmc input[type=checkbox]:where(.svelte-181dlmc):checked{background-image:var(--checkbox-check);background-color:var(--checkbox-background-color-selected, #f97316);border-color:var(--checkbox-border-color-selected, #f97316)}.checkbox-list.svelte-181dlmc{max-height:300px;overflow-y:auto;margin-top:8px}.alert-panel.svelte-x5aqew{position:fixed;bottom:16px;right:16px;width:380px;max-height:400px;background:var(--background-fill-primary, white);border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-lg, 8px);box-shadow:var(--shadow-drop-lg);z-index:1000;overflow:hidden;display:flex;flex-direction:column}.alert-header.svelte-x5aqew{padding:10px 12px;border-bottom:1px solid var(--border-color-primary, #e5e7eb);display:flex;align-items:center;justify-content:space-between}.alert-title.svelte-x5aqew{font-size:13px;font-weight:600;color:var(--body-text-color, #1f2937)}.filter-pills.svelte-x5aqew{display:flex;gap:4px}.pill.svelte-x5aqew{border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-xxl, 22px);padding:2px 8px;font-size:11px;background:var(--background-fill-secondary, #f9fafb);color:var(--body-text-color-subdued, #6b7280);cursor:pointer}.pill.active.svelte-x5aqew{background:var(--color-accent, #f97316);color:#fff;border-color:var(--color-accent, #f97316)}.alert-list.svelte-x5aqew{overflow-y:auto;flex:1}.alert-item.svelte-x5aqew{border-bottom:1px solid var(--neutral-100, #f3f4f6)}.alert-row.svelte-x5aqew{display:flex;align-items:center;gap:8px;width:100%;padding:8px 12px;border:none;background:none;text-align:left;cursor:pointer;font-size:var(--text-sm, 12px)}.alert-row.svelte-x5aqew:hover{background:var(--background-fill-secondary, #f9fafb)}.alert-text.svelte-x5aqew{flex:1;color:var(--body-text-color, #1f2937)}.alert-meta.svelte-x5aqew{font-size:var(--text-xs, 10px);color:var(--body-text-color-subdued, #9ca3af);white-space:nowrap}.alert-detail.svelte-x5aqew{padding:4px 12px 8px 32px;font-size:var(--text-sm, 12px);color:var(--body-text-color-subdued, #6b7280)}.plot-container.svelte-9thu1j{min-width:350px;flex:1;background:var(--background-fill-primary, white);border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-lg, 8px);padding:12px;overflow:hidden;position:relative}.plot-container[draggable=true].svelte-9thu1j{cursor:grab}.plot-container[draggable=true].svelte-9thu1j:active{cursor:grabbing}.hidden-plot.svelte-9thu1j{visibility:hidden;height:0;padding:0;margin:0;border:none;overflow:hidden;pointer-events:none}.drag-handle.svelte-9thu1j{position:absolute;top:8px;left:8px;color:var(--body-text-color-subdued, #9ca3af);opacity:0;transition:opacity .15s;z-index:5}.plot-container.svelte-9thu1j:hover .drag-handle:where(.svelte-9thu1j){opacity:.5}.drag-handle.svelte-9thu1j:hover{opacity:1!important}.plot-toolbar.svelte-9thu1j{position:absolute;top:8px;right:8px;display:flex;gap:4px;z-index:5;opacity:0;transition:opacity .15s}.plot-container.svelte-9thu1j:hover .plot-toolbar:where(.svelte-9thu1j){opacity:1}.toolbar-btn.svelte-9thu1j{border:1px solid var(--border-color-primary, #e5e7eb);background:var(--background-fill-primary, white);color:var(--body-text-color-subdued, #6b7280);cursor:pointer;padding:4px 6px;border-radius:var(--radius-sm, 4px);display:flex;align-items:center;justify-content:center}.toolbar-btn.svelte-9thu1j:hover{background:var(--neutral-100, #f3f4f6);color:var(--body-text-color, #1f2937)}.plot-chart-wrap.svelte-9thu1j{position:relative;width:100%}.plot-chart-wrap--fs.svelte-9thu1j{flex:1;min-height:0;display:flex;flex-direction:column}.reset-zoom-btn.svelte-9thu1j{position:absolute;bottom:1px;right:1px;z-index:6;display:inline-flex;align-items:center;justify-content:center;margin:0;min-width:52px;padding:5px 12px 5px 10px;border:none;border-radius:4px;background:transparent;color:var(--body-text-color-subdued, #334155);cursor:pointer;opacity:.92;transform:translateY(6px);transition:opacity .15s ease,color .15s ease,background .15s ease;box-shadow:none}.reset-zoom-btn.svelte-9thu1j:hover{opacity:1;color:var(--body-text-color, #0f172a);background:var(--background-fill-secondary, rgba(226, 232, 240, .85));transform:translateY(6px)}.reset-zoom-btn.svelte-9thu1j svg:where(.svelte-9thu1j){display:block;flex-shrink:0;filter:drop-shadow(0 0 .5px rgba(255,255,255,.95))}.plot.svelte-9thu1j{width:100%}.plot.svelte-9thu1j .vega-embed{width:100%!important}.plot.svelte-9thu1j .vega-embed summary{display:none}.fullscreen-host.svelte-9thu1j{position:fixed;top:0;right:0;bottom:0;left:0;z-index:10000;box-sizing:border-box;display:flex;flex-direction:column;background:var(--background-fill-primary, white);padding:12px;gap:8px;pointer-events:auto}.fullscreen-host.svelte-9thu1j:fullscreen{width:100%;height:100%}.fullscreen-host.svelte-9thu1j:-webkit-full-screen{width:100%;height:100%}.fullscreen-toolbar.svelte-9thu1j{flex-shrink:0;display:flex;justify-content:flex-end;gap:4px;z-index:5}.fullscreen-chart-wrap.svelte-9thu1j{flex:1;min-height:0;display:flex;flex-direction:column}.fullscreen-legend.svelte-9thu1j{flex-shrink:0}.fullscreen-plot.svelte-9thu1j{flex:1;min-height:0;width:100%;overflow:hidden}.fullscreen-plot.svelte-9thu1j .vega-embed{width:100%!important;height:100%!important;min-height:0;display:flex;flex-direction:column}.fullscreen-plot.svelte-9thu1j .vega-embed .vega-view{flex:1;min-height:0}.fullscreen-plot.svelte-9thu1j .vega-embed summary{display:none}.custom-legend.svelte-9thu1j{display:flex;align-items:center;justify-content:center;gap:12px;padding:6px 0 0;flex-wrap:wrap}.legend-title.svelte-9thu1j{font-size:11px;color:var(--body-text-color-subdued, #6b7280);font-weight:600}.legend-item.svelte-9thu1j{display:flex;align-items:center;gap:4px}.legend-dot.svelte-9thu1j{width:10px;height:10px;border-radius:50%;flex-shrink:0}.legend-label.svelte-9thu1j{font-size:11px;color:var(--body-text-color-subdued, #6b7280)}.accordion.svelte-1jep0a{margin-bottom:12px;border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-lg, 8px);background:var(--background-fill-primary, white);overflow:hidden}.accordion-hidden.svelte-1jep0a{margin-bottom:8px}.accordion-header.svelte-1jep0a{display:flex;align-items:center;gap:8px;width:100%;padding:10px 14px;border:none;background:var(--background-fill-primary, white);color:var(--body-text-color, #1f2937);font-size:var(--text-md, 14px);font-weight:600;cursor:pointer;text-align:left}.accordion-header.svelte-1jep0a:hover{background:var(--background-fill-secondary, #f9fafb)}.arrow.svelte-1jep0a{font-size:14px;transition:transform .15s;color:var(--body-text-color, #1f2937);display:inline-block}.arrow.svelte-1jep0a:not(.rotated){transform:rotate(-90deg)}.accordion-body.svelte-1jep0a{padding:0 14px 14px}.trackio-loading.svelte-1kc6b2l{display:flex;align-items:center;justify-content:center;width:100%;min-height:min(70vh,640px);padding:32px 24px;box-sizing:border-box;background:transparent}.logo-stack.svelte-1kc6b2l{position:relative;width:min(100%,200px);max-width:min(92vw,200px);line-height:0;background:transparent;isolation:isolate}.logo-base.svelte-1kc6b2l{display:block;background:transparent}.logo-img.svelte-1kc6b2l{width:100%;height:auto;display:block;background:transparent}.logo-overlay.svelte-1kc6b2l{position:absolute;left:0;top:0;width:100%;animation:svelte-1kc6b2l-trackio-logo-sweep 4s linear infinite;pointer-events:none;background:transparent}.logo-overlay.svelte-1kc6b2l .logo-img:where(.svelte-1kc6b2l){width:100%;height:auto;object-position:left center}.logo-img--gray.svelte-1kc6b2l{filter:grayscale(1)}@keyframes svelte-1kc6b2l-trackio-logo-sweep{0%{clip-path:inset(0 0 0 0)}50%{clip-path:inset(0 0 0 100%)}to{clip-path:inset(0 0 0 0)}}@media(prefers-reduced-motion:reduce){.logo-overlay.svelte-1kc6b2l{display:none}}.sr-only.svelte-1kc6b2l{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);white-space:nowrap;border:0}.metrics-page.svelte-2bul55{padding:20px 24px;overflow-y:auto;flex:1;min-height:0}.plot-grid.svelte-2bul55{display:flex;flex-wrap:wrap;gap:16px}.empty-state.svelte-2bul55{max-width:640px;padding:40px 24px;color:var(--body-text-color, #1f2937)}.empty-state.svelte-2bul55 h2:where(.svelte-2bul55){margin:0 0 8px;font-size:20px;font-weight:700}.empty-state.svelte-2bul55 p:where(.svelte-2bul55){margin:12px 0 8px;color:var(--body-text-color-subdued, #6b7280)}.empty-state.svelte-2bul55 pre:where(.svelte-2bul55){background:var(--background-fill-secondary, #f9fafb);padding:16px;border-radius:var(--radius-lg, 8px);border:1px solid var(--border-color-primary, #e5e7eb);font-size:13px;overflow-x:auto}.empty-state.svelte-2bul55 code:where(.svelte-2bul55){background:var(--background-fill-secondary, #f0f0f0);padding:1px 5px;border-radius:var(--radius-sm, 4px);font-size:13px}.empty-state.svelte-2bul55 pre:where(.svelte-2bul55) code:where(.svelte-2bul55){background:none;padding:0}.system-page.svelte-nv5os4{padding:20px 24px;overflow-y:auto;flex:1;min-height:0}.plot-grid.svelte-nv5os4{display:flex;flex-wrap:wrap;gap:12px}.empty-state.svelte-nv5os4{max-width:640px;padding:40px 24px;color:var(--body-text-color, #1f2937)}.empty-state.svelte-nv5os4 h2:where(.svelte-nv5os4){margin:0 0 8px;font-size:20px;font-weight:700}.empty-state.svelte-nv5os4 p:where(.svelte-nv5os4){margin:12px 0 8px;color:var(--body-text-color-subdued, #6b7280)}.empty-state.svelte-nv5os4 pre:where(.svelte-nv5os4){background:var(--background-fill-secondary, #f9fafb);padding:16px;border-radius:var(--radius-lg, 8px);border:1px solid var(--border-color-primary, #e5e7eb);font-size:13px;overflow-x:auto}.empty-state.svelte-nv5os4 ul:where(.svelte-nv5os4){list-style:disc;padding-left:20px;margin:4px 0 0}.empty-state.svelte-nv5os4 li:where(.svelte-nv5os4){margin:4px 0;color:var(--body-text-color, #1f2937)}.empty-state.svelte-nv5os4 code:where(.svelte-nv5os4){background:var(--background-fill-secondary, #f0f0f0);padding:1px 5px;border-radius:var(--radius-sm, 4px);font-size:13px}.empty-state.svelte-nv5os4 pre:where(.svelte-nv5os4) code:where(.svelte-nv5os4){background:none;padding:0}.table-container.svelte-1cp60rw{display:flex;flex-direction:column;gap:var(--size-2, 8px);position:relative}.header-row.svelte-1cp60rw{display:flex;justify-content:flex-end;align-items:center;min-height:var(--size-6, 24px);width:100%}.header-row.svelte-1cp60rw .label:where(.svelte-1cp60rw){flex:1;margin:0;color:var(--block-label-text-color, var(--neutral-500, #6b7280));font-size:var(--block-label-text-size, 12px);line-height:var(--line-sm, 1.4)}.table-wrap.svelte-1cp60rw{position:relative;overflow:auto;border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--table-radius, var(--radius-lg, 8px))}table.svelte-1cp60rw{width:100%;table-layout:auto;color:var(--body-text-color, #1f2937);font-size:var(--input-text-size, 14px);line-height:var(--line-sm, 1.4);border-spacing:0;border-collapse:separate}thead.svelte-1cp60rw{position:sticky;top:0;z-index:5;box-shadow:var(--shadow-drop, rgba(0,0,0,.05) 0px 1px 2px 0px)}th.svelte-1cp60rw{padding:0;background:var(--table-even-background-fill, white);border-right-width:0px;border-left-width:1px;border-bottom-width:1px;border-style:solid;border-color:var(--border-color-primary, #e5e7eb);text-align:left;cursor:pointer;-webkit-user-select:none;user-select:none}th.first.svelte-1cp60rw{border-left-width:0;border-top-left-radius:var(--table-radius, var(--radius-lg, 8px))}th.last.svelte-1cp60rw{border-top-right-radius:var(--table-radius, var(--radius-lg, 8px))}.th-inner.svelte-1cp60rw{padding:var(--size-2, 8px);display:flex;align-items:center;gap:4px;font-weight:600;font-size:var(--text-sm, 12px);white-space:nowrap}.sort-arrow.svelte-1cp60rw{font-size:10px;color:var(--body-text-color-subdued, #9ca3af)}td.svelte-1cp60rw{padding:var(--size-2, 8px);border-right-width:0px;border-left-width:1px;border-bottom-width:1px;border-style:solid;border-color:var(--border-color-primary, #e5e7eb);font-size:var(--text-sm, 12px)}td.first.svelte-1cp60rw{border-left-width:0}tr.svelte-1cp60rw{background:var(--table-even-background-fill, white);border-bottom:1px solid var(--border-color-primary, #e5e7eb);text-align:left}tr.row-odd.svelte-1cp60rw{background:var(--table-odd-background-fill, var(--neutral-50, #f9fafb))}tr.selected.svelte-1cp60rw{background:var(--color-accent-soft, var(--primary-50, #fff7ed))}tr.svelte-1cp60rw:last-child td.first:where(.svelte-1cp60rw){border-bottom-left-radius:var(--table-radius, var(--radius-lg, 8px))}tr.svelte-1cp60rw:last-child td.last:where(.svelte-1cp60rw){border-bottom-right-radius:var(--table-radius, var(--radius-lg, 8px))}.check-col.svelte-1cp60rw{width:40px;text-align:center;padding:var(--size-2, 8px);border-left-width:0}.check-col.svelte-1cp60rw input[type=checkbox]:where(.svelte-1cp60rw){-moz-appearance:none;appearance:none;-webkit-appearance:none;width:16px;height:16px;border:1px solid var(--checkbox-border-color, #d1d5db);border-radius:var(--checkbox-border-radius, 4px);background-color:var(--checkbox-background-color, white);cursor:pointer;flex-shrink:0;transition:background-color .15s,border-color .15s}.check-col.svelte-1cp60rw input[type=checkbox]:where(.svelte-1cp60rw):checked{background-image:var(--checkbox-check);background-color:var(--checkbox-background-color-selected, #2563eb);border-color:var(--checkbox-border-color-selected, #2563eb)}.media-page.svelte-outb32{padding:20px 24px;overflow-y:auto;flex:1}.section-title.svelte-outb32{font-size:var(--text-lg, 16px);font-weight:600;color:var(--body-text-color, #1f2937);margin:16px 0 8px}.gallery.svelte-outb32{display:grid;grid-template-columns:repeat(auto-fill,minmax(200px,1fr));gap:12px}.gallery-item.svelte-outb32{border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-lg, 8px);overflow:hidden;background:var(--background-fill-secondary, #f9fafb)}.gallery-item.svelte-outb32 img:where(.svelte-outb32),.gallery-item.svelte-outb32 video:where(.svelte-outb32){width:100%;display:block}.caption.svelte-outb32{padding:4px 8px;font-size:var(--text-sm, 12px);color:var(--body-text-color-subdued, #9ca3af)}.step-label.svelte-outb32{padding:4px 8px;font-size:var(--text-xs, 10px);color:var(--body-text-color-subdued, #9ca3af)}.audio-list.svelte-outb32{display:flex;flex-direction:column;gap:8px}.audio-item.svelte-outb32{display:flex;align-items:center;gap:12px;padding:8px;border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-lg, 8px)}.audio-label.svelte-outb32{font-size:var(--text-sm, 12px);color:var(--body-text-color-subdued, #9ca3af);min-width:120px}.table-section.svelte-outb32{margin-bottom:16px}.empty-state.svelte-outb32{max-width:640px;padding:40px 24px;color:var(--body-text-color, #1f2937)}.empty-state.svelte-outb32 h2:where(.svelte-outb32){margin:0 0 8px;font-size:20px;font-weight:700}.empty-state.svelte-outb32 p:where(.svelte-outb32){margin:12px 0 8px;color:var(--body-text-color-subdued, #6b7280)}.empty-state.svelte-outb32 pre:where(.svelte-outb32){background:var(--background-fill-secondary, #f9fafb);padding:16px;border-radius:var(--radius-lg, 8px);border:1px solid var(--border-color-primary, #e5e7eb);font-size:13px;overflow-x:auto}.empty-state.svelte-outb32 code:where(.svelte-outb32){background:var(--background-fill-secondary, #f0f0f0);padding:1px 5px;border-radius:var(--radius-sm, 4px);font-size:13px}.empty-state.svelte-outb32 pre:where(.svelte-outb32) code:where(.svelte-outb32){background:none;padding:0}.reports-page.svelte-iufsej{padding:20px 24px;overflow-y:auto;flex:1}.controls.svelte-iufsej{display:flex;gap:16px;margin-bottom:16px;flex-wrap:wrap;align-items:flex-end}.control.svelte-iufsej{min-width:200px}.block-title.svelte-iufsej{display:block;font-size:var(--block-title-text-size, 14px);font-weight:var(--block-title-text-weight, 400);color:var(--block-title-text-color, #6b7280);margin-bottom:var(--spacing-lg, 8px)}.filter-pills.svelte-iufsej{display:flex;gap:4px}.pill.svelte-iufsej{border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-xxl, 22px);padding:4px 12px;font-size:var(--text-sm, 12px);background:var(--background-fill-secondary, #f9fafb);color:var(--body-text-color-subdued, #6b7280);cursor:pointer;transition:background-color .15s,color .15s}.pill.svelte-iufsej:hover{background:var(--neutral-100, #f3f4f6)}.pill.active.svelte-iufsej{background:var(--color-accent, #f97316);color:#fff;border-color:var(--color-accent, #f97316)}.empty-state.svelte-iufsej{max-width:640px;padding:40px 24px;color:var(--body-text-color, #1f2937)}.empty-state.svelte-iufsej h2:where(.svelte-iufsej){margin:0 0 8px;font-size:20px;font-weight:700}.empty-state.svelte-iufsej p:where(.svelte-iufsej){margin:12px 0 8px;color:var(--body-text-color-subdued, #6b7280)}.empty-state.svelte-iufsej pre:where(.svelte-iufsej){background:var(--background-fill-secondary, #f9fafb);padding:16px;border-radius:var(--radius-lg, 8px);border:1px solid var(--border-color-primary, #e5e7eb);font-size:13px;overflow-x:auto}.empty-state.svelte-iufsej code:where(.svelte-iufsej){background:var(--background-fill-secondary, #f0f0f0);padding:1px 5px;border-radius:var(--radius-sm, 4px);font-size:13px}.empty-state.svelte-iufsej pre:where(.svelte-iufsej) code:where(.svelte-iufsej){background:none;padding:0}.alerts-table.svelte-iufsej{width:100%;border-collapse:collapse;font-size:var(--text-md, 14px)}.alerts-table.svelte-iufsej th:where(.svelte-iufsej){text-align:left;padding:8px 12px;border-bottom:2px solid var(--border-color-primary, #e5e7eb);color:var(--body-text-color-subdued, #6b7280);font-weight:600;font-size:var(--text-sm, 12px);text-transform:uppercase;letter-spacing:.05em}.alerts-table.svelte-iufsej td:where(.svelte-iufsej){padding:8px 12px;border-bottom:1px solid var(--border-color-primary, #e5e7eb);color:var(--body-text-color, #1f2937)}.alerts-table.svelte-iufsej tbody:where(.svelte-iufsej) tr:where(.svelte-iufsej):nth-child(odd){background:var(--table-odd-background-fill, var(--background-fill-primary, white))}.alerts-table.svelte-iufsej tbody:where(.svelte-iufsej) tr:where(.svelte-iufsej):nth-child(2n){background:var(--table-even-background-fill, var(--background-fill-secondary, #f9fafb))}.alerts-table.svelte-iufsej tr:where(.svelte-iufsej):hover{background:var(--background-fill-secondary, #f3f4f6)}.runs-page.svelte-1yb6d54{padding:20px 24px;overflow-y:auto;flex:1}.empty-state.svelte-1yb6d54{max-width:640px;padding:40px 24px;color:var(--body-text-color, #1f2937)}.empty-state.svelte-1yb6d54 h2:where(.svelte-1yb6d54){margin:0 0 8px;font-size:20px;font-weight:700}.empty-state.svelte-1yb6d54 p:where(.svelte-1yb6d54){margin:12px 0 8px;color:var(--body-text-color-subdued, #6b7280)}.empty-state.svelte-1yb6d54 pre:where(.svelte-1yb6d54){background:var(--background-fill-secondary, #f9fafb);padding:16px;border-radius:var(--radius-lg, 8px);border:1px solid var(--border-color-primary, #e5e7eb);font-size:13px;overflow-x:auto}.empty-state.svelte-1yb6d54 code:where(.svelte-1yb6d54){background:var(--background-fill-secondary, #f0f0f0);padding:1px 5px;border-radius:var(--radius-sm, 4px);font-size:13px}.empty-state.svelte-1yb6d54 pre:where(.svelte-1yb6d54) code:where(.svelte-1yb6d54){background:none;padding:0}.runs-table.svelte-1yb6d54{width:100%;border-collapse:collapse;font-size:var(--text-md, 14px)}.runs-table.svelte-1yb6d54 th:where(.svelte-1yb6d54){text-align:left;padding:8px 12px;border-bottom:2px solid var(--border-color-primary, #e5e7eb);color:var(--body-text-color-subdued, #6b7280);font-weight:600;font-size:var(--text-sm, 12px);text-transform:uppercase;letter-spacing:.05em}.runs-table.svelte-1yb6d54 td:where(.svelte-1yb6d54){padding:8px 12px;border-bottom:1px solid var(--border-color-primary, #e5e7eb);color:var(--body-text-color, #1f2937)}.runs-table.svelte-1yb6d54 tbody:where(.svelte-1yb6d54) tr:where(.svelte-1yb6d54):nth-child(odd){background:var(--table-odd-background-fill, var(--background-fill-primary, white))}.runs-table.svelte-1yb6d54 tbody:where(.svelte-1yb6d54) tr:where(.svelte-1yb6d54):nth-child(2n){background:var(--table-even-background-fill, var(--background-fill-secondary, #f9fafb))}.runs-table.svelte-1yb6d54 tr:where(.svelte-1yb6d54):hover{background:var(--background-fill-secondary, #f3f4f6)}.run-name-cell.svelte-1yb6d54{font-weight:500}.run-name-with-dot.svelte-1yb6d54{display:inline-flex;align-items:center;gap:8px;max-width:100%}.run-dot.svelte-1yb6d54{width:10px;height:10px;border-radius:50%;flex-shrink:0}.link-btn.svelte-1yb6d54{background:none;border:none;color:var(--color-accent, #f97316);cursor:pointer;font:inherit;font-weight:500;padding:0;text-align:left}.link-btn.svelte-1yb6d54:hover{text-decoration:underline}.rename-input.svelte-1yb6d54{font:inherit;padding:2px 6px;border:1px solid var(--color-accent, #f97316);border-radius:var(--radius-sm, 4px);outline:none;width:100%}.actions-cell.svelte-1yb6d54{display:flex;gap:4px}.action-btn.svelte-1yb6d54{background:none;border:1px solid transparent;color:var(--body-text-color-subdued, #6b7280);cursor:pointer;padding:4px;border-radius:var(--radius-sm, 4px);display:flex;align-items:center}.action-btn.svelte-1yb6d54:hover{background:var(--background-fill-secondary, #f9fafb);border-color:var(--border-color-primary, #e5e7eb);color:var(--body-text-color, #1f2937)}.delete-btn.svelte-1yb6d54:hover{color:#dc2626;border-color:#fecaca;background:#fef2f2}.action-btn.svelte-1yb6d54:disabled{opacity:.45;cursor:not-allowed;pointer-events:none}.run-detail-page.svelte-1bpgsx2{padding:20px 24px;overflow-y:auto;flex:1}.detail-card.svelte-1bpgsx2{background:var(--background-fill-primary, white);border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-lg, 8px);padding:24px;max-width:800px}.detail-card.svelte-1bpgsx2 h2:where(.svelte-1bpgsx2){color:var(--body-text-color, #1f2937);margin:0 0 16px;font-size:var(--text-xl, 22px)}.detail-card.svelte-1bpgsx2 h3:where(.svelte-1bpgsx2){color:var(--body-text-color, #1f2937);margin:20px 0 8px;font-size:var(--text-lg, 16px)}.detail-grid.svelte-1bpgsx2{display:grid;grid-template-columns:repeat(auto-fill,minmax(200px,1fr));gap:12px}.detail-item.svelte-1bpgsx2{display:flex;flex-direction:column;gap:2px}.detail-label.svelte-1bpgsx2{font-size:var(--text-xs, 10px);font-weight:600;color:var(--body-text-color-subdued, #9ca3af);text-transform:uppercase}.detail-value.svelte-1bpgsx2{font-size:var(--text-md, 14px);color:var(--body-text-color, #1f2937)}.config-block.svelte-1bpgsx2{background:var(--background-fill-secondary, #f9fafb);padding:12px;border-radius:var(--radius-lg, 8px);border:1px solid var(--border-color-primary, #e5e7eb);font-size:var(--text-sm, 12px);color:var(--body-text-color, #1f2937);overflow-x:auto}.empty-state.svelte-1bpgsx2{max-width:640px;padding:40px 24px;color:var(--body-text-color, #1f2937)}.empty-state.svelte-1bpgsx2 h2:where(.svelte-1bpgsx2){margin:0 0 8px;font-size:20px;font-weight:700}.empty-state.svelte-1bpgsx2 p:where(.svelte-1bpgsx2){margin:12px 0 8px;color:var(--body-text-color-subdued, #6b7280)}.empty-state.svelte-1bpgsx2 pre:where(.svelte-1bpgsx2){background:var(--background-fill-secondary, #f9fafb);padding:16px;border-radius:var(--radius-lg, 8px);border:1px solid var(--border-color-primary, #e5e7eb);font-size:13px;overflow-x:auto}.empty-state.svelte-1bpgsx2 code:where(.svelte-1bpgsx2){background:var(--background-fill-secondary, #f0f0f0);padding:1px 5px;border-radius:var(--radius-sm, 4px);font-size:13px}.empty-state.svelte-1bpgsx2 pre:where(.svelte-1bpgsx2) code:where(.svelte-1bpgsx2){background:none;padding:0}.files-page.svelte-1xvfk9n{padding:20px 24px;overflow-y:auto;flex:1}h2.svelte-1xvfk9n{color:var(--body-text-color, #1f2937);font-size:var(--text-xl, 22px);margin-bottom:16px}.file-list.svelte-1xvfk9n{list-style:none;padding:0}.file-list.svelte-1xvfk9n li:where(.svelte-1xvfk9n){padding:8px 12px;border:1px solid var(--border-color-primary, #e5e7eb);border-radius:var(--radius-lg, 8px);margin-bottom:4px}.file-list.svelte-1xvfk9n a:where(.svelte-1xvfk9n){color:var(--secondary-600, #2563eb);text-decoration:none;font-size:var(--text-md, 14px)}.file-list.svelte-1xvfk9n a:where(.svelte-1xvfk9n):hover{text-decoration:underline}.empty-state.svelte-1xvfk9n{max-width:640px;padding:40px 24px;color:var(--body-text-color, #1f2937)}.empty-state.svelte-1xvfk9n h2:where(.svelte-1xvfk9n){margin:0 0 8px;font-size:20px;font-weight:700}.empty-state.svelte-1xvfk9n p:where(.svelte-1xvfk9n){margin:12px 0 8px;color:var(--body-text-color-subdued, #6b7280)}.empty-state.svelte-1xvfk9n pre:where(.svelte-1xvfk9n){background:var(--background-fill-secondary, #f9fafb);padding:16px;border-radius:var(--radius-lg, 8px);border:1px solid var(--border-color-primary, #e5e7eb);font-size:13px;overflow-x:auto}.empty-state.svelte-1xvfk9n code:where(.svelte-1xvfk9n){background:var(--background-fill-secondary, #f0f0f0);padding:1px 5px;border-radius:var(--radius-sm, 4px);font-size:13px}.empty-state.svelte-1xvfk9n pre:where(.svelte-1xvfk9n) code:where(.svelte-1xvfk9n){background:none;padding:0}*{margin:0;padding:0;box-sizing:border-box}body{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica Neue,Arial,sans-serif;background:var(--background-fill-primary, #fff);color:var(--body-text-color, #1f2937);font-size:var(--text-md, 14px);-webkit-font-smoothing:antialiased}.app.svelte-1n46o8q{display:flex;height:100vh;overflow:hidden}.main.svelte-1n46o8q{flex:1;display:flex;flex-direction:column;overflow:hidden;min-width:0}.page-content.svelte-1n46o8q{flex:1;overflow:hidden;display:flex;background:var(--bg-primary)}

trackio/frontend/dist/index.html

@@ -0,0 +1,14 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Trackio Dashboard</title>
+    <link rel="icon" type="image/png" href="/static/trackio/trackio_logo_light.png" />
+    <script type="module" crossorigin src="/assets/index-D1G_q77u.js"></script>
+    <link rel="stylesheet" crossorigin href="/assets/index-0Wf8YNCR.css">
+  </head>
+  <body>
+    <div id="app"></div>
+  </body>
+</html>

trackio/frontend/eslint.config.js

@@ -0,0 +1,42 @@
+import js from "@eslint/js";
+import svelte from "eslint-plugin-svelte";
+import svelteParser from "svelte-eslint-parser";
+import globals from "globals";
+
+export default [
+  { ignores: ["dist/**", "node_modules/**"] },
+  {
+    files: ["**/*.js"],
+    languageOptions: {
+      globals: {
+        ...globals.browser,
+        ...globals.es2021,
+        $state: "readonly",
+        $derived: "readonly",
+        $effect: "readonly",
+        $props: "readonly",
+        $bindable: "readonly",
+        $inspect: "readonly",
+      },
+    },
+    rules: {
+      ...js.configs.recommended.rules,
+      "no-unused-vars": ["error", { argsIgnorePattern: "^_" }],
+      "no-empty": "off",
+    },
+  },
+  {
+    files: ["**/*.svelte"],
+    languageOptions: {
+      parser: svelteParser,
+      globals: { ...globals.browser, ...globals.es2021 },
+    },
+    plugins: { svelte },
+    rules: {
+      ...js.configs.recommended.rules,
+      ...svelte.configs.recommended.rules,
+      "no-unused-vars": ["error", { argsIgnorePattern: "^_", varsIgnorePattern: "^\\$" }],
+      "no-empty": "off",
+    },
+  },
+];

trackio/frontend/index.html

@@ -0,0 +1,13 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Trackio Dashboard</title>
+    <link rel="icon" type="image/png" href="/static/trackio/trackio_logo_light.png" />
+  </head>
+  <body>
+    <div id="app"></div>
+    <script type="module" src="/src/main.js"></script>
+  </body>
+</html>

trackio/frontend_server.py

@@ -0,0 +1,63 @@
+"""Serves the built Svelte frontend alongside the Gradio API."""
+
+import logging
+import re
+from pathlib import Path
+
+from starlette.responses import HTMLResponse
+from starlette.routing import Mount, Route
+from starlette.staticfiles import StaticFiles
+
+FRONTEND_DIR = Path(__file__).parent / "frontend" / "dist"
+ASSETS_DIR = Path(__file__).parent / "assets"
+
+_logger = logging.getLogger(__name__)
+
+_SPA_SEGMENTS = (
+    "metrics",
+    "system",
+    "media",
+    "reports",
+    "runs",
+    "run",
+    "files",
+)
+
+
+def mount_frontend(app):
+    if not FRONTEND_DIR.exists():
+        _logger.warning(
+            "Trackio dashboard UI was not mounted: %s is missing. "
+            "Build the frontend with `npm ci && npm run build` in trackio/frontend.",
+            FRONTEND_DIR,
+        )
+        return
+
+    index_html_path = FRONTEND_DIR / "index.html"
+    if not index_html_path.exists():
+        _logger.warning(
+            "Trackio dashboard UI was not mounted: %s is missing.",
+            index_html_path,
+        )
+        return
+
+    index_html_content = index_html_path.read_text()
+    patched_html = re.sub(
+        r'/assets/(index-[^"]+)',
+        r"/assets/app/\1",
+        index_html_content,
+    )
+
+    async def serve_frontend(request):
+        return HTMLResponse(patched_html)
+
+    vite_assets = StaticFiles(directory=str(FRONTEND_DIR / "assets"))
+    static_assets = StaticFiles(directory=str(ASSETS_DIR))
+
+    app.routes.insert(0, Mount("/static/trackio", app=static_assets))
+    app.routes.insert(0, Mount("/assets/app", app=vite_assets))
+
+    for seg in reversed(_SPA_SEGMENTS):
+        app.routes.insert(0, Route(f"/{seg}/", serve_frontend, methods=["GET"]))
+        app.routes.insert(0, Route(f"/{seg}", serve_frontend, methods=["GET"]))
+    app.routes.insert(0, Route("/", serve_frontend, methods=["GET"]))

trackio/gpu.py

@@ -0,0 +1,357 @@
+import os
+import threading
+import warnings
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    from trackio.run import Run
+
+pynvml: Any = None
+PYNVML_AVAILABLE = False
+_nvml_initialized = False
+_nvml_lock = threading.Lock()
+_energy_baseline: dict[int, float] = {}
+
+
+def _ensure_pynvml():
+    global PYNVML_AVAILABLE, pynvml
+    if PYNVML_AVAILABLE:
+        return pynvml
+    try:
+        import pynvml as _pynvml
+
+        pynvml = _pynvml
+        PYNVML_AVAILABLE = True
+        return pynvml
+    except ImportError:
+        raise ImportError(
+            "nvidia-ml-py is required for GPU monitoring. "
+            "Install it with: pip install nvidia-ml-py"
+        )
+
+
+def _init_nvml() -> bool:
+    global _nvml_initialized
+    with _nvml_lock:
+        if _nvml_initialized:
+            return True
+        try:
+            nvml = _ensure_pynvml()
+            nvml.nvmlInit()
+            _nvml_initialized = True
+            return True
+        except Exception:
+            return False
+
+
+def get_gpu_count() -> tuple[int, list[int]]:
+    """
+    Get the number of GPUs visible to this process and their physical indices.
+    Respects CUDA_VISIBLE_DEVICES environment variable.
+
+    Returns:
+        Tuple of (count, physical_indices) where:
+        - count: Number of visible GPUs
+        - physical_indices: List mapping logical index to physical GPU index.
+          e.g., if CUDA_VISIBLE_DEVICES=2,3 returns (2, [2, 3])
+          meaning logical GPU 0 = physical GPU 2, logical GPU 1 = physical GPU 3
+    """
+    if not _init_nvml():
+        return 0, []
+
+    cuda_visible = os.environ.get("CUDA_VISIBLE_DEVICES")
+    if cuda_visible is not None and cuda_visible.strip():
+        try:
+            indices = [int(x.strip()) for x in cuda_visible.split(",") if x.strip()]
+            return len(indices), indices
+        except ValueError:
+            pass
+
+    try:
+        total = pynvml.nvmlDeviceGetCount()
+        return total, list(range(total))
+    except Exception:
+        return 0, []
+
+
+def gpu_available() -> bool:
+    """
+    Check if GPU monitoring is available.
+
+    Returns True if nvidia-ml-py is installed and at least one NVIDIA GPU is detected.
+    This is used for auto-detection of GPU logging.
+    """
+    try:
+        _ensure_pynvml()
+        count, _ = get_gpu_count()
+        return count > 0
+    except ImportError:
+        return False
+    except Exception:
+        return False
+
+
+def reset_energy_baseline():
+    """Reset the energy baseline for all GPUs. Called when a new run starts."""
+    global _energy_baseline
+    _energy_baseline = {}
+
+
+def collect_gpu_metrics(device: int | None = None) -> dict:
+    """
+    Collect GPU metrics for visible GPUs.
+
+    Args:
+        device: CUDA device index to collect metrics from. If None, collects
+                from all GPUs visible to this process (respects CUDA_VISIBLE_DEVICES).
+                The device index is the logical CUDA index (0, 1, 2...), not the
+                physical GPU index.
+
+    Returns:
+        Dictionary of GPU metrics. Keys use logical device indices (gpu/0/, gpu/1/, etc.)
+        which correspond to CUDA device indices, not physical GPU indices.
+    """
+    if not _init_nvml():
+        return {}
+
+    gpu_count, visible_gpus = get_gpu_count()
+    if gpu_count == 0:
+        return {}
+
+    if device is not None:
+        if device < 0 or device >= gpu_count:
+            return {}
+        gpu_indices = [(device, visible_gpus[device])]
+    else:
+        gpu_indices = list(enumerate(visible_gpus))
+
+    metrics = {}
+    total_util = 0.0
+    total_mem_used_gib = 0.0
+    total_power = 0.0
+    max_temp = 0.0
+    valid_util_count = 0
+
+    for logical_idx, physical_idx in gpu_indices:
+        prefix = f"gpu/{logical_idx}"
+        try:
+            handle = pynvml.nvmlDeviceGetHandleByIndex(physical_idx)
+
+            try:
+                util = pynvml.nvmlDeviceGetUtilizationRates(handle)
+                metrics[f"{prefix}/utilization"] = util.gpu
+                metrics[f"{prefix}/memory_utilization"] = util.memory
+                total_util += util.gpu
+                valid_util_count += 1
+            except Exception:
+                pass
+
+            try:
+                mem = pynvml.nvmlDeviceGetMemoryInfo(handle)
+                mem_used_gib = mem.used / (1024**3)
+                mem_total_gib = mem.total / (1024**3)
+                metrics[f"{prefix}/allocated_memory"] = mem_used_gib
+                metrics[f"{prefix}/total_memory"] = mem_total_gib
+                if mem.total > 0:
+                    metrics[f"{prefix}/memory_usage"] = mem.used / mem.total
+                total_mem_used_gib += mem_used_gib
+            except Exception:
+                pass
+
+            try:
+                power_mw = pynvml.nvmlDeviceGetPowerUsage(handle)
+                power_w = power_mw / 1000.0
+                metrics[f"{prefix}/power"] = power_w
+                total_power += power_w
+            except Exception:
+                pass
+
+            try:
+                power_limit_mw = pynvml.nvmlDeviceGetPowerManagementLimit(handle)
+                power_limit_w = power_limit_mw / 1000.0
+                metrics[f"{prefix}/power_limit"] = power_limit_w
+                if power_limit_w > 0 and f"{prefix}/power" in metrics:
+                    metrics[f"{prefix}/power_percent"] = (
+                        metrics[f"{prefix}/power"] / power_limit_w
+                    ) * 100
+            except Exception:
+                pass
+
+            try:
+                temp = pynvml.nvmlDeviceGetTemperature(
+                    handle, pynvml.NVML_TEMPERATURE_GPU
+                )
+                metrics[f"{prefix}/temp"] = temp
+                max_temp = max(max_temp, temp)
+            except Exception:
+                pass
+
+            try:
+                sm_clock = pynvml.nvmlDeviceGetClockInfo(handle, pynvml.NVML_CLOCK_SM)
+                metrics[f"{prefix}/sm_clock"] = sm_clock
+            except Exception:
+                pass
+
+            try:
+                mem_clock = pynvml.nvmlDeviceGetClockInfo(handle, pynvml.NVML_CLOCK_MEM)
+                metrics[f"{prefix}/memory_clock"] = mem_clock
+            except Exception:
+                pass
+
+            try:
+                fan_speed = pynvml.nvmlDeviceGetFanSpeed(handle)
+                metrics[f"{prefix}/fan_speed"] = fan_speed
+            except Exception:
+                pass
+
+            try:
+                pstate = pynvml.nvmlDeviceGetPerformanceState(handle)
+                metrics[f"{prefix}/performance_state"] = pstate
+            except Exception:
+                pass
+
+            try:
+                energy_mj = pynvml.nvmlDeviceGetTotalEnergyConsumption(handle)
+                if logical_idx not in _energy_baseline:
+                    _energy_baseline[logical_idx] = energy_mj
+                energy_consumed_mj = energy_mj - _energy_baseline[logical_idx]
+                metrics[f"{prefix}/energy_consumed"] = energy_consumed_mj / 1000.0
+            except Exception:
+                pass
+
+            try:
+                pcie_tx = pynvml.nvmlDeviceGetPcieThroughput(
+                    handle, pynvml.NVML_PCIE_UTIL_TX_BYTES
+                )
+                pcie_rx = pynvml.nvmlDeviceGetPcieThroughput(
+                    handle, pynvml.NVML_PCIE_UTIL_RX_BYTES
+                )
+                metrics[f"{prefix}/pcie_tx"] = pcie_tx / 1024.0
+                metrics[f"{prefix}/pcie_rx"] = pcie_rx / 1024.0
+            except Exception:
+                pass
+
+            try:
+                throttle = pynvml.nvmlDeviceGetCurrentClocksThrottleReasons(handle)
+                metrics[f"{prefix}/throttle_thermal"] = int(
+                    bool(throttle & pynvml.nvmlClocksThrottleReasonSwThermalSlowdown)
+                )
+                metrics[f"{prefix}/throttle_power"] = int(
+                    bool(throttle & pynvml.nvmlClocksThrottleReasonSwPowerCap)
+                )
+                metrics[f"{prefix}/throttle_hw_slowdown"] = int(
+                    bool(throttle & pynvml.nvmlClocksThrottleReasonHwSlowdown)
+                )
+                metrics[f"{prefix}/throttle_apps"] = int(
+                    bool(
+                        throttle
+                        & pynvml.nvmlClocksThrottleReasonApplicationsClocksSetting
+                    )
+                )
+            except Exception:
+                pass
+
+            try:
+                ecc_corrected = pynvml.nvmlDeviceGetTotalEccErrors(
+                    handle,
+                    pynvml.NVML_MEMORY_ERROR_TYPE_CORRECTED,
+                    pynvml.NVML_VOLATILE_ECC,
+                )
+                metrics[f"{prefix}/corrected_memory_errors"] = ecc_corrected
+            except Exception:
+                pass
+
+            try:
+                ecc_uncorrected = pynvml.nvmlDeviceGetTotalEccErrors(
+                    handle,
+                    pynvml.NVML_MEMORY_ERROR_TYPE_UNCORRECTED,
+                    pynvml.NVML_VOLATILE_ECC,
+                )
+                metrics[f"{prefix}/uncorrected_memory_errors"] = ecc_uncorrected
+            except Exception:
+                pass
+
+        except Exception:
+            continue
+
+    if valid_util_count > 0:
+        metrics["gpu/mean_utilization"] = total_util / valid_util_count
+    if total_mem_used_gib > 0:
+        metrics["gpu/total_allocated_memory"] = total_mem_used_gib
+    if total_power > 0:
+        metrics["gpu/total_power"] = total_power
+    if max_temp > 0:
+        metrics["gpu/max_temp"] = max_temp
+
+    return metrics
+
+
+class GpuMonitor:
+    def __init__(self, run: "Run", interval: float = 10.0):
+        self._run = run
+        self._interval = interval
+        self._stop_flag = threading.Event()
+        self._thread: "threading.Thread | None" = None
+
+    def start(self):
+        count, _ = get_gpu_count()
+        if count == 0:
+            warnings.warn(
+                "auto_log_gpu=True but no NVIDIA GPUs detected. GPU logging disabled."
+            )
+            return
+
+        reset_energy_baseline()
+        self._thread = threading.Thread(target=self._monitor_loop, daemon=True)
+        self._thread.start()
+
+    def stop(self):
+        self._stop_flag.set()
+        if self._thread is not None:
+            self._thread.join(timeout=2.0)
+
+    def _monitor_loop(self):
+        while not self._stop_flag.is_set():
+            try:
+                metrics = collect_gpu_metrics()
+                if metrics:
+                    self._run.log_system(metrics)
+            except Exception:
+                pass
+
+            self._stop_flag.wait(timeout=self._interval)
+
+
+def log_gpu(run: "Run | None" = None, device: int | None = None) -> dict:
+    """
+    Log GPU metrics to the current or specified run as system metrics.
+
+    Args:
+        run: Optional Run instance. If None, uses current run from context.
+        device: CUDA device index to collect metrics from. If None, collects
+                from all GPUs visible to this process (respects CUDA_VISIBLE_DEVICES).
+
+    Returns:
+        dict: The GPU metrics that were logged.
+
+    Example:
+        ```python
+        import trackio
+
+        run = trackio.init(project="my-project")
+        trackio.log({"loss": 0.5})
+        trackio.log_gpu()  # logs all visible GPUs
+        trackio.log_gpu(device=0)  # logs only CUDA device 0
+        ```
+    """
+    from trackio import context_vars
+
+    if run is None:
+        run = context_vars.current_run.get()
+        if run is None:
+            raise RuntimeError("Call trackio.init() before trackio.log_gpu().")
+
+    metrics = collect_gpu_metrics(device=device)
+    if metrics:
+        run.log_system(metrics)
+    return metrics

trackio/histogram.py

@@ -0,0 +1,71 @@
+from typing import Sequence
+
+import numpy as np
+
+
+class Histogram:
+    """
+    Histogram data type for Trackio, compatible with wandb.Histogram.
+
+    Args:
+        sequence (`np.ndarray` or `Sequence[float]` or `Sequence[int]`, *optional*):
+            Sequence of values to create the histogram from.
+        np_histogram (`tuple`, *optional*):
+            Pre-computed NumPy histogram as a `(hist, bins)` tuple.
+        num_bins (`int`, *optional*, defaults to `64`):
+            Number of bins for the histogram (maximum `512`).
+
+    Example:
+        ```python
+        import trackio
+        import numpy as np
+
+        # Create histogram from sequence
+        data = np.random.randn(1000)
+        trackio.log({"distribution": trackio.Histogram(data)})
+
+        # Create histogram from numpy histogram
+        hist, bins = np.histogram(data, bins=30)
+        trackio.log({"distribution": trackio.Histogram(np_histogram=(hist, bins))})
+
+        # Specify custom number of bins
+        trackio.log({"distribution": trackio.Histogram(data, num_bins=50)})
+        ```
+    """
+
+    TYPE = "trackio.histogram"
+
+    def __init__(
+        self,
+        sequence: np.ndarray | Sequence[float] | Sequence[int] | None = None,
+        np_histogram: tuple | None = None,
+        num_bins: int = 64,
+    ):
+        if sequence is None and np_histogram is None:
+            raise ValueError("Must provide either sequence or np_histogram")
+
+        if sequence is not None and np_histogram is not None:
+            raise ValueError("Cannot provide both sequence and np_histogram")
+
+        num_bins = min(num_bins, 512)
+
+        if np_histogram is not None:
+            self.histogram, self.bins = np_histogram
+            self.histogram = np.asarray(self.histogram)
+            self.bins = np.asarray(self.bins)
+        else:
+            data = np.asarray(sequence).flatten()
+            data = data[np.isfinite(data)]
+            if len(data) == 0:
+                self.histogram = np.array([])
+                self.bins = np.array([])
+            else:
+                self.histogram, self.bins = np.histogram(data, bins=num_bins)
+
+    def _to_dict(self) -> dict:
+        """Convert histogram to dictionary for storage."""
+        return {
+            "_type": self.TYPE,
+            "bins": self.bins.tolist(),
+            "values": self.histogram.tolist(),
+        }

trackio/imports.py

@@ -0,0 +1,304 @@
+import os
+from pathlib import Path
+
+import pandas as pd
+
+from trackio import deploy, utils
+from trackio.sqlite_storage import SQLiteStorage
+
+
+def import_csv(
+    csv_path: str | Path,
+    project: str,
+    name: str | None = None,
+    space_id: str | None = None,
+    dataset_id: str | None = None,
+    private: bool | None = None,
+    force: bool = False,
+) -> None:
+    """
+    Imports a CSV file into a Trackio project. The CSV file must contain a `"step"`
+    column, may optionally contain a `"timestamp"` column, and any other columns will be
+    treated as metrics. It should also include a header row with the column names.
+
+    TODO: call init() and return a Run object so that the user can continue to log metrics to it.
+
+    Args:
+        csv_path (`str` or `Path`):
+            The str or Path to the CSV file to import.
+        project (`str`):
+            The name of the project to import the CSV file into. Must not be an existing
+            project.
+        name (`str`, *optional*):
+            The name of the Run to import the CSV file into. If not provided, a default
+            name will be generated.
+        name (`str`, *optional*):
+            The name of the run (if not provided, a default name will be generated).
+        space_id (`str`, *optional*):
+            If provided, the project will be logged to a Hugging Face Space instead of a
+            local directory. Should be a complete Space name like `"username/reponame"`
+            or `"orgname/reponame"`, or just `"reponame"` in which case the Space will
+            be created in the currently-logged-in Hugging Face user's namespace. If the
+            Space does not exist, it will be created. If the Space already exists, the
+            project will be logged to it.
+        dataset_id (`str`, *optional*):
+            If provided, a persistent Hugging Face Dataset will be created and the
+            metrics will be synced to it every 5 minutes. Should be a complete Dataset
+            name like `"username/datasetname"` or `"orgname/datasetname"`, or just
+            `"datasetname"` in which case the Dataset will be created in the
+            currently-logged-in Hugging Face user's namespace. If the Dataset does not
+            exist, it will be created. If the Dataset already exists, the project will
+            be appended to it. If not provided, the metrics will be logged to a local
+            SQLite database, unless a `space_id` is provided, in which case a Dataset
+            will be automatically created with the same name as the Space but with the
+            `"_dataset"` suffix.
+        private (`bool`, *optional*):
+            Whether to make the Space private. If None (default), the repo will be
+            public unless the organization's default is private. This value is ignored
+            if the repo already exists.
+    """
+    if SQLiteStorage.get_runs(project):
+        raise ValueError(
+            f"Project '{project}' already exists. Cannot import CSV into existing project."
+        )
+
+    csv_path = Path(csv_path)
+    if not csv_path.exists():
+        raise FileNotFoundError(f"CSV file not found: {csv_path}")
+
+    df = pd.read_csv(csv_path)
+    if df.empty:
+        raise ValueError("CSV file is empty")
+
+    column_mapping = utils.simplify_column_names(df.columns.tolist())
+    df = df.rename(columns=column_mapping)
+
+    step_column = None
+    for col in df.columns:
+        if col.lower() == "step":
+            step_column = col
+            break
+
+    if step_column is None:
+        raise ValueError("CSV file must contain a 'step' or 'Step' column")
+
+    if name is None:
+        name = csv_path.stem
+
+    metrics_list = []
+    steps = []
+    timestamps = []
+
+    numeric_columns = []
+    for column in df.columns:
+        if column == step_column:
+            continue
+        if column == "timestamp":
+            continue
+
+        try:
+            pd.to_numeric(df[column], errors="raise")
+            numeric_columns.append(column)
+        except (ValueError, TypeError):
+            continue
+
+    for _, row in df.iterrows():
+        metrics = {}
+        for column in numeric_columns:
+            value = row[column]
+            if bool(pd.notna(value)):
+                metrics[column] = float(value)
+
+        if metrics:
+            metrics_list.append(metrics)
+            steps.append(int(row[step_column]))
+
+            if "timestamp" in df.columns and bool(pd.notna(row["timestamp"])):
+                timestamps.append(str(row["timestamp"]))
+            else:
+                timestamps.append("")
+
+    if metrics_list:
+        SQLiteStorage.bulk_log(
+            project=project,
+            run=name,
+            metrics_list=metrics_list,
+            steps=steps,
+            timestamps=timestamps,
+        )
+
+    print(
+        f"* Imported {len(metrics_list)} rows from {csv_path} into project '{project}' as run '{name}'"
+    )
+    print(f"* Metrics found: {', '.join(metrics_list[0].keys())}")
+
+    space_id, dataset_id = utils.preprocess_space_and_dataset_ids(space_id, dataset_id)
+    if dataset_id is not None:
+        os.environ["TRACKIO_DATASET_ID"] = dataset_id
+        print(f"* Trackio metrics will be synced to Hugging Face Dataset: {dataset_id}")
+
+    if space_id is None:
+        utils.print_dashboard_instructions(project)
+    else:
+        deploy.create_space_if_not_exists(
+            space_id=space_id, dataset_id=dataset_id, private=private
+        )
+        deploy.wait_until_space_exists(space_id=space_id)
+        deploy.upload_db_to_space(project=project, space_id=space_id, force=force)
+        print(
+            f"* View dashboard by going to: {deploy.SPACE_URL.format(space_id=space_id)}"
+        )
+
+
+def import_tf_events(
+    log_dir: str | Path,
+    project: str,
+    name: str | None = None,
+    space_id: str | None = None,
+    dataset_id: str | None = None,
+    private: bool | None = None,
+    force: bool = False,
+) -> None:
+    """
+    Imports TensorFlow Events files from a directory into a Trackio project. Each
+    subdirectory in the log directory will be imported as a separate run.
+
+    Args:
+        log_dir (`str` or `Path`):
+            The str or Path to the directory containing TensorFlow Events files.
+        project (`str`):
+            The name of the project to import the TensorFlow Events files into. Must not
+            be an existing project.
+        name (`str`, *optional*):
+            The name prefix for runs (if not provided, will use directory names). Each
+            subdirectory will create a separate run.
+        space_id (`str`, *optional*):
+            If provided, the project will be logged to a Hugging Face Space instead of a
+            local directory. Should be a complete Space name like `"username/reponame"`
+            or `"orgname/reponame"`, or just `"reponame"` in which case the Space will
+            be created in the currently-logged-in Hugging Face user's namespace. If the
+            Space does not exist, it will be created. If the Space already exists, the
+            project will be logged to it.
+        dataset_id (`str`, *optional*):
+            If provided, a persistent Hugging Face Dataset will be created and the
+            metrics will be synced to it every 5 minutes. Should be a complete Dataset
+            name like `"username/datasetname"` or `"orgname/datasetname"`, or just
+            `"datasetname"` in which case the Dataset will be created in the
+            currently-logged-in Hugging Face user's namespace. If the Dataset does not
+            exist, it will be created. If the Dataset already exists, the project will
+            be appended to it. If not provided, the metrics will be logged to a local
+            SQLite database, unless a `space_id` is provided, in which case a Dataset
+            will be automatically created with the same name as the Space but with the
+            `"_dataset"` suffix.
+        private (`bool`, *optional*):
+            Whether to make the Space private. If None (default), the repo will be
+            public unless the organization's default is private. This value is ignored
+            if the repo already exists.
+    """
+    try:
+        from tbparse import SummaryReader
+    except ImportError:
+        raise ImportError(
+            "The `tbparse` package is not installed but is required for `import_tf_events`. Please install trackio with the `tensorboard` extra: `pip install trackio[tensorboard]`."
+        )
+
+    if SQLiteStorage.get_runs(project):
+        raise ValueError(
+            f"Project '{project}' already exists. Cannot import TF events into existing project."
+        )
+
+    path = Path(log_dir)
+    if not path.exists():
+        raise FileNotFoundError(f"TF events directory not found: {path}")
+
+    # Use tbparse to read all tfevents files in the directory structure
+    reader = SummaryReader(str(path), extra_columns={"dir_name"})
+    df = reader.scalars
+
+    if df.empty:
+        raise ValueError(f"No TensorFlow events data found in {path}")
+
+    total_imported = 0
+    imported_runs = []
+
+    # Group by dir_name to create separate runs
+    for dir_name, group_df in df.groupby("dir_name"):
+        try:
+            # Determine run name based on directory name
+            if dir_name == "":
+                run_name = "main"  # For files in the root directory
+            else:
+                run_name = dir_name  # Use directory name
+
+            if name:
+                run_name = f"{name}_{run_name}"
+
+            if group_df.empty:
+                print(f"* Skipping directory {dir_name}: no scalar data found")
+                continue
+
+            metrics_list = []
+            steps = []
+            timestamps = []
+
+            for _, row in group_df.iterrows():
+                # Convert row values to appropriate types
+                tag = str(row["tag"])
+                value = float(row["value"])
+                step = int(row["step"])
+
+                metrics = {tag: value}
+                metrics_list.append(metrics)
+                steps.append(step)
+
+                # Use wall_time if present, else fallback
+                if "wall_time" in group_df.columns and not bool(
+                    pd.isna(row["wall_time"])
+                ):
+                    timestamps.append(str(row["wall_time"]))
+                else:
+                    timestamps.append("")
+
+            if metrics_list:
+                SQLiteStorage.bulk_log(
+                    project=project,
+                    run=str(run_name),
+                    metrics_list=metrics_list,
+                    steps=steps,
+                    timestamps=timestamps,
+                )
+
+                total_imported += len(metrics_list)
+                imported_runs.append(run_name)
+
+                print(
+                    f"* Imported {len(metrics_list)} scalar events from directory '{dir_name}' as run '{run_name}'"
+                )
+                print(f"* Metrics in this run: {', '.join(set(group_df['tag']))}")
+
+        except Exception as e:
+            print(f"* Error processing directory {dir_name}: {e}")
+            continue
+
+    if not imported_runs:
+        raise ValueError("No valid TensorFlow events data could be imported")
+
+    print(f"* Total imported events: {total_imported}")
+    print(f"* Created runs: {', '.join(imported_runs)}")
+
+    space_id, dataset_id = utils.preprocess_space_and_dataset_ids(space_id, dataset_id)
+    if dataset_id is not None:
+        os.environ["TRACKIO_DATASET_ID"] = dataset_id
+        print(f"* Trackio metrics will be synced to Hugging Face Dataset: {dataset_id}")
+
+    if space_id is None:
+        utils.print_dashboard_instructions(project)
+    else:
+        deploy.create_space_if_not_exists(
+            space_id, dataset_id=dataset_id, private=private
+        )
+        deploy.wait_until_space_exists(space_id)
+        deploy.upload_db_to_space(project, space_id, force=force)
+        print(
+            f"* View dashboard by going to: {deploy.SPACE_URL.format(space_id=space_id)}"
+        )

trackio/markdown.py

@@ -0,0 +1,21 @@
+class Markdown:
+    """
+    Markdown report data type for Trackio.
+
+    Args:
+        text (`str`):
+            Markdown content to log.
+    """
+
+    TYPE = "trackio.markdown"
+
+    def __init__(self, text: str = ""):
+        if not isinstance(text, str):
+            raise ValueError("Markdown text must be a string")
+        self.text = text
+
+    def _to_dict(self) -> dict:
+        return {
+            "_type": self.TYPE,
+            "_value": self.text,
+        }

trackio/media/__init__.py

@@ -0,0 +1,27 @@
+"""
+Media module for Trackio.
+
+This module contains all media-related functionality including:
+- TrackioImage, TrackioVideo, TrackioAudio classes
+- Video writing utilities
+- Audio conversion utilities
+"""
+
+from trackio.media.audio import TrackioAudio
+from trackio.media.image import TrackioImage
+from trackio.media.media import TrackioMedia
+from trackio.media.utils import get_project_media_path
+from trackio.media.video import TrackioVideo
+
+write_audio = TrackioAudio.write_audio
+write_video = TrackioVideo.write_video
+
+__all__ = [
+    "TrackioMedia",
+    "TrackioImage",
+    "TrackioVideo",
+    "TrackioAudio",
+    "get_project_media_path",
+    "write_video",
+    "write_audio",
+]

trackio/media/audio.py

@@ -0,0 +1,167 @@
+import os
+import shutil
+import warnings
+from pathlib import Path
+from typing import Literal
+
+import numpy as np
+from pydub import AudioSegment
+
+from trackio.media.media import TrackioMedia
+from trackio.media.utils import check_ffmpeg_installed, check_path
+
+SUPPORTED_FORMATS = ["wav", "mp3"]
+AudioFormatType = Literal["wav", "mp3"]
+TrackioAudioSourceType = str | Path | np.ndarray
+
+
+class TrackioAudio(TrackioMedia):
+    """
+    Initializes an Audio object.
+
+    Example:
+        ```python
+        import trackio
+        import numpy as np
+
+        # Generate a 1-second 440 Hz sine wave (mono)
+        sr = 16000
+        t = np.linspace(0, 1, sr, endpoint=False)
+        wave = 0.2 * np.sin(2 * np.pi * 440 * t)
+        audio = trackio.Audio(wave, caption="A4 sine", sample_rate=sr, format="wav")
+        trackio.log({"tone": audio})
+
+        # Stereo from numpy array (shape: samples, 2)
+        stereo = np.stack([wave, wave], axis=1)
+        audio = trackio.Audio(stereo, caption="Stereo", sample_rate=sr, format="mp3")
+        trackio.log({"stereo": audio})
+
+        # From an existing file
+        audio = trackio.Audio("path/to/audio.wav", caption="From file")
+        trackio.log({"file_audio": audio})
+        ```
+
+    Args:
+        value (`str`, `Path`, or `numpy.ndarray`, *optional*):
+            A path to an audio file, or a numpy array.
+            The array should be shaped `(samples,)` for mono or `(samples, 2)` for stereo.
+            Float arrays will be peak-normalized and converted to 16-bit PCM; integer arrays will be converted to 16-bit PCM as needed.
+        caption (`str`, *optional*):
+            A string caption for the audio.
+        sample_rate (`int`, *optional*):
+            Sample rate in Hz. Required when `value` is a numpy array.
+        format (`Literal["wav", "mp3"]`, *optional*):
+            Audio format used when `value` is a numpy array. Default is "wav".
+    """
+
+    TYPE = "trackio.audio"
+
+    def __init__(
+        self,
+        value: TrackioAudioSourceType,
+        caption: str | None = None,
+        sample_rate: int | None = None,
+        format: AudioFormatType | None = None,
+    ):
+        super().__init__(value, caption)
+        if isinstance(value, np.ndarray):
+            if sample_rate is None:
+                raise ValueError("Sample rate is required when value is an ndarray")
+            if format is None:
+                format = "wav"
+        self._format = format
+        self._sample_rate = sample_rate
+
+    def _save_media(self, file_path: Path):
+        if isinstance(self._value, np.ndarray):
+            TrackioAudio.write_audio(
+                data=self._value,
+                sample_rate=self._sample_rate,
+                filename=file_path,
+                format=self._format,
+            )
+        elif isinstance(self._value, str | Path):
+            if os.path.isfile(self._value):
+                shutil.copy(self._value, file_path)
+            else:
+                raise ValueError(f"File not found: {self._value}")
+
+    @staticmethod
+    def ensure_int16_pcm(data: np.ndarray) -> np.ndarray:
+        """
+        Convert input audio array to contiguous int16 PCM.
+        Peak normalization is applied to floating inputs.
+        """
+        arr = np.asarray(data)
+        if arr.ndim not in (1, 2):
+            raise ValueError("Audio data must be 1D (mono) or 2D ([samples, channels])")
+
+        if arr.dtype != np.int16:
+            warnings.warn(
+                f"Converting {arr.dtype} audio to int16 PCM; pass int16 to avoid conversion.",
+                stacklevel=2,
+            )
+
+        arr = np.nan_to_num(arr, copy=False)
+
+        # Floating types: normalize to peak 1.0, then scale to int16
+        if np.issubdtype(arr.dtype, np.floating):
+            max_abs = float(np.max(np.abs(arr))) if arr.size else 0.0
+            if max_abs > 0.0:
+                arr = arr / max_abs
+            out = (arr * 32767.0).clip(-32768, 32767).astype(np.int16, copy=False)
+            return np.ascontiguousarray(out)
+
+        converters: dict[np.dtype, callable] = {
+            np.dtype(np.int16): lambda a: a,
+            np.dtype(np.int32): lambda a: (a.astype(np.int32) // 65536).astype(
+                np.int16, copy=False
+            ),
+            np.dtype(np.uint16): lambda a: (a.astype(np.int32) - 32768).astype(
+                np.int16, copy=False
+            ),
+            np.dtype(np.uint8): lambda a: (a.astype(np.int32) * 257 - 32768).astype(
+                np.int16, copy=False
+            ),
+            np.dtype(np.int8): lambda a: (a.astype(np.int32) * 256).astype(
+                np.int16, copy=False
+            ),
+        }
+
+        conv = converters.get(arr.dtype)
+        if conv is not None:
+            out = conv(arr)
+            return np.ascontiguousarray(out)
+        raise TypeError(f"Unsupported audio dtype: {arr.dtype}")
+
+    @staticmethod
+    def write_audio(
+        data: np.ndarray,
+        sample_rate: int,
+        filename: str | Path,
+        format: AudioFormatType = "wav",
+    ) -> None:
+        if not isinstance(sample_rate, int) or sample_rate <= 0:
+            raise ValueError(f"Invalid sample_rate: {sample_rate}")
+        if format not in SUPPORTED_FORMATS:
+            raise ValueError(
+                f"Unsupported format: {format}. Supported: {SUPPORTED_FORMATS}"
+            )
+
+        check_path(filename)
+
+        pcm = TrackioAudio.ensure_int16_pcm(data)
+
+        if format != "wav":
+            check_ffmpeg_installed()
+
+        channels = 1 if pcm.ndim == 1 else pcm.shape[1]
+        audio = AudioSegment(
+            pcm.tobytes(),
+            frame_rate=sample_rate,
+            sample_width=2,  # int16
+            channels=channels,
+        )
+
+        file = audio.export(str(filename), format=format)
+        file.close()

trackio/media/image.py

@@ -0,0 +1,84 @@
+import os
+import shutil
+from pathlib import Path
+
+import numpy as np
+from PIL import Image as PILImage
+
+from trackio.media.media import TrackioMedia
+
+TrackioImageSourceType = str | Path | np.ndarray | PILImage.Image
+
+
+class TrackioImage(TrackioMedia):
+    """
+    Initializes an Image object.
+
+    Example:
+        ```python
+        import trackio
+        import numpy as np
+        from PIL import Image
+
+        # Create an image from numpy array
+        image_data = np.random.randint(0, 255, (64, 64, 3), dtype=np.uint8)
+        image = trackio.Image(image_data, caption="Random image")
+        trackio.log({"my_image": image})
+
+        # Create an image from PIL Image
+        pil_image = Image.new('RGB', (100, 100), color='red')
+        image = trackio.Image(pil_image, caption="Red square")
+        trackio.log({"red_image": image})
+
+        # Create an image from file path
+        image = trackio.Image("path/to/image.jpg", caption="Photo from file")
+        trackio.log({"file_image": image})
+        ```
+
+    Args:
+        value (`str`, `Path`, `numpy.ndarray`, or `PIL.Image`, *optional*):
+            A path to an image, a PIL Image, or a numpy array of shape (height, width, channels).
+            If numpy array, should be of type `np.uint8` with RGB values in the range `[0, 255]`.
+        caption (`str`, *optional*):
+            A string caption for the image.
+    """
+
+    TYPE = "trackio.image"
+
+    def __init__(self, value: TrackioImageSourceType, caption: str | None = None):
+        super().__init__(value, caption)
+        self._format: str | None = None
+
+        if not isinstance(self._value, TrackioImageSourceType):
+            raise ValueError(
+                f"Invalid value type, expected {TrackioImageSourceType}, got {type(self._value)}"
+            )
+        if isinstance(self._value, np.ndarray) and self._value.dtype != np.uint8:
+            raise ValueError(
+                f"Invalid value dtype, expected np.uint8, got {self._value.dtype}"
+            )
+        if (
+            isinstance(self._value, np.ndarray | PILImage.Image)
+            and self._format is None
+        ):
+            self._format = "png"
+
+    def _as_pil(self) -> PILImage.Image | None:
+        try:
+            if isinstance(self._value, np.ndarray):
+                arr = np.asarray(self._value).astype("uint8")
+                return PILImage.fromarray(arr).convert("RGBA")
+            if isinstance(self._value, PILImage.Image):
+                return self._value.convert("RGBA")
+        except Exception as e:
+            raise ValueError(f"Failed to process image data: {self._value}") from e
+        return None
+
+    def _save_media(self, file_path: Path):
+        if pil := self._as_pil():
+            pil.save(file_path, format=self._format)
+        elif isinstance(self._value, str | Path):
+            if os.path.isfile(self._value):
+                shutil.copy(self._value, file_path)
+            else:
+                raise ValueError(f"File not found: {self._value}")

trackio/media/media.py

@@ -0,0 +1,79 @@
+import os
+import uuid
+from abc import ABC, abstractmethod
+from pathlib import Path
+
+from trackio.media.utils import get_project_media_path
+from trackio.utils import MEDIA_DIR
+
+
+class TrackioMedia(ABC):
+    """
+    Abstract base class for Trackio media objects
+    Provides shared functionality for file handling and serialization.
+    """
+
+    TYPE: str
+
+    def __init_subclass__(cls, **kwargs):
+        """Ensure subclasses define the TYPE attribute."""
+        super().__init_subclass__(**kwargs)
+        if not hasattr(cls, "TYPE") or cls.TYPE is None:
+            raise TypeError(f"Class {cls.__name__} must define TYPE attribute")
+
+    def __init__(self, value, caption: str | None = None):
+        """
+        Saves the value and caption, and if the value is a file path, checks if the file exists.
+        """
+        self.caption = caption
+        self._value = value
+        self._file_path: Path | None = None
+
+        if isinstance(self._value, str | Path):
+            if not os.path.isfile(self._value):
+                raise ValueError(f"File not found: {self._value}")
+
+    def _file_extension(self) -> str:
+        if self._file_path:
+            return self._file_path.suffix[1:].lower()
+        if isinstance(self._value, str | Path):
+            path = Path(self._value)
+            return path.suffix[1:].lower()
+        if hasattr(self, "_format") and self._format:
+            return self._format
+        return "unknown"
+
+    def _get_relative_file_path(self) -> Path | None:
+        return self._file_path
+
+    def _get_absolute_file_path(self) -> Path | None:
+        if self._file_path:
+            return MEDIA_DIR / self._file_path
+        return None
+
+    def _save(self, project: str, run: str, step: int = 0):
+        if self._file_path:
+            return
+
+        media_dir = get_project_media_path(project=project, run=run, step=step)
+        filename = f"{uuid.uuid4()}.{self._file_extension()}"
+        file_path = media_dir / filename
+
+        self._save_media(file_path)
+        self._file_path = file_path.relative_to(MEDIA_DIR)
+
+    @abstractmethod
+    def _save_media(self, file_path: Path):
+        """
+        Performs the actual media saving logic.
+        """
+        pass
+
+    def _to_dict(self) -> dict:
+        if not self._file_path:
+            raise ValueError("Media must be saved to file before serialization")
+        return {
+            "_type": self.TYPE,
+            "file_path": str(self._get_relative_file_path()),
+            "caption": self.caption,
+        }

trackio/media/utils.py

@@ -0,0 +1,60 @@
+import shutil
+from pathlib import Path
+
+from trackio.utils import MEDIA_DIR
+
+
+def check_path(file_path: str | Path) -> None:
+    """Raise an error if the parent directory does not exist."""
+    file_path = Path(file_path)
+    if not file_path.parent.exists():
+        try:
+            file_path.parent.mkdir(parents=True, exist_ok=True)
+        except OSError as e:
+            raise ValueError(
+                f"Failed to create parent directory {file_path.parent}: {e}"
+            )
+
+
+def check_ffmpeg_installed() -> None:
+    """Raise an error if ffmpeg is not available on the system PATH."""
+    if shutil.which("ffmpeg") is None:
+        raise RuntimeError(
+            "ffmpeg is required to write video but was not found on your system. "
+            "Please install ffmpeg and ensure it is available on your PATH."
+        )
+
+
+def get_project_media_path(
+    project: str,
+    run: str | None = None,
+    step: int | None = None,
+    relative_path: str | Path | None = None,
+) -> Path:
+    """
+    Get the full path where uploaded files are stored for a Trackio project (and create the directory if it doesn't exist).
+    If a run is not provided, the files are stored in a project-level directory with the given relative path.
+
+    Args:
+        project: The project name
+        run: The run name
+        step: The step number
+        relative_path: The relative path within the directory (only used if run is not provided)
+
+    Returns:
+        The full path to the media file
+    """
+    if step is not None and run is None:
+        raise ValueError("Uploading files at a specific step requires a run")
+
+    path = MEDIA_DIR / project
+    if run:
+        path /= run
+        if step is not None:
+            path /= str(step)
+    else:
+        path /= "files"
+        if relative_path:
+            path /= relative_path
+    path.mkdir(parents=True, exist_ok=True)
+    return path

trackio/media/video.py

@@ -0,0 +1,246 @@
+import os
+import shutil
+import subprocess
+from pathlib import Path
+from typing import Literal
+
+import numpy as np
+
+from trackio.media.media import TrackioMedia
+from trackio.media.utils import check_ffmpeg_installed, check_path
+
+TrackioVideoSourceType = str | Path | np.ndarray
+TrackioVideoFormatType = Literal["gif", "mp4", "webm"]
+VideoCodec = Literal["h264", "vp9", "gif"]
+
+
+class TrackioVideo(TrackioMedia):
+    """
+    Initializes a Video object.
+
+    Example:
+        ```python
+        import trackio
+        import numpy as np
+
+        # Create a simple video from numpy array
+        frames = np.random.randint(0, 255, (10, 3, 64, 64), dtype=np.uint8)
+        video = trackio.Video(frames, caption="Random video", fps=30)
+
+        # Create a batch of videos
+        batch_frames = np.random.randint(0, 255, (3, 10, 3, 64, 64), dtype=np.uint8)
+        batch_video = trackio.Video(batch_frames, caption="Batch of videos", fps=15)
+
+        # Create video from file path
+        video = trackio.Video("path/to/video.mp4", caption="Video from file")
+        ```
+
+    Args:
+        value (`str`, `Path`, or `numpy.ndarray`, *optional*):
+            A path to a video file, or a numpy array.
+            If numpy array, should be of type `np.uint8` with RGB values in the range `[0, 255]`.
+            It is expected to have shape of either (frames, channels, height, width) or (batch, frames, channels, height, width).
+            For the latter, the videos will be tiled into a grid.
+        caption (`str`, *optional*):
+            A string caption for the video.
+        fps (`int`, *optional*):
+            Frames per second for the video. Only used when value is an ndarray. Default is `24`.
+        format (`Literal["gif", "mp4", "webm"]`, *optional*):
+            Video format ("gif", "mp4", or "webm"). Only used when value is an ndarray. Default is "gif".
+    """
+
+    TYPE = "trackio.video"
+
+    def __init__(
+        self,
+        value: TrackioVideoSourceType,
+        caption: str | None = None,
+        fps: int | None = None,
+        format: TrackioVideoFormatType | None = None,
+    ):
+        super().__init__(value, caption)
+
+        if not isinstance(self._value, TrackioVideoSourceType):
+            raise ValueError(
+                f"Invalid value type, expected {TrackioVideoSourceType}, got {type(self._value)}"
+            )
+        if isinstance(self._value, np.ndarray):
+            if self._value.dtype != np.uint8:
+                raise ValueError(
+                    f"Invalid value dtype, expected np.uint8, got {self._value.dtype}"
+                )
+            if format is None:
+                format = "gif"
+            if fps is None:
+                fps = 24
+        self._fps = fps
+        self._format = format
+
+    @staticmethod
+    def _check_array_format(video: np.ndarray) -> None:
+        """Raise an error if the array is not in the expected format."""
+        if not (video.ndim == 4 and video.shape[-1] == 3):
+            raise ValueError(
+                f"Expected RGB input shaped (F, H, W, 3), got {video.shape}. "
+                f"Input has {video.ndim} dimensions, expected 4."
+            )
+        if video.dtype != np.uint8:
+            raise TypeError(
+                f"Expected dtype=uint8, got {video.dtype}. "
+                "Please convert your video data to uint8 format."
+            )
+
+    @staticmethod
+    def write_video(
+        file_path: str | Path, video: np.ndarray, fps: float, codec: VideoCodec
+    ) -> None:
+        """RGB uint8 only, shape (F, H, W, 3)."""
+        check_ffmpeg_installed()
+        check_path(file_path)
+
+        if codec not in {"h264", "vp9", "gif"}:
+            raise ValueError("Unsupported codec. Use h264, vp9, or gif.")
+
+        arr = np.asarray(video)
+        TrackioVideo._check_array_format(arr)
+
+        frames = np.ascontiguousarray(arr)
+        _, height, width, _ = frames.shape
+        out_path = str(file_path)
+
+        cmd = [
+            "ffmpeg",
+            "-y",
+            "-f",
+            "rawvideo",
+            "-s",
+            f"{width}x{height}",
+            "-pix_fmt",
+            "rgb24",
+            "-r",
+            str(fps),
+            "-i",
+            "-",
+            "-an",
+        ]
+
+        if codec == "gif":
+            video_filter = "split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse"
+            cmd += [
+                "-vf",
+                video_filter,
+                "-loop",
+                "0",
+            ]
+        elif codec == "h264":
+            cmd += [
+                "-vcodec",
+                "libx264",
+                "-pix_fmt",
+                "yuv420p",
+                "-movflags",
+                "+faststart",
+            ]
+        elif codec == "vp9":
+            bpp = 0.08
+            bps = int(width * height * fps * bpp)
+            if bps >= 1_000_000:
+                bitrate = f"{round(bps / 1_000_000)}M"
+            elif bps >= 1_000:
+                bitrate = f"{round(bps / 1_000)}k"
+            else:
+                bitrate = str(max(bps, 1))
+            cmd += [
+                "-vcodec",
+                "libvpx-vp9",
+                "-b:v",
+                bitrate,
+                "-pix_fmt",
+                "yuv420p",
+            ]
+        cmd += [out_path]
+        proc = subprocess.Popen(cmd, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+        try:
+            for frame in frames:
+                proc.stdin.write(frame.tobytes())
+        finally:
+            if proc.stdin:
+                proc.stdin.close()
+            stderr = (
+                proc.stderr.read().decode("utf-8", errors="ignore")
+                if proc.stderr
+                else ""
+            )
+            ret = proc.wait()
+            if ret != 0:
+                raise RuntimeError(f"ffmpeg failed with code {ret}\n{stderr}")
+
+    @property
+    def _codec(self) -> str:
+        match self._format:
+            case "gif":
+                return "gif"
+            case "mp4":
+                return "h264"
+            case "webm":
+                return "vp9"
+            case _:
+                raise ValueError(f"Unsupported format: {self._format}")
+
+    def _save_media(self, file_path: Path):
+        if isinstance(self._value, np.ndarray):
+            video = TrackioVideo._process_ndarray(self._value)
+            TrackioVideo.write_video(file_path, video, fps=self._fps, codec=self._codec)
+        elif isinstance(self._value, str | Path):
+            if os.path.isfile(self._value):
+                shutil.copy(self._value, file_path)
+            else:
+                raise ValueError(f"File not found: {self._value}")
+
+    @staticmethod
+    def _process_ndarray(value: np.ndarray) -> np.ndarray:
+        # Verify value is either 4D (single video) or 5D array (batched videos).
+        # Expected format: (frames, channels, height, width) or (batch, frames, channels, height, width)
+        if value.ndim < 4:
+            raise ValueError(
+                "Video requires at least 4 dimensions (frames, channels, height, width)"
+            )
+        if value.ndim > 5:
+            raise ValueError(
+                "Videos can have at most 5 dimensions (batch, frames, channels, height, width)"
+            )
+        if value.ndim == 4:
+            # Reshape to 5D with single batch: (1, frames, channels, height, width)
+            value = value[np.newaxis, ...]
+
+        value = TrackioVideo._tile_batched_videos(value)
+        return value
+
+    @staticmethod
+    def _tile_batched_videos(video: np.ndarray) -> np.ndarray:
+        """
+        Tiles a batch of videos into a grid of videos.
+
+        Input format: (batch, frames, channels, height, width) - original FCHW format
+        Output format: (frames, total_height, total_width, channels)
+        """
+        batch_size, frames, channels, height, width = video.shape
+
+        next_pow2 = 1 << (batch_size - 1).bit_length()
+        if batch_size != next_pow2:
+            pad_len = next_pow2 - batch_size
+            pad_shape = (pad_len, frames, channels, height, width)
+            padding = np.zeros(pad_shape, dtype=video.dtype)
+            video = np.concatenate((video, padding), axis=0)
+            batch_size = next_pow2
+
+        n_rows = 1 << ((batch_size.bit_length() - 1) // 2)
+        n_cols = batch_size // n_rows
+
+        # Reshape to grid layout: (n_rows, n_cols, frames, channels, height, width)
+        video = video.reshape(n_rows, n_cols, frames, channels, height, width)
+
+        # Rearrange dimensions to (frames, total_height, total_width, channels)
+        video = video.transpose(2, 0, 4, 1, 5, 3)
+        video = video.reshape(frames, n_rows * height, n_cols * width, channels)
+        return video

trackio/package.json

@@ -0,0 +1,6 @@
+{
+	"name": "trackio",
+	"version": "0.20.0",
+	"description": "",
+	"python": "true"
+}

trackio/remote_client.py

@@ -0,0 +1,28 @@
+from __future__ import annotations
+
+from gradio_client import Client
+
+
+class RemoteClient:
+    def __init__(self, space: str, hf_token: str | None = None):
+        self._space = space
+        kwargs: dict = {"verbose": False}
+        if hf_token:
+            kwargs["hf_token"] = hf_token
+        try:
+            self._client = Client(space, **kwargs)
+        except Exception as e:
+            raise ConnectionError(
+                f"Could not connect to Space '{space}'. Is it running?\n{e}"
+            )
+
+    def predict(self, *args, api_name: str):
+        try:
+            return self._client.predict(*args, api_name=api_name)
+        except Exception as e:
+            if "API Not Found" in str(e) or "api_name" in str(e):
+                raise RuntimeError(
+                    f"Space '{self._space}' does not support '{api_name}'. "
+                    "Redeploy with `trackio sync`."
+                )
+            raise

trackio/run.py

@@ -0,0 +1,739 @@
+import os
+import shutil
+import threading
+import uuid
+import warnings
+from datetime import datetime, timezone
+from pathlib import Path
+
+import huggingface_hub
+from gradio_client import Client, handle_file
+
+from trackio import utils
+from trackio.alerts import (
+    AlertLevel,
+    format_alert_terminal,
+    resolve_webhook_min_level,
+    send_webhook,
+    should_send_webhook,
+)
+from trackio.apple_gpu import AppleGpuMonitor, apple_gpu_available
+from trackio.gpu import GpuMonitor, gpu_available
+from trackio.histogram import Histogram
+from trackio.markdown import Markdown
+from trackio.media import TrackioMedia, get_project_media_path
+from trackio.sqlite_storage import SQLiteStorage
+from trackio.table import Table
+from trackio.typehints import AlertEntry, LogEntry, SystemLogEntry, UploadEntry
+from trackio.utils import _get_default_namespace
+
+BATCH_SEND_INTERVAL = 0.5
+MAX_BACKOFF = 30
+
+
+class Run:
+    def __init__(
+        self,
+        url: str | None,
+        project: str,
+        client: Client | None,
+        name: str | None = None,
+        group: str | None = None,
+        config: dict | None = None,
+        space_id: str | None = None,
+        auto_log_gpu: bool = False,
+        gpu_log_interval: float = 10.0,
+        webhook_url: str | None = None,
+        webhook_min_level: AlertLevel | str | None = None,
+    ):
+        """
+        Initialize a Run for logging metrics to Trackio.
+
+        Args:
+            url: The URL of the Trackio server (local Gradio app or HF Space).
+            project: The name of the project to log metrics to.
+            client: A pre-configured gradio_client.Client instance, or None to
+                create one automatically in a background thread with retry logic.
+                Passing None is recommended for normal usage. Passing a client
+                is useful for testing (e.g., injecting a mock client).
+            name: The name of this run. If None, a readable name like
+                "brave-sunset-0" is auto-generated. If space_id is provided,
+                generates a "username-timestamp" format instead.
+            group: Optional group name to organize related runs together.
+            config: A dictionary of configuration/hyperparameters for this run.
+                Keys starting with '_' are reserved for internal use.
+            space_id: The HF Space ID if logging to a Space (e.g., "user/space").
+                If provided, media files will be uploaded to the Space.
+            auto_log_gpu: Whether to automatically log GPU metrics (utilization,
+                memory, temperature) at regular intervals.
+            gpu_log_interval: The interval in seconds between GPU metric logs.
+                Only used when auto_log_gpu is True.
+            webhook_url: A webhook URL to POST alert payloads to. Supports
+                Slack and Discord webhook URLs natively. Can also be set via
+                the TRACKIO_WEBHOOK_URL environment variable.
+            webhook_min_level: Minimum alert level that should trigger webhook
+                delivery. For example, `AlertLevel.WARN` sends only WARN and
+                ERROR alerts to webhook destinations. Can also be set via
+                `TRACKIO_WEBHOOK_MIN_LEVEL`.
+        """
+        self.url = url
+        self.project = project
+        self._client_lock = threading.Lock()
+        self._client_thread = None
+        self._client = client
+        self._space_id = space_id
+        self.name = name or utils.generate_readable_name(
+            SQLiteStorage.get_runs(project), space_id
+        )
+        self.group = group
+        self.config = utils.to_json_safe(config or {})
+
+        if isinstance(self.config, dict):
+            for key in self.config:
+                if key.startswith("_"):
+                    raise ValueError(
+                        f"Config key '{key}' is reserved (keys starting with '_' are reserved for internal use)"
+                    )
+
+        self.config["_Username"] = self._get_username()
+        self.config["_Created"] = datetime.now(timezone.utc).isoformat()
+        self.config["_Group"] = self.group
+
+        self._queued_logs: list[LogEntry] = []
+        self._queued_system_logs: list[SystemLogEntry] = []
+        self._queued_uploads: list[UploadEntry] = []
+        self._queued_alerts: list[AlertEntry] = []
+        self._stop_flag = threading.Event()
+        self._config_logged = False
+        max_step = SQLiteStorage.get_max_step_for_run(self.project, self.name)
+        self._next_step = 0 if max_step is None else max_step + 1
+        self._has_local_buffer = False
+
+        self._is_local = space_id is None
+        self._webhook_url = webhook_url or os.environ.get("TRACKIO_WEBHOOK_URL")
+        self._webhook_min_level = resolve_webhook_min_level(
+            webhook_min_level or os.environ.get("TRACKIO_WEBHOOK_MIN_LEVEL")
+        )
+
+        if self._is_local:
+            self._local_sender_thread = threading.Thread(
+                target=self._local_batch_sender
+            )
+            self._local_sender_thread.daemon = True
+            self._local_sender_thread.start()
+        else:
+            self._client_thread = threading.Thread(target=self._init_client_background)
+            self._client_thread.daemon = True
+            self._client_thread.start()
+
+        self._gpu_monitor: "GpuMonitor | AppleGpuMonitor | None" = None
+        if auto_log_gpu:
+            if gpu_available():
+                self._gpu_monitor = GpuMonitor(self, interval=gpu_log_interval)
+                self._gpu_monitor.start()
+            elif apple_gpu_available():
+                self._gpu_monitor = AppleGpuMonitor(self, interval=gpu_log_interval)
+                self._gpu_monitor.start()
+
+    def _get_username(self) -> str | None:
+        try:
+            return _get_default_namespace()
+        except Exception:
+            return None
+
+    def _local_batch_sender(self):
+        while (
+            not self._stop_flag.is_set()
+            or len(self._queued_logs) > 0
+            or len(self._queued_system_logs) > 0
+            or len(self._queued_alerts) > 0
+        ):
+            if not self._stop_flag.is_set():
+                self._stop_flag.wait(timeout=BATCH_SEND_INTERVAL)
+
+            with self._client_lock:
+                if self._queued_logs:
+                    logs_to_send = self._queued_logs.copy()
+                    self._queued_logs.clear()
+                    self._write_logs_to_sqlite(logs_to_send)
+
+                if self._queued_system_logs:
+                    system_logs_to_send = self._queued_system_logs.copy()
+                    self._queued_system_logs.clear()
+                    self._write_system_logs_to_sqlite(system_logs_to_send)
+
+                if self._queued_alerts:
+                    alerts_to_send = self._queued_alerts.copy()
+                    self._queued_alerts.clear()
+                    self._write_alerts_to_sqlite(alerts_to_send)
+
+    def _write_logs_to_sqlite(self, logs: list[LogEntry]):
+        logs_by_run: dict[tuple, dict] = {}
+        for entry in logs:
+            key = (entry["project"], entry["run"])
+            if key not in logs_by_run:
+                logs_by_run[key] = {
+                    "metrics": [],
+                    "steps": [],
+                    "log_ids": [],
+                    "config": None,
+                }
+            logs_by_run[key]["metrics"].append(entry["metrics"])
+            logs_by_run[key]["steps"].append(entry.get("step"))
+            logs_by_run[key]["log_ids"].append(entry.get("log_id"))
+            if entry.get("config") and logs_by_run[key]["config"] is None:
+                logs_by_run[key]["config"] = entry["config"]
+
+        for (project, run), data in logs_by_run.items():
+            has_log_ids = any(lid is not None for lid in data["log_ids"])
+            SQLiteStorage.bulk_log(
+                project=project,
+                run=run,
+                metrics_list=data["metrics"],
+                steps=data["steps"],
+                config=data["config"],
+                log_ids=data["log_ids"] if has_log_ids else None,
+            )
+
+    def _write_system_logs_to_sqlite(self, logs: list[SystemLogEntry]):
+        logs_by_run: dict[tuple, dict] = {}
+        for entry in logs:
+            key = (entry["project"], entry["run"])
+            if key not in logs_by_run:
+                logs_by_run[key] = {"metrics": [], "timestamps": [], "log_ids": []}
+            logs_by_run[key]["metrics"].append(entry["metrics"])
+            logs_by_run[key]["timestamps"].append(entry.get("timestamp"))
+            logs_by_run[key]["log_ids"].append(entry.get("log_id"))
+
+        for (project, run), data in logs_by_run.items():
+            has_log_ids = any(lid is not None for lid in data["log_ids"])
+            SQLiteStorage.bulk_log_system(
+                project=project,
+                run=run,
+                metrics_list=data["metrics"],
+                timestamps=data["timestamps"],
+                log_ids=data["log_ids"] if has_log_ids else None,
+            )
+
+    def _write_alerts_to_sqlite(self, alerts: list[AlertEntry]):
+        alerts_by_run: dict[tuple, dict] = {}
+        for entry in alerts:
+            key = (entry["project"], entry["run"])
+            if key not in alerts_by_run:
+                alerts_by_run[key] = {
+                    "titles": [],
+                    "texts": [],
+                    "levels": [],
+                    "steps": [],
+                    "timestamps": [],
+                    "alert_ids": [],
+                }
+            alerts_by_run[key]["titles"].append(entry["title"])
+            alerts_by_run[key]["texts"].append(entry.get("text"))
+            alerts_by_run[key]["levels"].append(entry["level"])
+            alerts_by_run[key]["steps"].append(entry.get("step"))
+            alerts_by_run[key]["timestamps"].append(entry.get("timestamp"))
+            alerts_by_run[key]["alert_ids"].append(entry.get("alert_id"))
+
+        for (project, run), data in alerts_by_run.items():
+            has_alert_ids = any(aid is not None for aid in data["alert_ids"])
+            SQLiteStorage.bulk_alert(
+                project=project,
+                run=run,
+                titles=data["titles"],
+                texts=data["texts"],
+                levels=data["levels"],
+                steps=data["steps"],
+                timestamps=data["timestamps"],
+                alert_ids=data["alert_ids"] if has_alert_ids else None,
+            )
+
+    def _batch_sender(self):
+        consecutive_failures = 0
+        while (
+            not self._stop_flag.is_set()
+            or len(self._queued_logs) > 0
+            or len(self._queued_system_logs) > 0
+            or len(self._queued_uploads) > 0
+            or len(self._queued_alerts) > 0
+            or self._has_local_buffer
+        ):
+            if not self._stop_flag.is_set():
+                if consecutive_failures:
+                    sleep_time = min(
+                        BATCH_SEND_INTERVAL * (2**consecutive_failures), MAX_BACKOFF
+                    )
+                else:
+                    sleep_time = BATCH_SEND_INTERVAL
+                self._stop_flag.wait(timeout=sleep_time)
+            elif self._has_local_buffer:
+                self._stop_flag.wait(timeout=BATCH_SEND_INTERVAL)
+
+            with self._client_lock:
+                if self._client is None:
+                    if self._stop_flag.is_set():
+                        if self._queued_logs:
+                            self._persist_logs_locally(self._queued_logs)
+                            self._queued_logs.clear()
+                        if self._queued_system_logs:
+                            self._persist_system_logs_locally(self._queued_system_logs)
+                            self._queued_system_logs.clear()
+                        if self._queued_uploads:
+                            self._persist_uploads_locally(self._queued_uploads)
+                            self._queued_uploads.clear()
+                        if self._queued_alerts:
+                            self._write_alerts_to_sqlite(self._queued_alerts)
+                            self._queued_alerts.clear()
+                    return
+
+                failed = False
+
+                if self._queued_logs:
+                    logs_to_send = self._queued_logs.copy()
+                    self._queued_logs.clear()
+                    try:
+                        self._client.predict(
+                            api_name="/bulk_log",
+                            logs=logs_to_send,
+                            hf_token=huggingface_hub.utils.get_token(),
+                        )
+                    except Exception:
+                        self._persist_logs_locally(logs_to_send)
+                        failed = True
+
+                if self._queued_system_logs:
+                    system_logs_to_send = self._queued_system_logs.copy()
+                    self._queued_system_logs.clear()
+                    try:
+                        self._client.predict(
+                            api_name="/bulk_log_system",
+                            logs=system_logs_to_send,
+                            hf_token=huggingface_hub.utils.get_token(),
+                        )
+                    except Exception:
+                        self._persist_system_logs_locally(system_logs_to_send)
+                        failed = True
+
+                if self._queued_uploads:
+                    uploads_to_send = self._queued_uploads.copy()
+                    self._queued_uploads.clear()
+                    try:
+                        self._client.predict(
+                            api_name="/bulk_upload_media",
+                            uploads=uploads_to_send,
+                            hf_token=huggingface_hub.utils.get_token(),
+                        )
+                    except Exception:
+                        self._persist_uploads_locally(uploads_to_send)
+                        failed = True
+
+                if self._queued_alerts:
+                    alerts_to_send = self._queued_alerts.copy()
+                    self._queued_alerts.clear()
+                    try:
+                        self._client.predict(
+                            api_name="/bulk_alert",
+                            alerts=alerts_to_send,
+                            hf_token=huggingface_hub.utils.get_token(),
+                        )
+                    except Exception:
+                        self._write_alerts_to_sqlite(alerts_to_send)
+                        failed = True
+
+                if failed:
+                    consecutive_failures += 1
+                else:
+                    consecutive_failures = 0
+                    if self._has_local_buffer:
+                        self._flush_local_buffer()
+
+    def _persist_logs_locally(self, logs: list[LogEntry]):
+        if not self._space_id:
+            return
+        logs_by_run: dict[tuple, dict] = {}
+        for entry in logs:
+            key = (entry["project"], entry["run"])
+            if key not in logs_by_run:
+                logs_by_run[key] = {
+                    "metrics": [],
+                    "steps": [],
+                    "log_ids": [],
+                    "config": None,
+                }
+            logs_by_run[key]["metrics"].append(entry["metrics"])
+            logs_by_run[key]["steps"].append(entry.get("step"))
+            logs_by_run[key]["log_ids"].append(entry.get("log_id"))
+            if entry.get("config") and logs_by_run[key]["config"] is None:
+                logs_by_run[key]["config"] = entry["config"]
+
+        for (project, run), data in logs_by_run.items():
+            SQLiteStorage.bulk_log(
+                project=project,
+                run=run,
+                metrics_list=data["metrics"],
+                steps=data["steps"],
+                log_ids=data["log_ids"],
+                config=data["config"],
+                space_id=self._space_id,
+            )
+        self._has_local_buffer = True
+
+    def _persist_system_logs_locally(self, logs: list[SystemLogEntry]):
+        if not self._space_id:
+            return
+        logs_by_run: dict[tuple, dict] = {}
+        for entry in logs:
+            key = (entry["project"], entry["run"])
+            if key not in logs_by_run:
+                logs_by_run[key] = {"metrics": [], "timestamps": [], "log_ids": []}
+            logs_by_run[key]["metrics"].append(entry["metrics"])
+            logs_by_run[key]["timestamps"].append(entry.get("timestamp"))
+            logs_by_run[key]["log_ids"].append(entry.get("log_id"))
+
+        for (project, run), data in logs_by_run.items():
+            SQLiteStorage.bulk_log_system(
+                project=project,
+                run=run,
+                metrics_list=data["metrics"],
+                timestamps=data["timestamps"],
+                log_ids=data["log_ids"],
+                space_id=self._space_id,
+            )
+        self._has_local_buffer = True
+
+    def _persist_uploads_locally(self, uploads: list[UploadEntry]):
+        if not self._space_id:
+            return
+        for entry in uploads:
+            file_data = entry.get("uploaded_file")
+            file_path = ""
+            if isinstance(file_data, dict):
+                file_path = file_data.get("path", "")
+            elif hasattr(file_data, "path"):
+                file_path = str(file_data.path)
+            else:
+                file_path = str(file_data)
+            SQLiteStorage.add_pending_upload(
+                project=entry["project"],
+                space_id=self._space_id,
+                run_name=entry.get("run"),
+                step=entry.get("step"),
+                file_path=file_path,
+                relative_path=entry.get("relative_path"),
+            )
+        self._has_local_buffer = True
+
+    def _flush_local_buffer(self):
+        try:
+            buffered_logs = SQLiteStorage.get_pending_logs(self.project)
+            if buffered_logs:
+                self._client.predict(
+                    api_name="/bulk_log",
+                    logs=buffered_logs["logs"],
+                    hf_token=huggingface_hub.utils.get_token(),
+                )
+                SQLiteStorage.clear_pending_logs(self.project, buffered_logs["ids"])
+
+            buffered_sys = SQLiteStorage.get_pending_system_logs(self.project)
+            if buffered_sys:
+                self._client.predict(
+                    api_name="/bulk_log_system",
+                    logs=buffered_sys["logs"],
+                    hf_token=huggingface_hub.utils.get_token(),
+                )
+                SQLiteStorage.clear_pending_system_logs(
+                    self.project, buffered_sys["ids"]
+                )
+
+            buffered_uploads = SQLiteStorage.get_pending_uploads(self.project)
+            if buffered_uploads:
+                upload_entries = []
+                for u in buffered_uploads["uploads"]:
+                    fp = u["file_path"]
+                    if Path(fp).exists():
+                        upload_entries.append(
+                            {
+                                "project": u["project"],
+                                "run": u["run"],
+                                "step": u["step"],
+                                "relative_path": u["relative_path"],
+                                "uploaded_file": handle_file(fp),
+                            }
+                        )
+                if upload_entries:
+                    self._client.predict(
+                        api_name="/bulk_upload_media",
+                        uploads=upload_entries,
+                        hf_token=huggingface_hub.utils.get_token(),
+                    )
+                SQLiteStorage.clear_pending_uploads(
+                    self.project, buffered_uploads["ids"]
+                )
+
+            self._has_local_buffer = False
+        except Exception:
+            pass
+
+    def _init_client_background(self):
+        if self._client is None:
+            fib = utils.fibo()
+            for sleep_coefficient in fib:
+                if self._stop_flag.is_set():
+                    break
+                try:
+                    client = Client(self.url, verbose=False)
+
+                    with self._client_lock:
+                        self._client = client
+                    break
+                except Exception:
+                    pass
+                sleep_time = min(0.1 * sleep_coefficient, MAX_BACKOFF)
+                self._stop_flag.wait(timeout=sleep_time)
+
+        self._batch_sender()
+
+    def _queue_upload(
+        self,
+        file_path,
+        step: int | None,
+        relative_path: str | None = None,
+        use_run_name: bool = True,
+    ):
+        if self._is_local:
+            self._save_upload_locally(file_path, step, relative_path, use_run_name)
+        else:
+            upload_entry: UploadEntry = {
+                "project": self.project,
+                "run": self.name if use_run_name else None,
+                "step": step,
+                "relative_path": relative_path,
+                "uploaded_file": handle_file(file_path),
+            }
+            with self._client_lock:
+                self._queued_uploads.append(upload_entry)
+
+    def _save_upload_locally(
+        self,
+        file_path,
+        step: int | None,
+        relative_path: str | None = None,
+        use_run_name: bool = True,
+    ):
+        media_path = get_project_media_path(
+            project=self.project,
+            run=self.name if use_run_name else None,
+            step=step,
+            relative_path=relative_path,
+        )
+        src = Path(file_path)
+        if src.exists() and str(src.resolve()) != str(Path(media_path).resolve()):
+            shutil.copy(str(src), str(media_path))
+
+    def _process_media(self, value: TrackioMedia, step: int | None) -> dict:
+        value._save(self.project, self.name, step if step is not None else 0)
+        if self._space_id:
+            self._queue_upload(value._get_absolute_file_path(), step)
+        return value._to_dict()
+
+    def _scan_and_queue_media_uploads(self, table_dict: dict, step: int | None):
+        if not self._space_id:
+            return
+
+        table_data = table_dict.get("_value", [])
+        for row in table_data:
+            for value in row.values():
+                if isinstance(value, dict) and value.get("_type") in [
+                    "trackio.image",
+                    "trackio.video",
+                    "trackio.audio",
+                ]:
+                    file_path = value.get("file_path")
+                    if file_path:
+                        from trackio.utils import MEDIA_DIR
+
+                        absolute_path = MEDIA_DIR / file_path
+                        self._queue_upload(absolute_path, step)
+                elif isinstance(value, list):
+                    for item in value:
+                        if isinstance(item, dict) and item.get("_type") in [
+                            "trackio.image",
+                            "trackio.video",
+                            "trackio.audio",
+                        ]:
+                            file_path = item.get("file_path")
+                            if file_path:
+                                from trackio.utils import MEDIA_DIR
+
+                                absolute_path = MEDIA_DIR / file_path
+                                self._queue_upload(absolute_path, step)
+
+    def _ensure_sender_alive(self):
+        if self._is_local:
+            if (
+                hasattr(self, "_local_sender_thread")
+                and not self._local_sender_thread.is_alive()
+                and not self._stop_flag.is_set()
+            ):
+                self._local_sender_thread = threading.Thread(
+                    target=self._local_batch_sender
+                )
+                self._local_sender_thread.daemon = True
+                self._local_sender_thread.start()
+        else:
+            if (
+                self._client_thread is not None
+                and not self._client_thread.is_alive()
+                and not self._stop_flag.is_set()
+            ):
+                self._client_thread = threading.Thread(
+                    target=self._init_client_background
+                )
+                self._client_thread.daemon = True
+                self._client_thread.start()
+
+    def log(self, metrics: dict, step: int | None = None):
+        renamed_keys = []
+        new_metrics = {}
+
+        for k, v in metrics.items():
+            if k in utils.RESERVED_KEYS or k.startswith("__"):
+                new_key = f"__{k}"
+                renamed_keys.append(k)
+                new_metrics[new_key] = v
+            else:
+                new_metrics[k] = v
+
+        if renamed_keys:
+            warnings.warn(f"Reserved keys renamed: {renamed_keys} → '__{{key}}'")
+
+        metrics = new_metrics
+        for key, value in metrics.items():
+            if isinstance(value, Table):
+                metrics[key] = value._to_dict(
+                    project=self.project, run=self.name, step=step
+                )
+                self._scan_and_queue_media_uploads(metrics[key], step)
+            elif isinstance(value, Histogram):
+                metrics[key] = value._to_dict()
+            elif isinstance(value, Markdown):
+                metrics[key] = value._to_dict()
+            elif isinstance(value, TrackioMedia):
+                metrics[key] = self._process_media(value, step)
+        metrics = utils.serialize_values(metrics)
+
+        if step is None:
+            step = self._next_step
+        self._next_step = max(self._next_step, step + 1)
+
+        config_to_log = None
+        if not self._config_logged and self.config:
+            config_to_log = utils.to_json_safe(self.config)
+            self._config_logged = True
+
+        log_entry: LogEntry = {
+            "project": self.project,
+            "run": self.name,
+            "metrics": metrics,
+            "step": step,
+            "config": config_to_log,
+            "log_id": uuid.uuid4().hex,
+        }
+
+        with self._client_lock:
+            self._queued_logs.append(log_entry)
+            self._ensure_sender_alive()
+
+    def alert(
+        self,
+        title: str,
+        text: str | None = None,
+        level: AlertLevel = AlertLevel.WARN,
+        step: int | None = None,
+        webhook_url: str | None = None,
+    ):
+        if step is None:
+            step = max(self._next_step - 1, 0)
+        timestamp = datetime.now(timezone.utc).isoformat()
+
+        print(format_alert_terminal(level, title, text, step))
+
+        alert_entry: AlertEntry = {
+            "project": self.project,
+            "run": self.name,
+            "title": title,
+            "text": text,
+            "level": level.value,
+            "step": step,
+            "timestamp": timestamp,
+            "alert_id": uuid.uuid4().hex,
+        }
+
+        with self._client_lock:
+            self._queued_alerts.append(alert_entry)
+            self._ensure_sender_alive()
+
+        url = webhook_url or self._webhook_url
+        if url and should_send_webhook(level, self._webhook_min_level):
+            t = threading.Thread(
+                target=send_webhook,
+                args=(
+                    url,
+                    level,
+                    title,
+                    text,
+                    self.project,
+                    self.name,
+                    step,
+                    timestamp,
+                ),
+                daemon=True,
+            )
+            t.start()
+
+    def log_system(self, metrics: dict):
+        metrics = utils.serialize_values(metrics)
+        timestamp = datetime.now(timezone.utc).isoformat()
+
+        system_log_entry: SystemLogEntry = {
+            "project": self.project,
+            "run": self.name,
+            "metrics": metrics,
+            "timestamp": timestamp,
+            "log_id": uuid.uuid4().hex,
+        }
+
+        with self._client_lock:
+            self._queued_system_logs.append(system_log_entry)
+            self._ensure_sender_alive()
+
+    def finish(self):
+        if self._gpu_monitor is not None:
+            self._gpu_monitor.stop()
+
+        self._stop_flag.set()
+
+        if self._is_local:
+            if hasattr(self, "_local_sender_thread"):
+                print("* Run finished. Uploading logs to Trackio (please wait...)")
+                self._local_sender_thread.join(timeout=30)
+                if self._local_sender_thread.is_alive():
+                    warnings.warn(
+                        "Could not flush all logs within 30s. Some data may be buffered locally."
+                    )
+        else:
+            if self._client_thread is not None:
+                print(
+                    "* Run finished. Uploading logs to Trackio Space (please wait...)"
+                )
+                self._client_thread.join(timeout=30)
+                if self._client_thread.is_alive():
+                    warnings.warn(
+                        "Could not flush all logs within 30s. Some data may be buffered locally."
+                    )
+            if SQLiteStorage.has_pending_data(self.project):
+                warnings.warn(
+                    f"* Some logs could not be sent to the Space (it may still be starting up). "
+                    f"They have been saved locally and will be sent automatically next time you call: "
+                    f'trackio.init(project="{self.project}", space_id="{self._space_id}")'
+                )

trackio/server.py

@@ -0,0 +1,626 @@
+"""The main API layer for the Trackio UI."""
+
+import base64
+import os
+import re
+import secrets
+import shutil
+import time
+from functools import lru_cache
+from typing import Any
+from urllib.parse import urlencode
+
+import gradio as gr
+import httpx
+import huggingface_hub as hf
+from starlette.requests import Request
+from starlette.responses import RedirectResponse
+
+import trackio.utils as utils
+from trackio.media import get_project_media_path
+from trackio.sqlite_storage import SQLiteStorage
+from trackio.typehints import AlertEntry, LogEntry, SystemLogEntry, UploadEntry
+
+HfApi = hf.HfApi()
+
+write_token = secrets.token_urlsafe(32)
+
+OAUTH_CALLBACK_PATH = "/login/callback"
+OAUTH_START_PATH = "/oauth/hf/start"
+
+
+def _hf_access_token(request: gr.Request) -> str | None:
+    session_id = None
+    try:
+        session_id = request.headers.get("x-trackio-oauth-session")
+    except (AttributeError, TypeError):
+        pass
+    if session_id and session_id in _oauth_sessions:
+        token, created = _oauth_sessions[session_id]
+        if time.monotonic() - created <= _OAUTH_SESSION_TTL:
+            return token
+        del _oauth_sessions[session_id]
+    cookie_header = ""
+    try:
+        cookie_header = request.headers.get("cookie", "")
+    except (AttributeError, TypeError):
+        pass
+    if cookie_header:
+        for cookie in cookie_header.split(";"):
+            parts = cookie.strip().split("=", 1)
+            if len(parts) == 2 and parts[0] == "trackio_hf_access_token":
+                return parts[1] or None
+    return None
+
+
+def _oauth_redirect_uri(request: Request) -> str:
+    space_host = os.getenv("SPACE_HOST")
+    if space_host:
+        space_host = space_host.split(",")[0]
+        return f"https://{space_host}{OAUTH_CALLBACK_PATH}"
+    return str(request.base_url).rstrip("/") + OAUTH_CALLBACK_PATH
+
+
+class TrackioServer(gr.Server):
+    def close(self, verbose: bool = True) -> None:
+        if self.blocks is None:
+            return
+        if self.blocks.is_running:
+            self.blocks.close(verbose=verbose)
+
+
+_OAUTH_STATE_TTL = 86400
+_OAUTH_SESSION_TTL = 86400 * 30
+_pending_oauth_states: dict[str, float] = {}
+_oauth_sessions: dict[str, tuple[str, float]] = {}
+
+
+def _evict_expired_oauth():
+    now = time.monotonic()
+    expired_states = [
+        k for k, t in _pending_oauth_states.items() if now - t > _OAUTH_STATE_TTL
+    ]
+    for k in expired_states:
+        del _pending_oauth_states[k]
+    expired_sessions = [
+        k for k, (_, t) in _oauth_sessions.items() if now - t > _OAUTH_SESSION_TTL
+    ]
+    for k in expired_sessions:
+        del _oauth_sessions[k]
+
+
+def oauth_hf_start(request: Request):
+    client_id = os.getenv("OAUTH_CLIENT_ID")
+    if not client_id:
+        return RedirectResponse(url="/", status_code=302)
+    _evict_expired_oauth()
+    state = secrets.token_urlsafe(32)
+    _pending_oauth_states[state] = time.monotonic()
+    redirect_uri = _oauth_redirect_uri(request)
+    scope = os.getenv("OAUTH_SCOPES", "openid profile").strip()
+    url = "https://huggingface.co/oauth/authorize?" + urlencode(
+        {
+            "client_id": client_id,
+            "redirect_uri": redirect_uri,
+            "response_type": "code",
+            "scope": scope,
+            "state": state,
+        }
+    )
+    return RedirectResponse(url=url, status_code=302)
+
+
+def oauth_hf_callback(request: Request):
+    client_id = os.getenv("OAUTH_CLIENT_ID")
+    client_secret = os.getenv("OAUTH_CLIENT_SECRET")
+    err = "/?oauth_error=1"
+    if not client_id or not client_secret:
+        return RedirectResponse(url=err, status_code=302)
+    got_state = request.query_params.get("state")
+    code = request.query_params.get("code")
+    if not got_state or got_state not in _pending_oauth_states or not code:
+        return RedirectResponse(url=err, status_code=302)
+    state_created = _pending_oauth_states.pop(got_state)
+    if time.monotonic() - state_created > _OAUTH_STATE_TTL:
+        return RedirectResponse(url=err, status_code=302)
+    redirect_uri = _oauth_redirect_uri(request)
+    auth_b64 = base64.b64encode(f"{client_id}:{client_secret}".encode()).decode()
+    try:
+        with httpx.Client() as client:
+            token_resp = client.post(
+                "https://huggingface.co/oauth/token",
+                headers={"Authorization": f"Basic {auth_b64}"},
+                data={
+                    "grant_type": "authorization_code",
+                    "code": code,
+                    "redirect_uri": redirect_uri,
+                    "client_id": client_id,
+                },
+            )
+            token_resp.raise_for_status()
+        access_token = token_resp.json()["access_token"]
+    except Exception:
+        return RedirectResponse(url=err, status_code=302)
+    session_id = secrets.token_urlsafe(32)
+    _oauth_sessions[session_id] = (access_token, time.monotonic())
+    on_spaces = os.getenv("SYSTEM") == "spaces"
+    resp = RedirectResponse(url=f"/?oauth_session={session_id}", status_code=302)
+    resp.set_cookie(
+        key="trackio_hf_access_token",
+        value=access_token,
+        httponly=True,
+        samesite="none" if on_spaces else "lax",
+        max_age=86400 * 30,
+        path="/",
+        secure=on_spaces,
+    )
+    return resp
+
+
+def oauth_logout(request: Request):
+    on_spaces = os.getenv("SYSTEM") == "spaces"
+    resp = RedirectResponse(url="/", status_code=302)
+    resp.delete_cookie(
+        "trackio_hf_access_token",
+        path="/",
+        samesite="none" if on_spaces else "lax",
+        secure=on_spaces,
+    )
+    return resp
+
+
+@lru_cache(maxsize=32)
+def check_hf_token_has_write_access(hf_token: str | None) -> None:
+    if os.getenv("SYSTEM") == "spaces":
+        if hf_token is None:
+            raise PermissionError(
+                "Expected a HF_TOKEN to be provided when logging to a Space"
+            )
+        who = HfApi.whoami(hf_token)
+        owner_name = os.getenv("SPACE_AUTHOR_NAME")
+        repo_name = os.getenv("SPACE_REPO_NAME")
+        orgs = [o["name"] for o in who["orgs"]]
+        if owner_name != who["name"] and owner_name not in orgs:
+            raise PermissionError(
+                "Expected the provided hf_token to be the user owner of the space, or be a member of the org owner of the space"
+            )
+        access_token = who["auth"]["accessToken"]
+        if access_token["role"] == "fineGrained":
+            matched = False
+            for item in access_token["fineGrained"]["scoped"]:
+                if (
+                    item["entity"]["type"] == "space"
+                    and item["entity"]["name"] == f"{owner_name}/{repo_name}"
+                    and "repo.write" in item["permissions"]
+                ):
+                    matched = True
+                    break
+                if (
+                    (
+                        item["entity"]["type"] == "user"
+                        or item["entity"]["type"] == "org"
+                    )
+                    and item["entity"]["name"] == owner_name
+                    and "repo.write" in item["permissions"]
+                ):
+                    matched = True
+                    break
+            if not matched:
+                raise PermissionError(
+                    "Expected the provided hf_token with fine grained permissions to provide write access to the space"
+                )
+        elif access_token["role"] != "write":
+            raise PermissionError(
+                "Expected the provided hf_token to provide write permissions"
+            )
+
+
+@lru_cache(maxsize=32)
+def check_oauth_token_has_write_access(oauth_token: str | None) -> None:
+    if not os.getenv("SYSTEM") == "spaces":
+        return
+    if oauth_token is None:
+        raise PermissionError(
+            "Expected an oauth to be provided when logging to a Space"
+        )
+    who = HfApi.whoami(oauth_token)
+    user_name = who["name"]
+    owner_name = os.getenv("SPACE_AUTHOR_NAME")
+    if user_name == owner_name:
+        return
+    for org in who["orgs"]:
+        if org["name"] == owner_name and org["roleInOrg"] == "write":
+            return
+    raise PermissionError(
+        "Expected the oauth token to be the user owner of the space, or be a member of the org owner of the space"
+    )
+
+
+def check_write_access(request: gr.Request, token: str) -> bool:
+    cookies = request.headers.get("cookie", "")
+    if cookies:
+        for cookie in cookies.split(";"):
+            parts = cookie.strip().split("=", 1)
+            if len(parts) == 2 and parts[0] == "trackio_write_token":
+                return parts[1] == token
+    if hasattr(request, "query_params") and request.query_params:
+        qp = request.query_params.get("write_token")
+        return qp == token
+    return False
+
+
+def assert_can_mutate_runs(request: gr.Request) -> None:
+    if os.getenv("SYSTEM") != "spaces":
+        return
+    hf_tok = _hf_access_token(request)
+    if hf_tok is not None:
+        try:
+            check_oauth_token_has_write_access(hf_tok)
+        except PermissionError as e:
+            raise gr.Error(str(e)) from e
+        return
+    if check_write_access(request, write_token):
+        return
+    raise gr.Error(
+        "Sign in with Hugging Face to delete or rename runs. You need write access to this Space, "
+        "or open the dashboard using a link that includes the write_token query parameter."
+    )
+
+
+def get_run_mutation_status(request: gr.Request) -> dict[str, Any]:
+    if os.getenv("SYSTEM") != "spaces":
+        return {"spaces": False, "allowed": True, "auth": "local"}
+    hf_tok = _hf_access_token(request)
+    if hf_tok is not None:
+        try:
+            check_oauth_token_has_write_access(hf_tok)
+            return {"spaces": True, "allowed": True, "auth": "oauth"}
+        except PermissionError:
+            return {"spaces": True, "allowed": False, "auth": "oauth_insufficient"}
+    if check_write_access(request, write_token):
+        return {"spaces": True, "allowed": True, "auth": "write_token"}
+    return {"spaces": True, "allowed": False, "auth": "none"}
+
+
+def upload_db_to_space(
+    project: str, uploaded_db: gr.FileData, hf_token: str | None
+) -> None:
+    check_hf_token_has_write_access(hf_token)
+    db_project_path = SQLiteStorage.get_project_db_path(project)
+    os.makedirs(os.path.dirname(db_project_path), exist_ok=True)
+    shutil.copy(uploaded_db["path"], db_project_path)
+
+
+def bulk_upload_media(uploads: list[UploadEntry], hf_token: str | None) -> None:
+    check_hf_token_has_write_access(hf_token)
+    for upload in uploads:
+        media_path = get_project_media_path(
+            project=upload["project"],
+            run=upload["run"],
+            step=upload["step"],
+            relative_path=upload["relative_path"],
+        )
+        shutil.copy(upload["uploaded_file"]["path"], media_path)
+
+
+def log(
+    project: str,
+    run: str,
+    metrics: dict[str, Any],
+    step: int | None,
+    hf_token: str | None,
+) -> None:
+    check_hf_token_has_write_access(hf_token)
+    SQLiteStorage.log(project=project, run=run, metrics=metrics, step=step)
+
+
+def bulk_log(
+    logs: list[LogEntry],
+    hf_token: str | None,
+) -> None:
+    check_hf_token_has_write_access(hf_token)
+
+    logs_by_run = {}
+    for log_entry in logs:
+        key = (log_entry["project"], log_entry["run"])
+        if key not in logs_by_run:
+            logs_by_run[key] = {
+                "metrics": [],
+                "steps": [],
+                "log_ids": [],
+                "config": None,
+            }
+        logs_by_run[key]["metrics"].append(log_entry["metrics"])
+        logs_by_run[key]["steps"].append(log_entry.get("step"))
+        logs_by_run[key]["log_ids"].append(log_entry.get("log_id"))
+        if log_entry.get("config") and logs_by_run[key]["config"] is None:
+            logs_by_run[key]["config"] = log_entry["config"]
+
+    for (project, run), data in logs_by_run.items():
+        has_log_ids = any(lid is not None for lid in data["log_ids"])
+        SQLiteStorage.bulk_log(
+            project=project,
+            run=run,
+            metrics_list=data["metrics"],
+            steps=data["steps"],
+            config=data["config"],
+            log_ids=data["log_ids"] if has_log_ids else None,
+        )
+
+
+def bulk_log_system(
+    logs: list[SystemLogEntry],
+    hf_token: str | None,
+) -> None:
+    check_hf_token_has_write_access(hf_token)
+
+    logs_by_run = {}
+    for log_entry in logs:
+        key = (log_entry["project"], log_entry["run"])
+        if key not in logs_by_run:
+            logs_by_run[key] = {"metrics": [], "timestamps": [], "log_ids": []}
+        logs_by_run[key]["metrics"].append(log_entry["metrics"])
+        logs_by_run[key]["timestamps"].append(log_entry.get("timestamp"))
+        logs_by_run[key]["log_ids"].append(log_entry.get("log_id"))
+
+    for (project, run), data in logs_by_run.items():
+        has_log_ids = any(lid is not None for lid in data["log_ids"])
+        SQLiteStorage.bulk_log_system(
+            project=project,
+            run=run,
+            metrics_list=data["metrics"],
+            timestamps=data["timestamps"],
+            log_ids=data["log_ids"] if has_log_ids else None,
+        )
+
+
+def bulk_alert(
+    alerts: list[AlertEntry],
+    hf_token: str | None,
+) -> None:
+    check_hf_token_has_write_access(hf_token)
+
+    alerts_by_run: dict[tuple, dict] = {}
+    for entry in alerts:
+        key = (entry["project"], entry["run"])
+        if key not in alerts_by_run:
+            alerts_by_run[key] = {
+                "titles": [],
+                "texts": [],
+                "levels": [],
+                "steps": [],
+                "timestamps": [],
+                "alert_ids": [],
+            }
+        alerts_by_run[key]["titles"].append(entry["title"])
+        alerts_by_run[key]["texts"].append(entry.get("text"))
+        alerts_by_run[key]["levels"].append(entry["level"])
+        alerts_by_run[key]["steps"].append(entry.get("step"))
+        alerts_by_run[key]["timestamps"].append(entry.get("timestamp"))
+        alerts_by_run[key]["alert_ids"].append(entry.get("alert_id"))
+
+    for (project, run), data in alerts_by_run.items():
+        has_alert_ids = any(aid is not None for aid in data["alert_ids"])
+        SQLiteStorage.bulk_alert(
+            project=project,
+            run=run,
+            titles=data["titles"],
+            texts=data["texts"],
+            levels=data["levels"],
+            steps=data["steps"],
+            timestamps=data["timestamps"],
+            alert_ids=data["alert_ids"] if has_alert_ids else None,
+        )
+
+
+def get_alerts(
+    project: str,
+    run: str | None = None,
+    level: str | None = None,
+    since: str | None = None,
+) -> list[dict]:
+    return SQLiteStorage.get_alerts(project, run_name=run, level=level, since=since)
+
+
+def get_metric_values(
+    project: str,
+    run: str,
+    metric_name: str,
+    step: int | None = None,
+    around_step: int | None = None,
+    at_time: str | None = None,
+    window: int | None = None,
+) -> list[dict]:
+    return SQLiteStorage.get_metric_values(
+        project,
+        run,
+        metric_name,
+        step=step,
+        around_step=around_step,
+        at_time=at_time,
+        window=window,
+    )
+
+
+def get_runs_for_project(project: str) -> list[str]:
+    return SQLiteStorage.get_runs(project)
+
+
+def get_metrics_for_run(project: str, run: str) -> list[str]:
+    return SQLiteStorage.get_all_metrics_for_run(project, run)
+
+
+def filter_metrics_by_regex(metrics: list[str], filter_pattern: str) -> list[str]:
+    if not filter_pattern.strip():
+        return metrics
+    try:
+        pattern = re.compile(filter_pattern, re.IGNORECASE)
+        return [metric for metric in metrics if pattern.search(metric)]
+    except re.error:
+        return [
+            metric for metric in metrics if filter_pattern.lower() in metric.lower()
+        ]
+
+
+def get_all_projects() -> list[str]:
+    return SQLiteStorage.get_projects()
+
+
+def get_project_summary(project: str) -> dict:
+    runs = SQLiteStorage.get_runs(project)
+    if not runs:
+        return {"project": project, "num_runs": 0, "runs": [], "last_activity": None}
+
+    last_steps = SQLiteStorage.get_max_steps_for_runs(project)
+
+    return {
+        "project": project,
+        "num_runs": len(runs),
+        "runs": runs,
+        "last_activity": max(last_steps.values()) if last_steps else None,
+    }
+
+
+def get_run_summary(project: str, run: str) -> dict:
+    num_logs = SQLiteStorage.get_log_count(project, run)
+    if num_logs == 0:
+        return {
+            "project": project,
+            "run": run,
+            "num_logs": 0,
+            "metrics": [],
+            "config": None,
+            "last_step": None,
+        }
+
+    metrics = SQLiteStorage.get_all_metrics_for_run(project, run)
+    config = SQLiteStorage.get_run_config(project, run)
+    last_step = SQLiteStorage.get_last_step(project, run)
+
+    return {
+        "project": project,
+        "run": run,
+        "num_logs": num_logs,
+        "metrics": metrics,
+        "config": config,
+        "last_step": last_step,
+    }
+
+
+def get_system_metrics_for_run(project: str, run: str) -> list[str]:
+    return SQLiteStorage.get_all_system_metrics_for_run(project, run)
+
+
+def get_system_logs(project: str, run: str) -> list[dict]:
+    return SQLiteStorage.get_system_logs(project, run)
+
+
+def get_snapshot(
+    project: str,
+    run: str,
+    step: int | None = None,
+    around_step: int | None = None,
+    at_time: str | None = None,
+    window: int | None = None,
+) -> dict:
+    return SQLiteStorage.get_snapshot(
+        project, run, step=step, around_step=around_step, at_time=at_time, window=window
+    )
+
+
+def get_logs(project: str, run: str) -> list[dict]:
+    return SQLiteStorage.get_logs(project, run, max_points=1500)
+
+
+def get_settings() -> dict:
+    return {
+        "logo_urls": utils.get_logo_urls(),
+        "color_palette": utils.get_color_palette(),
+        "plot_order": [
+            item.strip()
+            for item in os.environ.get("TRACKIO_PLOT_ORDER", "").split(",")
+            if item.strip()
+        ],
+        "table_truncate_length": int(
+            os.environ.get("TRACKIO_TABLE_TRUNCATE_LENGTH", "250")
+        ),
+    }
+
+
+def get_project_files(project: str) -> list[dict]:
+    files_dir = utils.MEDIA_DIR / project / "files"
+    if not files_dir.exists():
+        return []
+    results = []
+    for file_path in sorted(files_dir.rglob("*")):
+        if file_path.is_file():
+            relative = file_path.relative_to(files_dir)
+            results.append(
+                {
+                    "name": str(relative),
+                    "path": str(file_path),
+                }
+            )
+    return results
+
+
+def delete_run(request: gr.Request, project: str, run: str) -> bool:
+    assert_can_mutate_runs(request)
+    return SQLiteStorage.delete_run(project, run)
+
+
+def rename_run(
+    request: gr.Request,
+    project: str,
+    old_name: str,
+    new_name: str,
+) -> bool:
+    assert_can_mutate_runs(request)
+    SQLiteStorage.rename_run(project, old_name, new_name)
+    return True
+
+
+def force_sync() -> bool:
+    SQLiteStorage._dataset_import_attempted = True
+    SQLiteStorage.export_to_parquet()
+    scheduler = SQLiteStorage.get_scheduler()
+    scheduler.trigger().result()
+    return True
+
+
+CSS = ""
+HEAD = ""
+
+gr.set_static_paths(paths=[utils.MEDIA_DIR])
+
+
+def make_trackio_server() -> TrackioServer:
+    server = TrackioServer(title="Trackio Dashboard")
+    server.add_api_route(OAUTH_START_PATH, oauth_hf_start, methods=["GET"])
+    server.add_api_route(OAUTH_CALLBACK_PATH, oauth_hf_callback, methods=["GET"])
+    server.add_api_route("/oauth/logout", oauth_logout, methods=["GET"])
+    server.api(fn=get_run_mutation_status, name="get_run_mutation_status")
+    server.api(fn=upload_db_to_space, name="upload_db_to_space")
+    server.api(fn=bulk_upload_media, name="bulk_upload_media")
+    server.api(fn=log, name="log")
+    server.api(fn=bulk_log, name="bulk_log")
+    server.api(fn=bulk_log_system, name="bulk_log_system")
+    server.api(fn=bulk_alert, name="bulk_alert")
+    server.api(fn=get_alerts, name="get_alerts")
+    server.api(fn=get_metric_values, name="get_metric_values")
+    server.api(fn=get_runs_for_project, name="get_runs_for_project")
+    server.api(fn=get_metrics_for_run, name="get_metrics_for_run")
+    server.api(fn=get_all_projects, name="get_all_projects")
+    server.api(fn=get_project_summary, name="get_project_summary")
+    server.api(fn=get_run_summary, name="get_run_summary")
+    server.api(fn=get_system_metrics_for_run, name="get_system_metrics_for_run")
+    server.api(fn=get_system_logs, name="get_system_logs")
+    server.api(fn=get_snapshot, name="get_snapshot")
+    server.api(fn=get_logs, name="get_logs")
+    server.api(fn=get_settings, name="get_settings")
+    server.api(fn=get_project_files, name="get_project_files")
+    server.api(fn=delete_run, name="delete_run")
+    server.api(fn=rename_run, name="rename_run")
+    server.api(fn=force_sync, name="force_sync")
+    server.write_token = write_token
+    return server

trackio/sqlite_storage.py

@@ -0,0 +1,1821 @@
+import os
+import shutil
+import sqlite3
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from threading import Lock
+
+try:
+    import fcntl
+except ImportError:
+    fcntl = None
+
+try:
+    import msvcrt as _msvcrt
+except ImportError:
+    _msvcrt = None
+
+import huggingface_hub as hf
+import orjson
+import pandas as pd
+
+from trackio.commit_scheduler import CommitScheduler
+from trackio.dummy_commit_scheduler import DummyCommitScheduler
+from trackio.utils import (
+    MEDIA_DIR,
+    TRACKIO_DIR,
+    deserialize_values,
+    serialize_values,
+)
+
+DB_EXT = ".db"
+
+
+class ProcessLock:
+    """A file-based lock that works across processes using fcntl (Unix) or msvcrt (Windows)."""
+
+    def __init__(self, lockfile_path: Path):
+        self.lockfile_path = lockfile_path
+        self.lockfile = None
+
+    def __enter__(self):
+        if fcntl is None and _msvcrt is None:
+            return self
+        self.lockfile_path.parent.mkdir(parents=True, exist_ok=True)
+        self.lockfile = open(self.lockfile_path, "w")
+
+        max_retries = 100
+        for attempt in range(max_retries):
+            try:
+                if fcntl is not None:
+                    fcntl.flock(self.lockfile.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
+                else:
+                    _msvcrt.locking(self.lockfile.fileno(), _msvcrt.LK_NBLCK, 1)
+                return self
+            except (IOError, OSError):
+                if attempt < max_retries - 1:
+                    time.sleep(0.1)
+                else:
+                    raise IOError("Could not acquire database lock after 10 seconds")
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        if self.lockfile:
+            try:
+                if fcntl is not None:
+                    fcntl.flock(self.lockfile.fileno(), fcntl.LOCK_UN)
+                elif _msvcrt is not None:
+                    _msvcrt.locking(self.lockfile.fileno(), _msvcrt.LK_UNLCK, 1)
+            except (IOError, OSError):
+                pass
+            self.lockfile.close()
+
+
+class SQLiteStorage:
+    _dataset_import_attempted = False
+    _current_scheduler: CommitScheduler | DummyCommitScheduler | None = None
+    _scheduler_lock = Lock()
+
+    @staticmethod
+    def _get_connection(db_path: Path) -> sqlite3.Connection:
+        conn = sqlite3.connect(str(db_path), timeout=30.0)
+        # Keep WAL for concurrency + performance on many small writes
+        conn.execute("PRAGMA journal_mode = WAL")
+        # ---- Minimal perf tweaks for many tiny transactions ----
+        # NORMAL = fsync at critical points only (safer than OFF, much faster than FULL)
+        conn.execute("PRAGMA synchronous = NORMAL")
+        # Keep temp data in memory to avoid disk hits during small writes
+        conn.execute("PRAGMA temp_store = MEMORY")
+        # Give SQLite a bit more room for cache (negative = KB, engine-managed)
+        conn.execute("PRAGMA cache_size = -20000")
+        # --------------------------------------------------------
+        conn.row_factory = sqlite3.Row
+        return conn
+
+    @staticmethod
+    def _get_process_lock(project: str) -> ProcessLock:
+        lockfile_path = TRACKIO_DIR / f"{project}.lock"
+        return ProcessLock(lockfile_path)
+
+    @staticmethod
+    def get_project_db_filename(project: str) -> str:
+        """Get the database filename for a specific project."""
+        safe_project_name = "".join(
+            c for c in project if c.isalnum() or c in ("-", "_")
+        ).rstrip()
+        if not safe_project_name:
+            safe_project_name = "default"
+        return f"{safe_project_name}{DB_EXT}"
+
+    @staticmethod
+    def get_project_db_path(project: str) -> Path:
+        """Get the database path for a specific project."""
+        filename = SQLiteStorage.get_project_db_filename(project)
+        return TRACKIO_DIR / filename
+
+    @staticmethod
+    def init_db(project: str) -> Path:
+        """
+        Initialize the SQLite database with required tables.
+        Returns the database path.
+        """
+        db_path = SQLiteStorage.get_project_db_path(project)
+        db_path.parent.mkdir(parents=True, exist_ok=True)
+        with SQLiteStorage._get_process_lock(project):
+            with sqlite3.connect(str(db_path), timeout=30.0) as conn:
+                conn.execute("PRAGMA journal_mode = WAL")
+                conn.execute("PRAGMA synchronous = NORMAL")
+                conn.execute("PRAGMA temp_store = MEMORY")
+                conn.execute("PRAGMA cache_size = -20000")
+                cursor = conn.cursor()
+                cursor.execute(
+                    """
+                    CREATE TABLE IF NOT EXISTS metrics (
+                        id INTEGER PRIMARY KEY AUTOINCREMENT,
+                        timestamp TEXT NOT NULL,
+                        run_name TEXT NOT NULL,
+                        step INTEGER NOT NULL,
+                        metrics TEXT NOT NULL
+                    )
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE TABLE IF NOT EXISTS configs (
+                        id INTEGER PRIMARY KEY AUTOINCREMENT,
+                        run_name TEXT NOT NULL,
+                        config TEXT NOT NULL,
+                        created_at TEXT NOT NULL,
+                        UNIQUE(run_name)
+                    )
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE INDEX IF NOT EXISTS idx_metrics_run_step
+                    ON metrics(run_name, step)
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE INDEX IF NOT EXISTS idx_configs_run_name
+                    ON configs(run_name)
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE INDEX IF NOT EXISTS idx_metrics_run_timestamp
+                    ON metrics(run_name, timestamp)
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE TABLE IF NOT EXISTS system_metrics (
+                        id INTEGER PRIMARY KEY AUTOINCREMENT,
+                        timestamp TEXT NOT NULL,
+                        run_name TEXT NOT NULL,
+                        metrics TEXT NOT NULL
+                    )
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE INDEX IF NOT EXISTS idx_system_metrics_run_timestamp
+                    ON system_metrics(run_name, timestamp)
+                    """
+                )
+
+                cursor.execute(
+                    """
+                    CREATE TABLE IF NOT EXISTS project_metadata (
+                        key TEXT PRIMARY KEY,
+                        value TEXT NOT NULL
+                    )
+                    """
+                )
+
+                cursor.execute(
+                    """
+                    CREATE TABLE IF NOT EXISTS pending_uploads (
+                        id INTEGER PRIMARY KEY AUTOINCREMENT,
+                        space_id TEXT NOT NULL,
+                        run_name TEXT,
+                        step INTEGER,
+                        file_path TEXT NOT NULL,
+                        relative_path TEXT,
+                        created_at TEXT NOT NULL
+                    )
+                    """
+                )
+
+                cursor.execute(
+                    """
+                    CREATE TABLE IF NOT EXISTS alerts (
+                        id INTEGER PRIMARY KEY AUTOINCREMENT,
+                        timestamp TEXT NOT NULL,
+                        run_name TEXT NOT NULL,
+                        title TEXT NOT NULL,
+                        text TEXT,
+                        level TEXT NOT NULL DEFAULT 'warn',
+                        step INTEGER,
+                        alert_id TEXT
+                    )
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE INDEX IF NOT EXISTS idx_alerts_run
+                    ON alerts(run_name)
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE INDEX IF NOT EXISTS idx_alerts_timestamp
+                    ON alerts(timestamp)
+                    """
+                )
+                cursor.execute(
+                    """
+                    CREATE UNIQUE INDEX IF NOT EXISTS idx_alerts_alert_id
+                    ON alerts(alert_id) WHERE alert_id IS NOT NULL
+                    """
+                )
+
+                for table in ("metrics", "system_metrics"):
+                    for col in ("log_id TEXT", "space_id TEXT"):
+                        try:
+                            cursor.execute(f"ALTER TABLE {table} ADD COLUMN {col}")
+                        except sqlite3.OperationalError:
+                            pass
+                    cursor.execute(
+                        f"""CREATE UNIQUE INDEX IF NOT EXISTS idx_{table}_log_id
+                        ON {table}(log_id) WHERE log_id IS NOT NULL"""
+                    )
+                    cursor.execute(
+                        f"""CREATE INDEX IF NOT EXISTS idx_{table}_pending
+                        ON {table}(space_id) WHERE space_id IS NOT NULL"""
+                    )
+
+                conn.commit()
+        return db_path
+
+    @staticmethod
+    def export_to_parquet():
+        """
+        Exports all projects' DB files as Parquet under the same path but with extension ".parquet".
+        Also exports system_metrics to separate parquet files with "_system.parquet" suffix.
+        Also exports configs to separate parquet files with "_configs.parquet" suffix.
+        """
+        if not SQLiteStorage._dataset_import_attempted:
+            return
+        if not TRACKIO_DIR.exists():
+            return
+
+        all_paths = os.listdir(TRACKIO_DIR)
+        db_names = [f for f in all_paths if f.endswith(DB_EXT)]
+        for db_name in db_names:
+            db_path = TRACKIO_DIR / db_name
+            parquet_path = db_path.with_suffix(".parquet")
+            system_parquet_path = db_path.with_suffix("") / ""
+            system_parquet_path = TRACKIO_DIR / (db_path.stem + "_system.parquet")
+            configs_parquet_path = TRACKIO_DIR / (db_path.stem + "_configs.parquet")
+            if (not parquet_path.exists()) or (
+                db_path.stat().st_mtime > parquet_path.stat().st_mtime
+            ):
+                with sqlite3.connect(str(db_path)) as conn:
+                    df = pd.read_sql("SELECT * FROM metrics", conn)
+                if not df.empty:
+                    metrics = df["metrics"].copy()
+                    metrics = pd.DataFrame(
+                        metrics.apply(
+                            lambda x: deserialize_values(orjson.loads(x))
+                        ).values.tolist(),
+                        index=df.index,
+                    )
+                    del df["metrics"]
+                    for col in metrics.columns:
+                        df[col] = metrics[col]
+                    df.to_parquet(
+                        parquet_path,
+                        write_page_index=True,
+                        use_content_defined_chunking=True,
+                    )
+
+            if (not system_parquet_path.exists()) or (
+                db_path.stat().st_mtime > system_parquet_path.stat().st_mtime
+            ):
+                with sqlite3.connect(str(db_path)) as conn:
+                    try:
+                        sys_df = pd.read_sql("SELECT * FROM system_metrics", conn)
+                    except Exception:
+                        sys_df = pd.DataFrame()
+                if not sys_df.empty:
+                    sys_metrics = sys_df["metrics"].copy()
+                    sys_metrics = pd.DataFrame(
+                        sys_metrics.apply(
+                            lambda x: deserialize_values(orjson.loads(x))
+                        ).values.tolist(),
+                        index=sys_df.index,
+                    )
+                    del sys_df["metrics"]
+                    for col in sys_metrics.columns:
+                        sys_df[col] = sys_metrics[col]
+                    sys_df.to_parquet(
+                        system_parquet_path,
+                        write_page_index=True,
+                        use_content_defined_chunking=True,
+                    )
+
+            if (not configs_parquet_path.exists()) or (
+                db_path.stat().st_mtime > configs_parquet_path.stat().st_mtime
+            ):
+                with sqlite3.connect(str(db_path)) as conn:
+                    try:
+                        configs_df = pd.read_sql("SELECT * FROM configs", conn)
+                    except Exception:
+                        configs_df = pd.DataFrame()
+                if not configs_df.empty:
+                    config_data = configs_df["config"].copy()
+                    config_data = pd.DataFrame(
+                        config_data.apply(
+                            lambda x: deserialize_values(orjson.loads(x))
+                        ).values.tolist(),
+                        index=configs_df.index,
+                    )
+                    del configs_df["config"]
+                    for col in config_data.columns:
+                        configs_df[col] = config_data[col]
+                    configs_df.to_parquet(
+                        configs_parquet_path,
+                        write_page_index=True,
+                        use_content_defined_chunking=True,
+                    )
+
+    @staticmethod
+    def _cleanup_wal_sidecars(db_path: Path) -> None:
+        """Remove leftover -wal/-shm files for a DB basename (prevents disk I/O errors)."""
+        for suffix in ("-wal", "-shm"):
+            sidecar = Path(str(db_path) + suffix)
+            try:
+                if sidecar.exists():
+                    sidecar.unlink()
+            except Exception:
+                pass
+
+    @staticmethod
+    def import_from_parquet():
+        """
+        Imports to all DB files that have matching files under the same path but with extension ".parquet".
+        Also imports system_metrics from "_system.parquet" files.
+        Also imports configs from "_configs.parquet" files.
+        """
+        if not TRACKIO_DIR.exists():
+            return
+
+        all_paths = os.listdir(TRACKIO_DIR)
+        parquet_names = [
+            f
+            for f in all_paths
+            if f.endswith(".parquet")
+            and not f.endswith("_system.parquet")
+            and not f.endswith("_configs.parquet")
+        ]
+        imported_projects = {Path(name).stem for name in parquet_names}
+        for pq_name in parquet_names:
+            parquet_path = TRACKIO_DIR / pq_name
+            db_path = parquet_path.with_suffix(DB_EXT)
+
+            SQLiteStorage._cleanup_wal_sidecars(db_path)
+
+            df = pd.read_parquet(parquet_path)
+            if "metrics" not in df.columns:
+                metrics = df.copy()
+                structural_cols = [
+                    "id",
+                    "timestamp",
+                    "run_name",
+                    "step",
+                    "log_id",
+                    "space_id",
+                ]
+                df = df[[c for c in structural_cols if c in df.columns]]
+                for col in structural_cols:
+                    if col in metrics.columns:
+                        del metrics[col]
+                metrics = orjson.loads(metrics.to_json(orient="records"))
+                df["metrics"] = [orjson.dumps(serialize_values(row)) for row in metrics]
+
+            with sqlite3.connect(str(db_path), timeout=30.0) as conn:
+                df.to_sql("metrics", conn, if_exists="replace", index=False)
+                conn.commit()
+
+        system_parquet_names = [f for f in all_paths if f.endswith("_system.parquet")]
+        for pq_name in system_parquet_names:
+            parquet_path = TRACKIO_DIR / pq_name
+            db_name = pq_name.replace("_system.parquet", DB_EXT)
+            db_path = TRACKIO_DIR / db_name
+            project_name = db_path.stem
+            if project_name not in imported_projects and not db_path.exists():
+                continue
+
+            df = pd.read_parquet(parquet_path)
+            if "metrics" not in df.columns:
+                metrics = df.copy()
+                other_cols = ["id", "timestamp", "run_name"]
+                df = df[[c for c in other_cols if c in df.columns]]
+                for col in other_cols:
+                    if col in metrics.columns:
+                        del metrics[col]
+                metrics = orjson.loads(metrics.to_json(orient="records"))
+                df["metrics"] = [orjson.dumps(serialize_values(row)) for row in metrics]
+
+            with sqlite3.connect(str(db_path), timeout=30.0) as conn:
+                df.to_sql("system_metrics", conn, if_exists="replace", index=False)
+                conn.commit()
+
+        configs_parquet_names = [f for f in all_paths if f.endswith("_configs.parquet")]
+        for pq_name in configs_parquet_names:
+            parquet_path = TRACKIO_DIR / pq_name
+            db_name = pq_name.replace("_configs.parquet", DB_EXT)
+            db_path = TRACKIO_DIR / db_name
+            project_name = db_path.stem
+            if project_name not in imported_projects and not db_path.exists():
+                continue
+
+            df = pd.read_parquet(parquet_path)
+            if "config" not in df.columns:
+                config_data = df.copy()
+                other_cols = ["id", "run_name", "created_at"]
+                df = df[[c for c in other_cols if c in df.columns]]
+                for col in other_cols:
+                    if col in config_data.columns:
+                        del config_data[col]
+                config_data = orjson.loads(config_data.to_json(orient="records"))
+                df["config"] = [
+                    orjson.dumps(serialize_values(row)) for row in config_data
+                ]
+
+            with sqlite3.connect(str(db_path), timeout=30.0) as conn:
+                df.to_sql("configs", conn, if_exists="replace", index=False)
+                conn.commit()
+
+    @staticmethod
+    def get_scheduler():
+        """
+        Get the scheduler for the database based on the environment variables.
+        This applies to both local and Spaces.
+        """
+        with SQLiteStorage._scheduler_lock:
+            if SQLiteStorage._current_scheduler is not None:
+                return SQLiteStorage._current_scheduler
+            hf_token = os.environ.get("HF_TOKEN")
+            dataset_id = os.environ.get("TRACKIO_DATASET_ID")
+            space_repo_name = os.environ.get("SPACE_REPO_NAME")
+            if dataset_id is None or space_repo_name is None:
+                scheduler = DummyCommitScheduler()
+            else:
+                scheduler = CommitScheduler(
+                    repo_id=dataset_id,
+                    repo_type="dataset",
+                    folder_path=TRACKIO_DIR,
+                    private=True,
+                    allow_patterns=[
+                        "*.parquet",
+                        "*_system.parquet",
+                        "*_configs.parquet",
+                        "media/**/*",
+                    ],
+                    squash_history=True,
+                    token=hf_token,
+                    on_before_commit=SQLiteStorage.export_to_parquet,
+                )
+            SQLiteStorage._current_scheduler = scheduler
+            return scheduler
+
+    @staticmethod
+    def log(project: str, run: str, metrics: dict, step: int | None = None):
+        """
+        Safely log metrics to the database. Before logging, this method will ensure the database exists
+        and is set up with the correct tables. It also uses a cross-process lock to prevent
+        database locking errors when multiple processes access the same database.
+
+        This method is not used in the latest versions of Trackio (replaced by bulk_log) but
+        is kept for backwards compatibility for users who are connecting to a newer version of
+        a Trackio Spaces dashboard with an older version of Trackio installed locally.
+        """
+        db_path = SQLiteStorage.init_db(project)
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                cursor.execute(
+                    """
+                    SELECT MAX(step) 
+                    FROM metrics 
+                    WHERE run_name = ?
+                    """,
+                    (run,),
+                )
+                last_step = cursor.fetchone()[0]
+                current_step = (
+                    0
+                    if step is None and last_step is None
+                    else (step if step is not None else last_step + 1)
+                )
+                current_timestamp = datetime.now(timezone.utc).isoformat()
+                cursor.execute(
+                    """
+                    INSERT INTO metrics
+                    (timestamp, run_name, step, metrics)
+                    VALUES (?, ?, ?, ?)
+                    """,
+                    (
+                        current_timestamp,
+                        run,
+                        current_step,
+                        orjson.dumps(serialize_values(metrics)),
+                    ),
+                )
+                conn.commit()
+
+    @staticmethod
+    def bulk_log(
+        project: str,
+        run: str,
+        metrics_list: list[dict],
+        steps: list[int] | None = None,
+        timestamps: list[str] | None = None,
+        config: dict | None = None,
+        log_ids: list[str] | None = None,
+        space_id: str | None = None,
+    ):
+        """
+        Safely log bulk metrics to the database. Before logging, this method will ensure the database exists
+        and is set up with the correct tables. It also uses a cross-process lock to prevent
+        database locking errors when multiple processes access the same database.
+        """
+        if not metrics_list:
+            return
+
+        if timestamps is None:
+            timestamps = [datetime.now(timezone.utc).isoformat()] * len(metrics_list)
+
+        db_path = SQLiteStorage.init_db(project)
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+
+                if steps is None:
+                    steps = list(range(len(metrics_list)))
+                elif any(s is None for s in steps):
+                    cursor.execute(
+                        "SELECT MAX(step) FROM metrics WHERE run_name = ?", (run,)
+                    )
+                    last_step = cursor.fetchone()[0]
+                    current_step = 0 if last_step is None else last_step + 1
+                    processed_steps = []
+                    for step in steps:
+                        if step is None:
+                            processed_steps.append(current_step)
+                            current_step += 1
+                        else:
+                            processed_steps.append(step)
+                    steps = processed_steps
+
+                if len(metrics_list) != len(steps) or len(metrics_list) != len(
+                    timestamps
+                ):
+                    raise ValueError(
+                        "metrics_list, steps, and timestamps must have the same length"
+                    )
+
+                data = []
+                for i, metrics in enumerate(metrics_list):
+                    lid = log_ids[i] if log_ids else None
+                    data.append(
+                        (
+                            timestamps[i],
+                            run,
+                            steps[i],
+                            orjson.dumps(serialize_values(metrics)),
+                            lid,
+                            space_id,
+                        )
+                    )
+
+                cursor.executemany(
+                    """
+                    INSERT OR IGNORE INTO metrics
+                    (timestamp, run_name, step, metrics, log_id, space_id)
+                    VALUES (?, ?, ?, ?, ?, ?)
+                    """,
+                    data,
+                )
+
+                if config:
+                    current_timestamp = datetime.now(timezone.utc).isoformat()
+                    cursor.execute(
+                        """
+                        INSERT OR REPLACE INTO configs
+                        (run_name, config, created_at)
+                        VALUES (?, ?, ?)
+                        """,
+                        (
+                            run,
+                            orjson.dumps(serialize_values(config)),
+                            current_timestamp,
+                        ),
+                    )
+
+                conn.commit()
+
+    @staticmethod
+    def bulk_log_system(
+        project: str,
+        run: str,
+        metrics_list: list[dict],
+        timestamps: list[str] | None = None,
+        log_ids: list[str] | None = None,
+        space_id: str | None = None,
+    ):
+        """
+        Log system metrics (GPU, etc.) to the database without step numbers.
+        These metrics use timestamps for the x-axis instead of steps.
+        """
+        if not metrics_list:
+            return
+
+        if timestamps is None:
+            timestamps = [datetime.now(timezone.utc).isoformat()] * len(metrics_list)
+
+        if len(metrics_list) != len(timestamps):
+            raise ValueError("metrics_list and timestamps must have the same length")
+
+        db_path = SQLiteStorage.init_db(project)
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                data = []
+                for i, metrics in enumerate(metrics_list):
+                    lid = log_ids[i] if log_ids else None
+                    data.append(
+                        (
+                            timestamps[i],
+                            run,
+                            orjson.dumps(serialize_values(metrics)),
+                            lid,
+                            space_id,
+                        )
+                    )
+
+                cursor.executemany(
+                    """
+                    INSERT OR IGNORE INTO system_metrics
+                    (timestamp, run_name, metrics, log_id, space_id)
+                    VALUES (?, ?, ?, ?, ?)
+                    """,
+                    data,
+                )
+                conn.commit()
+
+    @staticmethod
+    def bulk_alert(
+        project: str,
+        run: str,
+        titles: list[str],
+        texts: list[str | None],
+        levels: list[str],
+        steps: list[int | None],
+        timestamps: list[str] | None = None,
+        alert_ids: list[str] | None = None,
+    ):
+        if not titles:
+            return
+
+        if timestamps is None:
+            timestamps = [datetime.now(timezone.utc).isoformat()] * len(titles)
+
+        db_path = SQLiteStorage.init_db(project)
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                data = []
+                for i in range(len(titles)):
+                    aid = alert_ids[i] if alert_ids else None
+                    data.append(
+                        (
+                            timestamps[i],
+                            run,
+                            titles[i],
+                            texts[i],
+                            levels[i],
+                            steps[i],
+                            aid,
+                        )
+                    )
+
+                cursor.executemany(
+                    """
+                    INSERT OR IGNORE INTO alerts
+                    (timestamp, run_name, title, text, level, step, alert_id)
+                    VALUES (?, ?, ?, ?, ?, ?, ?)
+                    """,
+                    data,
+                )
+                conn.commit()
+
+    @staticmethod
+    def get_alerts(
+        project: str,
+        run_name: str | None = None,
+        level: str | None = None,
+        since: str | None = None,
+    ) -> list[dict]:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return []
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                query = (
+                    "SELECT timestamp, run_name, title, text, level, step FROM alerts"
+                )
+                conditions = []
+                params = []
+                if run_name is not None:
+                    conditions.append("run_name = ?")
+                    params.append(run_name)
+                if level is not None:
+                    conditions.append("level = ?")
+                    params.append(level)
+                if since is not None:
+                    conditions.append("timestamp > ?")
+                    params.append(since)
+                if conditions:
+                    query += " WHERE " + " AND ".join(conditions)
+                query += " ORDER BY timestamp DESC"
+                cursor.execute(query, params)
+
+                rows = cursor.fetchall()
+                return [
+                    {
+                        "timestamp": row["timestamp"],
+                        "run": row["run_name"],
+                        "title": row["title"],
+                        "text": row["text"],
+                        "level": row["level"],
+                        "step": row["step"],
+                    }
+                    for row in rows
+                ]
+            except sqlite3.OperationalError as e:
+                if "no such table: alerts" in str(e):
+                    return []
+                raise
+
+    @staticmethod
+    def get_alert_count(project: str) -> int:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return 0
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute("SELECT COUNT(*) FROM alerts")
+                return cursor.fetchone()[0]
+            except sqlite3.OperationalError:
+                return 0
+
+    @staticmethod
+    def get_system_logs(project: str, run: str) -> list[dict]:
+        """Retrieve system metrics for a specific run. Returns metrics with timestamps (no steps)."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return []
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    """
+                    SELECT timestamp, metrics
+                    FROM system_metrics
+                    WHERE run_name = ?
+                    ORDER BY timestamp
+                    """,
+                    (run,),
+                )
+
+                rows = cursor.fetchall()
+                results = []
+                for row in rows:
+                    metrics = orjson.loads(row["metrics"])
+                    metrics = deserialize_values(metrics)
+                    metrics["timestamp"] = row["timestamp"]
+                    results.append(metrics)
+                return results
+            except sqlite3.OperationalError as e:
+                if "no such table: system_metrics" in str(e):
+                    return []
+                raise
+
+    @staticmethod
+    def get_all_system_metrics_for_run(project: str, run: str) -> list[str]:
+        """Get all system metric names for a specific project/run."""
+        return SQLiteStorage._get_metric_names(
+            project, run, "system_metrics", exclude_keys={"timestamp"}
+        )
+
+    @staticmethod
+    def has_system_metrics(project: str) -> bool:
+        """Check if a project has any system metrics logged."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return False
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute("SELECT COUNT(*) FROM system_metrics LIMIT 1")
+                count = cursor.fetchone()[0]
+                return count > 0
+            except sqlite3.OperationalError:
+                return False
+
+    @staticmethod
+    def get_log_count(project: str, run: str) -> int:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return 0
+        try:
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                cursor.execute(
+                    "SELECT COUNT(*) FROM metrics WHERE run_name = ?",
+                    (run,),
+                )
+                return cursor.fetchone()[0]
+        except sqlite3.OperationalError as e:
+            if "no such table: metrics" in str(e):
+                return 0
+            raise
+
+    @staticmethod
+    def get_last_step(project: str, run: str) -> int | None:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return None
+        try:
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                cursor.execute(
+                    "SELECT MAX(step) FROM metrics WHERE run_name = ?",
+                    (run,),
+                )
+                row = cursor.fetchone()
+                return row[0] if row and row[0] is not None else None
+        except sqlite3.OperationalError as e:
+            if "no such table: metrics" in str(e):
+                return None
+            raise
+
+    @staticmethod
+    def get_logs(project: str, run: str, max_points: int | None = None) -> list[dict]:
+        """Retrieve logs for a specific run. Logs include the step count (int) and the timestamp (datetime object)."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return []
+
+        try:
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                cursor.execute(
+                    """
+                    SELECT timestamp, step, metrics
+                    FROM metrics
+                    WHERE run_name = ?
+                    ORDER BY timestamp
+                    """,
+                    (run,),
+                )
+
+                rows = cursor.fetchall()
+                if max_points is not None and len(rows) > max_points:
+                    step = len(rows) / max_points
+                    indices = {int(i * step) for i in range(max_points)}
+                    indices.add(len(rows) - 1)
+                    rows = [rows[i] for i in sorted(indices)]
+
+                results = []
+                for row in rows:
+                    metrics = orjson.loads(row["metrics"])
+                    metrics = deserialize_values(metrics)
+                    metrics["timestamp"] = row["timestamp"]
+                    metrics["step"] = row["step"]
+                    results.append(metrics)
+                return results
+        except sqlite3.OperationalError as e:
+            if "no such table: metrics" in str(e):
+                return []
+            raise
+
+    @staticmethod
+    def load_from_dataset():
+        dataset_id = os.environ.get("TRACKIO_DATASET_ID")
+        space_repo_name = os.environ.get("SPACE_REPO_NAME")
+        if dataset_id is not None and space_repo_name is not None:
+            hfapi = hf.HfApi()
+            updated = False
+            if not TRACKIO_DIR.exists():
+                TRACKIO_DIR.mkdir(parents=True, exist_ok=True)
+            with SQLiteStorage.get_scheduler().lock:
+                try:
+                    files = hfapi.list_repo_files(dataset_id, repo_type="dataset")
+                    for file in files:
+                        # Download parquet and media assets
+                        if not (file.endswith(".parquet") or file.startswith("media/")):
+                            continue
+                        if (TRACKIO_DIR / file).exists():
+                            continue
+                        hf.hf_hub_download(
+                            dataset_id, file, repo_type="dataset", local_dir=TRACKIO_DIR
+                        )
+                        updated = True
+                except hf.errors.EntryNotFoundError:
+                    pass
+                except hf.errors.RepositoryNotFoundError:
+                    pass
+                if updated:
+                    SQLiteStorage.import_from_parquet()
+        SQLiteStorage._dataset_import_attempted = True
+
+    @staticmethod
+    def get_projects() -> list[str]:
+        """
+        Get list of all projects by scanning the database files in the trackio directory.
+        """
+        if not SQLiteStorage._dataset_import_attempted:
+            SQLiteStorage.load_from_dataset()
+
+        projects: set[str] = set()
+        if not TRACKIO_DIR.exists():
+            return []
+
+        for db_file in TRACKIO_DIR.glob(f"*{DB_EXT}"):
+            project_name = db_file.stem
+            projects.add(project_name)
+        return sorted(projects)
+
+    @staticmethod
+    def get_runs(project: str) -> list[str]:
+        """Get list of all runs for a project, ordered by creation time."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return []
+
+        try:
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                cursor.execute(
+                    """
+                    SELECT run_name
+                    FROM metrics
+                    GROUP BY run_name
+                    ORDER BY MIN(timestamp) ASC
+                    """,
+                )
+            return [row[0] for row in cursor.fetchall()]
+        except sqlite3.OperationalError as e:
+            if "no such table: metrics" in str(e):
+                return []
+            raise
+
+    @staticmethod
+    def get_max_steps_for_runs(project: str) -> dict[str, int]:
+        """Get the maximum step for each run in a project."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return {}
+
+        try:
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                cursor.execute(
+                    """
+                    SELECT run_name, MAX(step) as max_step
+                    FROM metrics
+                    GROUP BY run_name
+                    """
+                )
+
+                results = {}
+                for row in cursor.fetchall():
+                    results[row["run_name"]] = row["max_step"]
+
+                return results
+        except sqlite3.OperationalError as e:
+            if "no such table: metrics" in str(e):
+                return {}
+            raise
+
+    @staticmethod
+    def get_max_step_for_run(project: str, run: str) -> int | None:
+        """Get the maximum step for a specific run, or None if no logs exist."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return None
+
+        try:
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                cursor.execute(
+                    "SELECT MAX(step) FROM metrics WHERE run_name = ?", (run,)
+                )
+                result = cursor.fetchone()[0]
+                return result
+        except sqlite3.OperationalError as e:
+            if "no such table: metrics" in str(e):
+                return None
+            raise
+
+    @staticmethod
+    def get_run_config(project: str, run: str) -> dict | None:
+        """Get configuration for a specific run."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return None
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    """
+                    SELECT config FROM configs WHERE run_name = ?
+                    """,
+                    (run,),
+                )
+
+                row = cursor.fetchone()
+                if row:
+                    config = orjson.loads(row["config"])
+                    return deserialize_values(config)
+                return None
+            except sqlite3.OperationalError as e:
+                if "no such table: configs" in str(e):
+                    return None
+                raise
+
+    @staticmethod
+    def delete_run(project: str, run: str) -> bool:
+        """Delete a run from the database (metrics, config, and system_metrics)."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return False
+
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+                try:
+                    cursor.execute("DELETE FROM metrics WHERE run_name = ?", (run,))
+                    cursor.execute("DELETE FROM configs WHERE run_name = ?", (run,))
+                    try:
+                        cursor.execute(
+                            "DELETE FROM system_metrics WHERE run_name = ?", (run,)
+                        )
+                    except sqlite3.OperationalError:
+                        pass
+                    try:
+                        cursor.execute("DELETE FROM alerts WHERE run_name = ?", (run,))
+                    except sqlite3.OperationalError:
+                        pass
+                    conn.commit()
+                    return True
+                except sqlite3.Error:
+                    return False
+
+    @staticmethod
+    def _update_media_paths(obj, old_prefix, new_prefix):
+        """Update media file paths in nested data structures."""
+        if isinstance(obj, dict):
+            if obj.get("_type") in [
+                "trackio.image",
+                "trackio.video",
+                "trackio.audio",
+            ]:
+                old_path = obj.get("file_path", "")
+                if isinstance(old_path, str):
+                    normalized_path = old_path.replace("\\", "/")
+                    if normalized_path.startswith(old_prefix):
+                        new_path = normalized_path.replace(old_prefix, new_prefix, 1)
+                        return {**obj, "file_path": new_path}
+            return {
+                key: SQLiteStorage._update_media_paths(value, old_prefix, new_prefix)
+                for key, value in obj.items()
+            }
+        elif isinstance(obj, list):
+            return [
+                SQLiteStorage._update_media_paths(item, old_prefix, new_prefix)
+                for item in obj
+            ]
+        return obj
+
+    @staticmethod
+    def _rewrite_metrics_rows(metrics_rows, new_run_name, old_prefix, new_prefix):
+        """Deserialize metrics rows, update media paths, and reserialize."""
+        result = []
+        for row in metrics_rows:
+            metrics_data = orjson.loads(row["metrics"])
+            metrics_deserialized = deserialize_values(metrics_data)
+            updated = SQLiteStorage._update_media_paths(
+                metrics_deserialized, old_prefix, new_prefix
+            )
+            result.append(
+                (
+                    row["timestamp"],
+                    new_run_name,
+                    row["step"],
+                    orjson.dumps(serialize_values(updated)),
+                )
+            )
+        return result
+
+    @staticmethod
+    def _move_media_dir(source: Path, target: Path):
+        """Move a media directory from source to target."""
+        if source.exists():
+            target.parent.mkdir(parents=True, exist_ok=True)
+            if target.exists():
+                shutil.rmtree(target)
+            shutil.move(str(source), str(target))
+
+    @staticmethod
+    def rename_run(project: str, old_name: str, new_name: str) -> None:
+        """Rename a run within the same project.
+
+        Raises:
+            ValueError: If the new name is empty, the old run doesn't exist,
+                        or a run with the new name already exists.
+            RuntimeError: If the database operation fails.
+        """
+        if not new_name or not new_name.strip():
+            raise ValueError("New run name cannot be empty")
+
+        new_name = new_name.strip()
+
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            raise ValueError(f"Project '{project}' does not exist")
+
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                cursor = conn.cursor()
+
+                cursor.execute(
+                    "SELECT COUNT(*) FROM metrics WHERE run_name = ?", (old_name,)
+                )
+                if cursor.fetchone()[0] == 0:
+                    raise ValueError(
+                        f"Run '{old_name}' does not exist in project '{project}'"
+                    )
+
+                cursor.execute(
+                    "SELECT COUNT(*) FROM metrics WHERE run_name = ?", (new_name,)
+                )
+                if cursor.fetchone()[0] > 0:
+                    raise ValueError(
+                        f"A run named '{new_name}' already exists in project '{project}'"
+                    )
+
+                try:
+                    cursor.execute(
+                        "SELECT timestamp, step, metrics FROM metrics WHERE run_name = ?",
+                        (old_name,),
+                    )
+                    metrics_rows = cursor.fetchall()
+
+                    old_prefix = f"{project}/{old_name}/"
+                    new_prefix = f"{project}/{new_name}/"
+
+                    updated_rows = SQLiteStorage._rewrite_metrics_rows(
+                        metrics_rows, new_name, old_prefix, new_prefix
+                    )
+
+                    cursor.execute(
+                        "DELETE FROM metrics WHERE run_name = ?", (old_name,)
+                    )
+                    cursor.executemany(
+                        "INSERT INTO metrics (timestamp, run_name, step, metrics) VALUES (?, ?, ?, ?)",
+                        updated_rows,
+                    )
+
+                    cursor.execute(
+                        "UPDATE configs SET run_name = ? WHERE run_name = ?",
+                        (new_name, old_name),
+                    )
+
+                    try:
+                        cursor.execute(
+                            "UPDATE system_metrics SET run_name = ? WHERE run_name = ?",
+                            (new_name, old_name),
+                        )
+                    except sqlite3.OperationalError:
+                        pass
+
+                    try:
+                        cursor.execute(
+                            "UPDATE alerts SET run_name = ? WHERE run_name = ?",
+                            (new_name, old_name),
+                        )
+                    except sqlite3.OperationalError:
+                        pass
+
+                    conn.commit()
+
+                    SQLiteStorage._move_media_dir(
+                        MEDIA_DIR / project / old_name,
+                        MEDIA_DIR / project / new_name,
+                    )
+                except sqlite3.Error as e:
+                    raise RuntimeError(
+                        f"Database error while renaming run '{old_name}' to '{new_name}': {e}"
+                    ) from e
+
+    @staticmethod
+    def move_run(project: str, run: str, new_project: str) -> bool:
+        """Move a run from one project to another."""
+        source_db_path = SQLiteStorage.get_project_db_path(project)
+        if not source_db_path.exists():
+            return False
+
+        target_db_path = SQLiteStorage.init_db(new_project)
+
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_process_lock(new_project):
+                with SQLiteStorage._get_connection(source_db_path) as source_conn:
+                    source_cursor = source_conn.cursor()
+
+                    source_cursor.execute(
+                        "SELECT timestamp, step, metrics FROM metrics WHERE run_name = ?",
+                        (run,),
+                    )
+                    metrics_rows = source_cursor.fetchall()
+
+                    source_cursor.execute(
+                        "SELECT config, created_at FROM configs WHERE run_name = ?",
+                        (run,),
+                    )
+                    config_row = source_cursor.fetchone()
+
+                    try:
+                        source_cursor.execute(
+                            "SELECT timestamp, metrics FROM system_metrics WHERE run_name = ?",
+                            (run,),
+                        )
+                        system_metrics_rows = source_cursor.fetchall()
+                    except sqlite3.OperationalError:
+                        system_metrics_rows = []
+
+                    try:
+                        source_cursor.execute(
+                            "SELECT timestamp, title, text, level, step, alert_id FROM alerts WHERE run_name = ?",
+                            (run,),
+                        )
+                        alert_rows = source_cursor.fetchall()
+                    except sqlite3.OperationalError:
+                        alert_rows = []
+
+                    if not metrics_rows and not config_row and not system_metrics_rows:
+                        return False
+
+                    with SQLiteStorage._get_connection(target_db_path) as target_conn:
+                        target_cursor = target_conn.cursor()
+
+                        old_prefix = f"{project}/{run}/"
+                        new_prefix = f"{new_project}/{run}/"
+                        updated_rows = SQLiteStorage._rewrite_metrics_rows(
+                            metrics_rows, run, old_prefix, new_prefix
+                        )
+
+                        target_cursor.executemany(
+                            "INSERT INTO metrics (timestamp, run_name, step, metrics) VALUES (?, ?, ?, ?)",
+                            updated_rows,
+                        )
+
+                        if config_row:
+                            target_cursor.execute(
+                                """
+                                INSERT OR REPLACE INTO configs (run_name, config, created_at)
+                                VALUES (?, ?, ?)
+                                """,
+                                (run, config_row["config"], config_row["created_at"]),
+                            )
+
+                        for row in system_metrics_rows:
+                            try:
+                                target_cursor.execute(
+                                    """
+                                    INSERT INTO system_metrics (timestamp, run_name, metrics)
+                                    VALUES (?, ?, ?)
+                                    """,
+                                    (row["timestamp"], run, row["metrics"]),
+                                )
+                            except sqlite3.OperationalError:
+                                pass
+
+                        for row in alert_rows:
+                            try:
+                                target_cursor.execute(
+                                    """
+                                    INSERT OR IGNORE INTO alerts (timestamp, run_name, title, text, level, step, alert_id)
+                                    VALUES (?, ?, ?, ?, ?, ?, ?)
+                                    """,
+                                    (
+                                        row["timestamp"],
+                                        run,
+                                        row["title"],
+                                        row["text"],
+                                        row["level"],
+                                        row["step"],
+                                        row["alert_id"],
+                                    ),
+                                )
+                            except sqlite3.OperationalError:
+                                pass
+
+                        target_conn.commit()
+
+                        SQLiteStorage._move_media_dir(
+                            MEDIA_DIR / project / run,
+                            MEDIA_DIR / new_project / run,
+                        )
+
+                        source_cursor.execute(
+                            "DELETE FROM metrics WHERE run_name = ?", (run,)
+                        )
+                        source_cursor.execute(
+                            "DELETE FROM configs WHERE run_name = ?", (run,)
+                        )
+                        try:
+                            source_cursor.execute(
+                                "DELETE FROM system_metrics WHERE run_name = ?", (run,)
+                            )
+                        except sqlite3.OperationalError:
+                            pass
+                        try:
+                            source_cursor.execute(
+                                "DELETE FROM alerts WHERE run_name = ?", (run,)
+                            )
+                        except sqlite3.OperationalError:
+                            pass
+                        source_conn.commit()
+
+                        return True
+
+    @staticmethod
+    def get_all_run_configs(project: str) -> dict[str, dict]:
+        """Get configurations for all runs in a project."""
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return {}
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    """
+                    SELECT run_name, config FROM configs
+                    """
+                )
+
+                results = {}
+                for row in cursor.fetchall():
+                    config = orjson.loads(row["config"])
+                    results[row["run_name"]] = deserialize_values(config)
+                return results
+            except sqlite3.OperationalError as e:
+                if "no such table: configs" in str(e):
+                    return {}
+                raise
+
+    @staticmethod
+    def get_metric_values(
+        project: str,
+        run: str,
+        metric_name: str,
+        step: int | None = None,
+        around_step: int | None = None,
+        at_time: str | None = None,
+        window: int | float | None = None,
+    ) -> list[dict]:
+        """Get values for a specific metric in a project/run with optional filtering.
+
+        Filtering modes:
+          - step: return the single row at exactly this step
+          - around_step + window: return rows where step is in [around_step - window, around_step + window]
+          - at_time + window: return rows within ±window seconds of the ISO timestamp
+          - No filters: return all rows
+        """
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return []
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            query = "SELECT timestamp, step, metrics FROM metrics WHERE run_name = ?"
+            params: list = [run]
+
+            if step is not None:
+                query += " AND step = ?"
+                params.append(step)
+            elif around_step is not None and window is not None:
+                query += " AND step >= ? AND step <= ?"
+                params.extend([around_step - int(window), around_step + int(window)])
+            elif at_time is not None and window is not None:
+                query += (
+                    " AND timestamp >= datetime(?, '-' || ? || ' seconds')"
+                    " AND timestamp <= datetime(?, '+' || ? || ' seconds')"
+                )
+                params.extend([at_time, int(window), at_time, int(window)])
+
+            query += " ORDER BY timestamp"
+            cursor.execute(query, params)
+
+            rows = cursor.fetchall()
+            results = []
+            for row in rows:
+                metrics = orjson.loads(row["metrics"])
+                metrics = deserialize_values(metrics)
+                if metric_name in metrics:
+                    results.append(
+                        {
+                            "timestamp": row["timestamp"],
+                            "step": row["step"],
+                            "value": metrics[metric_name],
+                        }
+                    )
+            return results
+
+    @staticmethod
+    def get_snapshot(
+        project: str,
+        run: str,
+        step: int | None = None,
+        around_step: int | None = None,
+        at_time: str | None = None,
+        window: int | float | None = None,
+    ) -> dict[str, list[dict]]:
+        """Get all metrics at/around a point in time or step.
+
+        Returns a dict mapping metric names to lists of {timestamp, step, value}.
+        """
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return {}
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            query = "SELECT timestamp, step, metrics FROM metrics WHERE run_name = ?"
+            params: list = [run]
+
+            if step is not None:
+                query += " AND step = ?"
+                params.append(step)
+            elif around_step is not None and window is not None:
+                query += " AND step >= ? AND step <= ?"
+                params.extend([around_step - int(window), around_step + int(window)])
+            elif at_time is not None and window is not None:
+                query += (
+                    " AND timestamp >= datetime(?, '-' || ? || ' seconds')"
+                    " AND timestamp <= datetime(?, '+' || ? || ' seconds')"
+                )
+                params.extend([at_time, int(window), at_time, int(window)])
+
+            query += " ORDER BY timestamp"
+            cursor.execute(query, params)
+
+            result: dict[str, list[dict]] = {}
+            for row in cursor.fetchall():
+                metrics = orjson.loads(row["metrics"])
+                metrics = deserialize_values(metrics)
+                for key, value in metrics.items():
+                    if key not in result:
+                        result[key] = []
+                    result[key].append(
+                        {
+                            "timestamp": row["timestamp"],
+                            "step": row["step"],
+                            "value": value,
+                        }
+                    )
+            return result
+
+    @staticmethod
+    def get_all_metrics_for_run(project: str, run: str) -> list[str]:
+        """Get all metric names for a specific project/run."""
+        return SQLiteStorage._get_metric_names(
+            project, run, "metrics", exclude_keys={"timestamp", "step"}
+        )
+
+    @staticmethod
+    def _get_metric_names(
+        project: str, run: str, table: str, exclude_keys: set[str]
+    ) -> list[str]:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return []
+
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    f"""
+                    SELECT metrics
+                    FROM {table}
+                    WHERE run_name = ?
+                    ORDER BY timestamp
+                    """,
+                    (run,),
+                )
+
+                rows = cursor.fetchall()
+                all_metrics = set()
+                for row in rows:
+                    metrics = orjson.loads(row["metrics"])
+                    metrics = deserialize_values(metrics)
+                    for key in metrics.keys():
+                        if key not in exclude_keys:
+                            all_metrics.add(key)
+                return sorted(list(all_metrics))
+            except sqlite3.OperationalError as e:
+                if f"no such table: {table}" in str(e):
+                    return []
+                raise
+
+    @staticmethod
+    def set_project_metadata(project: str, key: str, value: str) -> None:
+        db_path = SQLiteStorage.init_db(project)
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                conn.execute(
+                    "INSERT OR REPLACE INTO project_metadata (key, value) VALUES (?, ?)",
+                    (key, value),
+                )
+                conn.commit()
+
+    @staticmethod
+    def get_project_metadata(project: str, key: str) -> str | None:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return None
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    "SELECT value FROM project_metadata WHERE key = ?", (key,)
+                )
+                row = cursor.fetchone()
+                return row[0] if row else None
+            except sqlite3.OperationalError:
+                return None
+
+    @staticmethod
+    def get_space_id(project: str) -> str | None:
+        return SQLiteStorage.get_project_metadata(project, "space_id")
+
+    @staticmethod
+    def has_pending_data(project: str) -> bool:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return False
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    "SELECT EXISTS(SELECT 1 FROM metrics WHERE space_id IS NOT NULL LIMIT 1)"
+                )
+                if cursor.fetchone()[0]:
+                    return True
+            except sqlite3.OperationalError:
+                pass
+            try:
+                cursor.execute(
+                    "SELECT EXISTS(SELECT 1 FROM system_metrics WHERE space_id IS NOT NULL LIMIT 1)"
+                )
+                if cursor.fetchone()[0]:
+                    return True
+            except sqlite3.OperationalError:
+                pass
+            try:
+                cursor.execute("SELECT EXISTS(SELECT 1 FROM pending_uploads LIMIT 1)")
+                if cursor.fetchone()[0]:
+                    return True
+            except sqlite3.OperationalError:
+                pass
+            return False
+
+    @staticmethod
+    def get_pending_logs(project: str) -> dict | None:
+        return SQLiteStorage._get_pending(
+            project, "metrics", extra_fields=["step"], include_config=True
+        )
+
+    @staticmethod
+    def clear_pending_logs(project: str, metric_ids: list[int]) -> None:
+        SQLiteStorage._clear_pending(project, "metrics", metric_ids)
+
+    @staticmethod
+    def get_pending_system_logs(project: str) -> dict | None:
+        return SQLiteStorage._get_pending(project, "system_metrics")
+
+    @staticmethod
+    def _get_pending(
+        project: str,
+        table: str,
+        extra_fields: list[str] | None = None,
+        include_config: bool = False,
+    ) -> dict | None:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return None
+        extra_cols = ", ".join(extra_fields) + ", " if extra_fields else ""
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    f"""SELECT id, timestamp, run_name, {extra_cols}metrics, log_id, space_id
+                    FROM {table} WHERE space_id IS NOT NULL"""
+                )
+            except sqlite3.OperationalError:
+                return None
+            rows = cursor.fetchall()
+            if not rows:
+                return None
+            logs = []
+            ids = []
+            for row in rows:
+                metrics = deserialize_values(orjson.loads(row["metrics"]))
+                entry = {
+                    "project": project,
+                    "run": row["run_name"],
+                    "metrics": metrics,
+                    "timestamp": row["timestamp"],
+                    "log_id": row["log_id"],
+                }
+                for field in extra_fields or []:
+                    entry[field] = row[field]
+                if include_config:
+                    entry["config"] = None
+                logs.append(entry)
+                ids.append(row["id"])
+            return {"logs": logs, "ids": ids, "space_id": rows[0]["space_id"]}
+
+    @staticmethod
+    def clear_pending_system_logs(project: str, metric_ids: list[int]) -> None:
+        SQLiteStorage._clear_pending(project, "system_metrics", metric_ids)
+
+    @staticmethod
+    def _clear_pending(project: str, table: str, ids: list[int]) -> None:
+        if not ids:
+            return
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                placeholders = ",".join("?" * len(ids))
+                conn.execute(
+                    f"DELETE FROM {table} WHERE id IN ({placeholders})",
+                    ids,
+                )
+                conn.commit()
+
+    @staticmethod
+    def get_pending_uploads(project: str) -> dict | None:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return None
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    """SELECT id, space_id, run_name, step, file_path, relative_path
+                    FROM pending_uploads"""
+                )
+            except sqlite3.OperationalError:
+                return None
+            rows = cursor.fetchall()
+            if not rows:
+                return None
+            uploads = []
+            ids = []
+            for row in rows:
+                uploads.append(
+                    {
+                        "project": project,
+                        "run": row["run_name"],
+                        "step": row["step"],
+                        "file_path": row["file_path"],
+                        "relative_path": row["relative_path"],
+                    }
+                )
+                ids.append(row["id"])
+            return {"uploads": uploads, "ids": ids, "space_id": rows[0]["space_id"]}
+
+    @staticmethod
+    def clear_pending_uploads(project: str, upload_ids: list[int]) -> None:
+        if not upload_ids:
+            return
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                placeholders = ",".join("?" * len(upload_ids))
+                conn.execute(
+                    f"DELETE FROM pending_uploads WHERE id IN ({placeholders})",
+                    upload_ids,
+                )
+                conn.commit()
+
+    @staticmethod
+    def add_pending_upload(
+        project: str,
+        space_id: str,
+        run_name: str | None,
+        step: int | None,
+        file_path: str,
+        relative_path: str | None,
+    ) -> None:
+        db_path = SQLiteStorage.init_db(project)
+        with SQLiteStorage._get_process_lock(project):
+            with SQLiteStorage._get_connection(db_path) as conn:
+                conn.execute(
+                    """INSERT INTO pending_uploads
+                    (space_id, run_name, step, file_path, relative_path, created_at)
+                    VALUES (?, ?, ?, ?, ?, ?)""",
+                    (
+                        space_id,
+                        run_name,
+                        step,
+                        file_path,
+                        relative_path,
+                        datetime.now(timezone.utc).isoformat(),
+                    ),
+                )
+                conn.commit()
+
+    @staticmethod
+    def get_all_logs_for_sync(project: str) -> list[dict]:
+        return SQLiteStorage._get_all_for_sync(
+            project,
+            "metrics",
+            order_by="run_name, step",
+            extra_fields=["step"],
+            include_config=True,
+        )
+
+    @staticmethod
+    def get_all_system_logs_for_sync(project: str) -> list[dict]:
+        return SQLiteStorage._get_all_for_sync(
+            project, "system_metrics", order_by="run_name, timestamp"
+        )
+
+    @staticmethod
+    def _get_all_for_sync(
+        project: str,
+        table: str,
+        order_by: str,
+        extra_fields: list[str] | None = None,
+        include_config: bool = False,
+    ) -> list[dict]:
+        db_path = SQLiteStorage.get_project_db_path(project)
+        if not db_path.exists():
+            return []
+        extra_cols = ", ".join(extra_fields) + ", " if extra_fields else ""
+        with SQLiteStorage._get_connection(db_path) as conn:
+            cursor = conn.cursor()
+            try:
+                cursor.execute(
+                    f"""SELECT timestamp, run_name, {extra_cols}metrics, log_id
+                    FROM {table} ORDER BY {order_by}"""
+                )
+            except sqlite3.OperationalError:
+                return []
+            rows = cursor.fetchall()
+            results = []
+            for row in rows:
+                metrics = deserialize_values(orjson.loads(row["metrics"]))
+                entry = {
+                    "project": project,
+                    "run": row["run_name"],
+                    "metrics": metrics,
+                    "timestamp": row["timestamp"],
+                    "log_id": row["log_id"],
+                }
+                for field in extra_fields or []:
+                    entry[field] = row[field]
+                if include_config:
+                    entry["config"] = None
+                results.append(entry)
+            return results

trackio/table.py

@@ -0,0 +1,173 @@
+import os
+from typing import Any, Literal
+
+from pandas import DataFrame
+
+from trackio.media.media import TrackioMedia
+from trackio.utils import MEDIA_DIR
+
+
+class Table:
+    """
+    Initializes a Table object.
+
+    Tables can be used to log tabular data including images, numbers, and text.
+
+    Args:
+        columns (`list[str]`, *optional*):
+            Names of the columns in the table. Optional if `data` is provided. Not
+            expected if `dataframe` is provided. Currently ignored.
+        data (`list[list[Any]]`, *optional*):
+            2D row-oriented array of values. Each value can be a number, a string
+            (treated as Markdown and truncated if too long), or a `Trackio.Image` or
+            list of `Trackio.Image` objects.
+        dataframe (`pandas.DataFrame`, *optional*):
+            DataFrame used to create the table. When set, `data` and `columns`
+            arguments are ignored.
+        rows (`list[list[Any]]`, *optional*):
+            Currently ignored.
+        optional (`bool` or `list[bool]`, *optional*, defaults to `True`):
+            Currently ignored.
+        allow_mixed_types (`bool`, *optional*, defaults to `False`):
+            Currently ignored.
+        log_mode: (`Literal["IMMUTABLE", "MUTABLE", "INCREMENTAL"]` or `None`, *optional*, defaults to `"IMMUTABLE"`):
+            Currently ignored.
+    """
+
+    TYPE = "trackio.table"
+
+    def __init__(
+        self,
+        columns: list[str] | None = None,
+        data: list[list[Any]] | None = None,
+        dataframe: DataFrame | None = None,
+        rows: list[list[Any]] | None = None,
+        optional: bool | list[bool] = True,
+        allow_mixed_types: bool = False,
+        log_mode: Literal["IMMUTABLE", "MUTABLE", "INCREMENTAL"] | None = "IMMUTABLE",
+    ):
+        # TODO: implement support for columns, dtype, optional, allow_mixed_types, and log_mode.
+        # for now (like `rows`) they are included for API compat but don't do anything.
+        if dataframe is None:
+            self.data = DataFrame(data) if data is not None else DataFrame()
+        else:
+            self.data = dataframe
+
+    def _has_media_objects(self, dataframe: DataFrame) -> bool:
+        """Check if dataframe contains any TrackioMedia objects or lists of TrackioMedia objects."""
+        for col in dataframe.columns:
+            if dataframe[col].apply(lambda x: isinstance(x, TrackioMedia)).any():
+                return True
+            if (
+                dataframe[col]
+                .apply(
+                    lambda x: (
+                        isinstance(x, list)
+                        and len(x) > 0
+                        and isinstance(x[0], TrackioMedia)
+                    )
+                )
+                .any()
+            ):
+                return True
+        return False
+
+    def _process_data(self, project: str, run: str, step: int = 0):
+        """Convert dataframe to dict format, processing any TrackioMedia objects if present."""
+        df = self.data
+        if not self._has_media_objects(df):
+            return df.to_dict(orient="records")
+
+        processed_df = df.copy()
+        for col in processed_df.columns:
+            for idx in processed_df.index:
+                value = processed_df.at[idx, col]
+                if isinstance(value, TrackioMedia):
+                    value._save(project, run, step)
+                    processed_df.at[idx, col] = value._to_dict()
+                if (
+                    isinstance(value, list)
+                    and len(value) > 0
+                    and isinstance(value[0], TrackioMedia)
+                ):
+                    [v._save(project, run, step) for v in value]
+                    processed_df.at[idx, col] = [v._to_dict() for v in value]
+
+        return processed_df.to_dict(orient="records")
+
+    @staticmethod
+    def to_display_format(table_data: list[dict]) -> list[dict]:
+        """
+        Converts stored table data to display format for UI rendering.
+
+        Note:
+            This does not use the `self.data` attribute, but instead uses the
+            `table_data` parameter, which is what the UI receives.
+
+        Args:
+            table_data (`list[dict]`):
+                List of dictionaries representing table rows (from stored `_value`).
+
+        Returns:
+            `list[dict]`: Table data with images converted to markdown syntax and long
+            text truncated.
+        """
+        truncate_length = int(os.getenv("TRACKIO_TABLE_TRUNCATE_LENGTH", "250"))
+
+        def convert_image_to_markdown(image_data: dict) -> str:
+            relative_path = image_data.get("file_path", "")
+            caption = image_data.get("caption", "")
+            absolute_path = MEDIA_DIR / relative_path
+            return f'<img src="/gradio_api/file={absolute_path}" alt="{caption}" />'
+
+        processed_data = []
+        for row in table_data:
+            processed_row = {}
+            for key, value in row.items():
+                if isinstance(value, dict) and value.get("_type") == "trackio.image":
+                    processed_row[key] = convert_image_to_markdown(value)
+                elif (
+                    isinstance(value, list)
+                    and len(value) > 0
+                    and isinstance(value[0], dict)
+                    and value[0].get("_type") == "trackio.image"
+                ):
+                    # This assumes that if the first item is an image, all items are images. Ok for now since we don't support mixed types in a single cell.
+                    processed_row[key] = (
+                        '<div style="display: flex; gap: 10px;">'
+                        + "".join([convert_image_to_markdown(item) for item in value])
+                        + "</div>"
+                    )
+                elif isinstance(value, str) and len(value) > truncate_length:
+                    truncated = value[:truncate_length]
+                    full_text = value.replace("<", "&lt;").replace(">", "&gt;")
+                    processed_row[key] = (
+                        f'<details style="display: inline;">'
+                        f'<summary style="display: inline; cursor: pointer;">{truncated}…<span><em>(truncated, click to expand)</em></span></summary>'
+                        f'<div style="margin-top: 10px; padding: 10px; background: #f5f5f5; border-radius: 4px; max-height: 400px; overflow: auto;">'
+                        f'<pre style="white-space: pre-wrap; word-wrap: break-word; margin: 0;">{full_text}</pre>'
+                        f"</div>"
+                        f"</details>"
+                    )
+                else:
+                    processed_row[key] = value
+            processed_data.append(processed_row)
+        return processed_data
+
+    def _to_dict(self, project: str, run: str, step: int = 0):
+        """
+        Converts the table to a dictionary representation.
+
+        Args:
+            project (`str`):
+                Project name for saving media files.
+            run (`str`):
+                Run name for saving media files.
+            step (`int`, *optional*, defaults to `0`):
+                Step number for saving media files.
+        """
+        data = self._process_data(project, run, step)
+        return {
+            "_type": self.TYPE,
+            "_value": data,
+        }

trackio/typehints.py

@@ -0,0 +1,39 @@
+from typing import Any, TypedDict
+
+from gradio import FileData
+
+
+class LogEntry(TypedDict, total=False):
+    project: str
+    run: str
+    metrics: dict[str, Any]
+    step: int | None
+    config: dict[str, Any] | None
+    log_id: str | None
+
+
+class SystemLogEntry(TypedDict, total=False):
+    project: str
+    run: str
+    metrics: dict[str, Any]
+    timestamp: str
+    log_id: str | None
+
+
+class AlertEntry(TypedDict, total=False):
+    project: str
+    run: str
+    title: str
+    text: str | None
+    level: str
+    step: int | None
+    timestamp: str
+    alert_id: str | None
+
+
+class UploadEntry(TypedDict):
+    project: str
+    run: str | None
+    step: int | None
+    relative_path: str | None
+    uploaded_file: FileData

trackio/utils.py

@@ -0,0 +1,919 @@
+import math
+import os
+import re
+import secrets
+import time
+from datetime import datetime, timezone
+from functools import lru_cache
+from pathlib import Path
+from typing import TYPE_CHECKING
+from urllib.parse import urlencode
+
+import huggingface_hub
+import numpy as np
+import pandas as pd
+from huggingface_hub.constants import HF_HOME
+
+if TYPE_CHECKING:
+    from trackio.commit_scheduler import CommitScheduler
+    from trackio.dummy_commit_scheduler import DummyCommitScheduler
+
+RESERVED_KEYS = ["project", "run", "timestamp", "step", "time", "metrics"]
+
+TRACKIO_LOGO_DIR = Path(__file__).parent / "assets"
+
+
+def get_logo_urls() -> dict[str, str]:
+    """Get logo URLs from environment variables or use defaults."""
+    light_url = os.environ.get(
+        "TRACKIO_LOGO_LIGHT_URL",
+        f"/gradio_api/file={TRACKIO_LOGO_DIR}/trackio_logo_type_light_transparent.png",
+    )
+    dark_url = os.environ.get(
+        "TRACKIO_LOGO_DARK_URL",
+        f"/gradio_api/file={TRACKIO_LOGO_DIR}/trackio_logo_type_dark_transparent.png",
+    )
+    return {"light": light_url, "dark": dark_url}
+
+
+def order_metrics_by_plot_preference(metrics: list[str]) -> tuple[list[str], dict]:
+    """
+    Order metrics based on TRACKIO_PLOT_ORDER environment variable and group them.
+
+    Args:
+        metrics: List of metric names to order and group
+
+    Returns:
+        Tuple of (ordered_group_names, grouped_metrics_dict)
+    """
+    plot_order_env = os.environ.get("TRACKIO_PLOT_ORDER", "")
+    if not plot_order_env.strip():
+        plot_order = []
+    else:
+        plot_order = [
+            item.strip() for item in plot_order_env.split(",") if item.strip()
+        ]
+
+    def get_metric_priority(metric: str) -> tuple[int, int, str]:
+        if not plot_order:
+            return (float("inf"), float("inf"), metric)
+
+        group_prefix = metric.split("/")[0] if "/" in metric else "charts"
+        no_match_priority = len(plot_order)
+
+        group_priority = no_match_priority
+        for i, pattern in enumerate(plot_order):
+            pattern_group = pattern.split("/")[0] if "/" in pattern else "charts"
+            if pattern_group == group_prefix:
+                group_priority = i
+                break
+
+        within_group_priority = no_match_priority
+        for i, pattern in enumerate(plot_order):
+            if pattern == metric:
+                within_group_priority = i
+                break
+            elif pattern.endswith("/*") and within_group_priority == no_match_priority:
+                pattern_prefix = pattern[:-2]
+                if metric.startswith(pattern_prefix + "/"):
+                    within_group_priority = i + len(plot_order)
+
+        return (group_priority, within_group_priority, metric)
+
+    result = {}
+    for metric in metrics:
+        if "/" not in metric:
+            if "charts" not in result:
+                result["charts"] = {"direct_metrics": [], "subgroups": {}}
+            result["charts"]["direct_metrics"].append(metric)
+        else:
+            parts = metric.split("/")
+            main_prefix = parts[0]
+            if main_prefix not in result:
+                result[main_prefix] = {"direct_metrics": [], "subgroups": {}}
+            if len(parts) == 2:
+                result[main_prefix]["direct_metrics"].append(metric)
+            else:
+                subprefix = parts[1]
+                if subprefix not in result[main_prefix]["subgroups"]:
+                    result[main_prefix]["subgroups"][subprefix] = []
+                result[main_prefix]["subgroups"][subprefix].append(metric)
+
+    for group_data in result.values():
+        group_data["direct_metrics"].sort(key=get_metric_priority)
+        for subgroup_name in group_data["subgroups"]:
+            group_data["subgroups"][subgroup_name].sort(key=get_metric_priority)
+
+    if "charts" in result and not result["charts"]["direct_metrics"]:
+        del result["charts"]
+
+    def get_group_priority(group_name: str) -> tuple[int, str]:
+        if not plot_order:
+            return (float("inf"), group_name)
+
+        min_priority = len(plot_order)
+        for i, pattern in enumerate(plot_order):
+            pattern_group = pattern.split("/")[0] if "/" in pattern else "charts"
+            if pattern_group == group_name:
+                min_priority = min(min_priority, i)
+        return (min_priority, group_name)
+
+    ordered_groups = sorted(result.keys(), key=get_group_priority)
+
+    return ordered_groups, result
+
+
+def persistent_storage_enabled() -> bool:
+    return (
+        os.environ.get("PERSISTANT_STORAGE_ENABLED") == "true"
+    )  # typo in the name of the environment variable
+
+
+def _get_trackio_dir() -> Path:
+    if persistent_storage_enabled():
+        return Path("/data/trackio")
+    elif os.environ.get("TRACKIO_DIR"):
+        return Path(os.environ.get("TRACKIO_DIR"))
+    return Path(HF_HOME) / "trackio"
+
+
+TRACKIO_DIR = _get_trackio_dir()
+MEDIA_DIR = TRACKIO_DIR / "media"
+
+
+def get_or_create_project_hash(project: str) -> str:
+    hash_path = TRACKIO_DIR / f"{project}.hash"
+    if hash_path.exists():
+        return hash_path.read_text().strip()
+    hash_value = secrets.token_urlsafe(8)
+    TRACKIO_DIR.mkdir(parents=True, exist_ok=True)
+    hash_path.write_text(hash_value)
+    return hash_value
+
+
+def generate_readable_name(used_names: list[str], space_id: str | None = None) -> str:
+    """
+    Generates a random, readable name like "dainty-sunset-0".
+    If space_id is provided, generates username-timestamp format instead.
+    """
+    if space_id is not None:
+        username = _get_default_namespace()
+        timestamp = int(time.time())
+        return f"{username}-{timestamp}"
+    adjectives = [
+        "dainty",
+        "brave",
+        "calm",
+        "eager",
+        "fancy",
+        "gentle",
+        "happy",
+        "jolly",
+        "kind",
+        "lively",
+        "merry",
+        "nice",
+        "proud",
+        "quick",
+        "hugging",
+        "silly",
+        "tidy",
+        "witty",
+        "zealous",
+        "bright",
+        "shy",
+        "bold",
+        "clever",
+        "daring",
+        "elegant",
+        "faithful",
+        "graceful",
+        "honest",
+        "inventive",
+        "jovial",
+        "keen",
+        "lucky",
+        "modest",
+        "noble",
+        "optimistic",
+        "patient",
+        "quirky",
+        "resourceful",
+        "sincere",
+        "thoughtful",
+        "upbeat",
+        "valiant",
+        "warm",
+        "youthful",
+        "zesty",
+        "adventurous",
+        "breezy",
+        "cheerful",
+        "delightful",
+        "energetic",
+        "fearless",
+        "glad",
+        "hopeful",
+        "imaginative",
+        "joyful",
+        "kindly",
+        "luminous",
+        "mysterious",
+        "neat",
+        "outgoing",
+        "playful",
+        "radiant",
+        "spirited",
+        "tranquil",
+        "unique",
+        "vivid",
+        "wise",
+        "zany",
+        "artful",
+        "bubbly",
+        "charming",
+        "dazzling",
+        "earnest",
+        "festive",
+        "gentlemanly",
+        "hearty",
+        "intrepid",
+        "jubilant",
+        "knightly",
+        "lively",
+        "magnetic",
+        "nimble",
+        "orderly",
+        "peaceful",
+        "quick-witted",
+        "robust",
+        "sturdy",
+        "trusty",
+        "upstanding",
+        "vibrant",
+        "whimsical",
+    ]
+    nouns = [
+        "sunset",
+        "forest",
+        "river",
+        "mountain",
+        "breeze",
+        "meadow",
+        "ocean",
+        "valley",
+        "sky",
+        "field",
+        "cloud",
+        "star",
+        "rain",
+        "leaf",
+        "stone",
+        "flower",
+        "bird",
+        "tree",
+        "wave",
+        "trail",
+        "island",
+        "desert",
+        "hill",
+        "lake",
+        "pond",
+        "grove",
+        "canyon",
+        "reef",
+        "bay",
+        "peak",
+        "glade",
+        "marsh",
+        "cliff",
+        "dune",
+        "spring",
+        "brook",
+        "cave",
+        "plain",
+        "ridge",
+        "wood",
+        "blossom",
+        "petal",
+        "root",
+        "branch",
+        "seed",
+        "acorn",
+        "pine",
+        "willow",
+        "cedar",
+        "elm",
+        "falcon",
+        "eagle",
+        "sparrow",
+        "robin",
+        "owl",
+        "finch",
+        "heron",
+        "crane",
+        "duck",
+        "swan",
+        "fox",
+        "wolf",
+        "bear",
+        "deer",
+        "moose",
+        "otter",
+        "beaver",
+        "lynx",
+        "hare",
+        "badger",
+        "butterfly",
+        "bee",
+        "ant",
+        "beetle",
+        "dragonfly",
+        "firefly",
+        "ladybug",
+        "moth",
+        "spider",
+        "worm",
+        "coral",
+        "kelp",
+        "shell",
+        "pebble",
+        "face",
+        "boulder",
+        "cobble",
+        "sand",
+        "wavelet",
+        "tide",
+        "current",
+        "mist",
+    ]
+    number = 0
+    name = f"{adjectives[0]}-{nouns[0]}-{number}"
+    while name in used_names:
+        number += 1
+        adjective = adjectives[number % len(adjectives)]
+        noun = nouns[number % len(nouns)]
+        name = f"{adjective}-{noun}-{number}"
+    return name
+
+
+def is_in_notebook():
+    """
+    Detect if code is running in a notebook environment (Jupyter, Colab, etc.).
+    """
+    try:
+        from IPython import get_ipython
+
+        if get_ipython() is not None:
+            return get_ipython().__class__.__name__ in [
+                "ZMQInteractiveShell",  # Jupyter notebook/lab
+                "Shell",  # IPython terminal
+            ] or "google.colab" in str(get_ipython())
+    except ImportError:
+        pass
+    return False
+
+
+def block_main_thread_until_keyboard_interrupt():
+    try:
+        while True:
+            time.sleep(0.1)
+    except (KeyboardInterrupt, OSError):
+        print("Keyboard interruption in main thread... closing dashboard.")
+
+
+def simplify_column_names(columns: list[str]) -> dict[str, str]:
+    """
+    Simplifies column names to first 10 alphanumeric or "/" characters with unique suffixes.
+
+    Args:
+        columns: List of original column names
+
+    Returns:
+        Dictionary mapping original column names to simplified names
+    """
+    simplified_names = {}
+    used_names = set()
+
+    for col in columns:
+        alphanumeric = re.sub(r"[^a-zA-Z0-9/]", "", col)
+        base_name = alphanumeric[:10] if alphanumeric else f"col_{len(used_names)}"
+
+        final_name = base_name
+        suffix = 1
+        while final_name in used_names:
+            final_name = f"{base_name}_{suffix}"
+            suffix += 1
+
+        simplified_names[col] = final_name
+        used_names.add(final_name)
+
+    return simplified_names
+
+
+def print_dashboard_instructions(project: str) -> None:
+    """
+    Prints instructions for viewing the Trackio dashboard.
+
+    Args:
+        project: The name of the project to show dashboard for.
+    """
+    ORANGE = "\033[38;5;208m"
+    BOLD = "\033[1m"
+    RESET = "\033[0m"
+
+    print("* View dashboard by running in your terminal:")
+    print(f'{BOLD}{ORANGE}trackio show --project "{project}"{RESET}')
+    print(f'* or by running in Python: trackio.show(project="{project}")')
+
+
+def preprocess_space_and_dataset_ids(
+    space_id: str | None, dataset_id: str | None
+) -> tuple[str | None, str | None]:
+    """
+    Preprocesses the Space and Dataset names to ensure they are valid "username/space_id" or "username/dataset_id" format.
+    """
+    if space_id is not None and "/" not in space_id:
+        username = _get_default_namespace()
+        space_id = f"{username}/{space_id}"
+    if dataset_id is not None and "/" not in dataset_id:
+        username = _get_default_namespace()
+        dataset_id = f"{username}/{dataset_id}"
+    if space_id is not None and dataset_id is None:
+        dataset_id = f"{space_id}-dataset"
+    return space_id, dataset_id
+
+
+def fibo():
+    """Generator for Fibonacci backoff: 1, 1, 2, 3, 5, 8, ..."""
+    a, b = 1, 1
+    while True:
+        yield a
+        a, b = b, a + b
+
+
+def format_timestamp(timestamp_str):
+    """Convert ISO timestamp to human-readable format like '3 minutes ago'."""
+    if not timestamp_str or pd.isna(timestamp_str):
+        return "Unknown"
+
+    try:
+        created_time = datetime.fromisoformat(timestamp_str.replace("Z", "+00:00"))
+        if created_time.tzinfo is None:
+            created_time = created_time.replace(tzinfo=timezone.utc)
+
+        now = datetime.now(timezone.utc)
+        diff = now - created_time
+
+        seconds = int(diff.total_seconds())
+        if seconds < 60:
+            return "Just now"
+        elif seconds < 3600:
+            minutes = seconds // 60
+            return f"{minutes} minute{'s' if minutes != 1 else ''} ago"
+        elif seconds < 86400:
+            hours = seconds // 3600
+            return f"{hours} hour{'s' if hours != 1 else ''} ago"
+        else:
+            days = seconds // 86400
+            return f"{days} day{'s' if days != 1 else ''} ago"
+    except Exception:
+        return "Unknown"
+
+
+DEFAULT_COLOR_PALETTE = [
+    "#A8769B",
+    "#E89957",
+    "#3B82F6",
+    "#10B981",
+    "#EF4444",
+    "#8B5CF6",
+    "#14B8A6",
+    "#F59E0B",
+    "#EC4899",
+    "#06B6D4",
+]
+
+
+def get_color_palette() -> list[str]:
+    """Get the color palette from environment variable or use default."""
+    env_palette = os.environ.get("TRACKIO_COLOR_PALETTE")
+    if env_palette:
+        return [color.strip() for color in env_palette.split(",")]
+    return DEFAULT_COLOR_PALETTE
+
+
+def get_color_mapping(
+    runs: list[str], smoothing: bool, color_palette: list[str] | None = None
+) -> dict[str, str]:
+    """Generate color mapping for runs, with transparency for original data when smoothing is enabled."""
+    if color_palette is None:
+        color_palette = get_color_palette()
+
+    color_map = {}
+
+    for i, run in enumerate(runs):
+        base_color = color_palette[i % len(color_palette)]
+
+        if smoothing:
+            color_map[run] = base_color + "4D"
+            color_map[f"{run}_smoothed"] = base_color
+        else:
+            color_map[run] = base_color
+
+    return color_map
+
+
+def downsample(
+    df: pd.DataFrame,
+    x: str,
+    y: str,
+    color: str | None,
+    x_lim: tuple[float | None, float | None] | None = None,
+) -> tuple[pd.DataFrame, tuple[float, float] | None]:
+    """
+    Downsample the dataframe to reduce the number of points plotted.
+    Also updates the x-axis limits to the data min/max if either of the x-axis limits are None.
+
+    Args:
+        df: The dataframe to downsample.
+        x: The column name to use for the x-axis.
+        y: The column name to use for the y-axis.
+        color: The column name to use for the color.
+        x_lim: The x-axis limits to use.
+
+    Returns:
+        A tuple containing the downsampled dataframe and the updated x-axis limits.
+    """
+    if df.empty:
+        if x_lim is not None:
+            x_lim = (x_lim[0] or 0, x_lim[1] or 0)
+        return df, x_lim
+
+    columns_to_keep = [x, y]
+    if color is not None and color in df.columns:
+        columns_to_keep.append(color)
+    df = df[columns_to_keep].copy()
+
+    data_x_min = df[x].min()
+    data_x_max = df[x].max()
+
+    if x_lim is not None:
+        x_min, x_max = x_lim
+        if x_min is None:
+            x_min = data_x_min
+        if x_max is None:
+            x_max = data_x_max
+        updated_x_lim = (x_min, x_max)
+    else:
+        updated_x_lim = None
+
+    n_bins = 100
+
+    if color is not None and color in df.columns:
+        groups = df.groupby(color)
+    else:
+        groups = [(None, df)]
+
+    downsampled_indices = []
+
+    for _, group_df in groups:
+        if group_df.empty:
+            continue
+
+        group_df = group_df.sort_values(x)
+
+        if updated_x_lim is not None:
+            x_min, x_max = updated_x_lim
+            before_point = group_df[group_df[x] < x_min].tail(1)
+            after_point = group_df[group_df[x] > x_max].head(1)
+            group_df = group_df[(group_df[x] >= x_min) & (group_df[x] <= x_max)]
+        else:
+            before_point = after_point = None
+            x_min = group_df[x].min()
+            x_max = group_df[x].max()
+
+        if before_point is not None and not before_point.empty:
+            downsampled_indices.extend(before_point.index.tolist())
+        if after_point is not None and not after_point.empty:
+            downsampled_indices.extend(after_point.index.tolist())
+
+        if group_df.empty:
+            continue
+
+        if x_min == x_max:
+            min_y_idx = group_df[y].idxmin()
+            max_y_idx = group_df[y].idxmax()
+            if min_y_idx != max_y_idx:
+                downsampled_indices.extend([min_y_idx, max_y_idx])
+            else:
+                downsampled_indices.append(min_y_idx)
+            continue
+
+        if len(group_df) < 500:
+            downsampled_indices.extend(group_df.index.tolist())
+            continue
+
+        bins = np.linspace(x_min, x_max, n_bins + 1)
+        group_df["bin"] = pd.cut(
+            group_df[x], bins=bins, labels=False, include_lowest=True
+        )
+
+        for bin_idx in group_df["bin"].dropna().unique():
+            bin_data = group_df[group_df["bin"] == bin_idx]
+            if bin_data.empty:
+                continue
+
+            min_y_idx = bin_data[y].idxmin()
+            max_y_idx = bin_data[y].idxmax()
+
+            downsampled_indices.append(min_y_idx)
+            if min_y_idx != max_y_idx:
+                downsampled_indices.append(max_y_idx)
+
+    unique_indices = list(set(downsampled_indices))
+
+    downsampled_df = df.loc[unique_indices].copy()
+
+    if color is not None:
+        downsampled_df = (
+            downsampled_df.groupby(color, sort=False)[downsampled_df.columns]
+            .apply(lambda group: group.sort_values(x))
+            .reset_index(drop=True)
+        )
+    else:
+        downsampled_df = downsampled_df.sort_values(x).reset_index(drop=True)
+
+    downsampled_df = downsampled_df.drop(columns=["bin"], errors="ignore")
+
+    return downsampled_df, updated_x_lim
+
+
+def sort_metrics_by_prefix(metrics: list[str]) -> list[str]:
+    """
+    Sort metrics by grouping prefixes together for dropdown/list display.
+    Metrics without prefixes come first, then grouped by prefix.
+
+    Args:
+        metrics: List of metric names
+
+    Returns:
+        List of metric names sorted by prefix
+
+    Example:
+    Input: ["train/loss", "loss", "train/acc", "val/loss"]
+    Output: ["loss", "train/acc", "train/loss", "val/loss"]
+    """
+    groups = group_metrics_by_prefix(metrics)
+    result = []
+
+    if "charts" in groups:
+        result.extend(groups["charts"])
+
+    for group_name in sorted(groups.keys()):
+        if group_name != "charts":
+            result.extend(groups[group_name])
+
+    return result
+
+
+def group_metrics_by_prefix(metrics: list[str]) -> dict[str, list[str]]:
+    """
+    Group metrics by their prefix. Metrics without prefix go to 'charts' group.
+
+    Args:
+        metrics: List of metric names
+
+    Returns:
+        Dictionary with prefix names as keys and lists of metrics as values
+
+    Example:
+        Input: ["loss", "accuracy", "train/loss", "train/acc", "val/loss"]
+        Output: {
+            "charts": ["loss", "accuracy"],
+            "train": ["train/loss", "train/acc"],
+            "val": ["val/loss"]
+        }
+    """
+    no_prefix = []
+    with_prefix = []
+
+    for metric in metrics:
+        if "/" in metric:
+            with_prefix.append(metric)
+        else:
+            no_prefix.append(metric)
+
+    no_prefix.sort()
+
+    prefix_groups = {}
+    for metric in with_prefix:
+        prefix = metric.split("/")[0]
+        if prefix not in prefix_groups:
+            prefix_groups[prefix] = []
+        prefix_groups[prefix].append(metric)
+
+    for prefix in prefix_groups:
+        prefix_groups[prefix].sort()
+
+    groups = {}
+    if no_prefix:
+        groups["charts"] = no_prefix
+
+    for prefix in sorted(prefix_groups.keys()):
+        groups[prefix] = prefix_groups[prefix]
+
+    return groups
+
+
+def get_sync_status(scheduler: "CommitScheduler | DummyCommitScheduler") -> int | None:
+    """Get the sync status from the CommitScheduler in an integer number of minutes, or None if not synced yet."""
+    if getattr(
+        scheduler, "last_push_time", None
+    ):  # DummyCommitScheduler doesn't have last_push_time
+        time_diff = time.time() - scheduler.last_push_time
+        return int(time_diff / 60)
+    else:
+        return None
+
+
+def generate_share_url(
+    project: str,
+    metrics: str,
+    selected_runs: list = None,
+    hide_headers: bool = False,
+) -> str:
+    """Generate the shareable Space URL based on current settings."""
+    space_host = os.environ.get("SPACE_HOST", "")
+    if not space_host:
+        return ""
+
+    params: dict[str, str] = {}
+
+    if project:
+        params["project"] = project
+
+    if metrics and metrics.strip():
+        params["metrics"] = metrics
+
+    if selected_runs:
+        params["runs"] = ",".join(selected_runs)
+
+    if hide_headers:
+        params["accordion"] = "hidden"
+    params["sidebar"] = "hidden"
+    params["navbar"] = "hidden"
+
+    query_string = urlencode(params)
+    return f"https://{space_host}?{query_string}"
+
+
+def generate_embed_code(
+    project: str,
+    metrics: str,
+    selected_runs: list = None,
+    hide_headers: bool = False,
+) -> str:
+    """Generate the embed iframe code based on current settings."""
+    embed_url = generate_share_url(project, metrics, selected_runs, hide_headers)
+    if not embed_url:
+        return ""
+
+    return f'<iframe src="{embed_url}" style="width:1600px; height:500px; border:0;"></iframe>'
+
+
+def serialize_values(metrics):
+    """
+    Serialize infinity and NaN values in metrics dict to make it JSON-compliant.
+    Only handles top-level float values.
+
+    Converts:
+    - float('inf') -> "Infinity"
+    - float('-inf') -> "-Infinity"
+    - float('nan') -> "NaN"
+
+    Example:
+        {"loss": float('inf'), "accuracy": 0.95} -> {"loss": "Infinity", "accuracy": 0.95}
+    """
+    if not isinstance(metrics, dict):
+        return metrics
+
+    result = {}
+    for key, value in metrics.items():
+        if isinstance(value, float):
+            if math.isinf(value):
+                result[key] = "Infinity" if value > 0 else "-Infinity"
+            elif math.isnan(value):
+                result[key] = "NaN"
+            else:
+                result[key] = value
+        elif isinstance(value, np.floating):
+            float_val = float(value)
+            if math.isinf(float_val):
+                result[key] = "Infinity" if float_val > 0 else "-Infinity"
+            elif math.isnan(float_val):
+                result[key] = "NaN"
+            else:
+                result[key] = float_val
+        else:
+            result[key] = value
+    return result
+
+
+def deserialize_values(metrics):
+    """
+    Deserialize infinity and NaN string values back to their numeric forms.
+    Only handles top-level string values.
+
+    Converts:
+    - "Infinity" -> float('inf')
+    - "-Infinity" -> float('-inf')
+    - "NaN" -> float('nan')
+
+    Example:
+        {"loss": "Infinity", "accuracy": 0.95} -> {"loss": float('inf'), "accuracy": 0.95}
+    """
+    if not isinstance(metrics, dict):
+        return metrics
+
+    result = {}
+    for key, value in metrics.items():
+        if value == "Infinity":
+            result[key] = float("inf")
+        elif value == "-Infinity":
+            result[key] = float("-inf")
+        elif value == "NaN":
+            result[key] = float("nan")
+        else:
+            result[key] = value
+    return result
+
+
+def get_full_url(
+    base_url: str, project: str | None, write_token: str, footer: bool = True
+) -> str:
+    params = []
+    if project:
+        params.append(f"project={project}")
+    params.append(f"write_token={write_token}")
+    if not footer:
+        params.append("footer=false")
+    return base_url + "?" + "&".join(params)
+
+
+def embed_url_in_notebook(url: str) -> None:
+    try:
+        from IPython.display import HTML, display
+
+        embed_code = HTML(
+            f'<div><iframe src="{url}" width="100%" height="1000px" allow="autoplay; camera; microphone; clipboard-read; clipboard-write;" frameborder="0" allowfullscreen></iframe></div>'
+        )
+        display(embed_code)
+    except ImportError:
+        pass
+
+
+def to_json_safe(obj):
+    if isinstance(obj, (str, int, float, bool, type(None))):
+        return obj
+    if isinstance(obj, np.generic):
+        return obj.item()
+    if isinstance(obj, dict):
+        return {str(k): to_json_safe(v) for k, v in obj.items()}
+    if isinstance(obj, (list, tuple, set)):
+        return [to_json_safe(v) for v in obj]
+    if hasattr(obj, "to_dict") and callable(obj.to_dict):
+        return to_json_safe(obj.to_dict())
+    if hasattr(obj, "__dict__"):
+        return {
+            str(k): to_json_safe(v)
+            for k, v in vars(obj).items()
+            if not k.startswith("_")
+        }
+    return str(obj)
+
+
+def get_space() -> str | None:
+    """
+    Get the space ID ("user/space") if Trackio is running in a Space, or None if not.
+    """
+    return os.environ.get("SPACE_ID")
+
+
+def ordered_subset(items: list[str], subset: list[str] | None) -> list[str]:
+    subset_set = set(subset or [])
+    return [item for item in items if item in subset_set]
+
+
+def _get_default_namespace() -> str:
+    """Get the default namespace (username).
+
+    This function uses caching to avoid repeated API calls to /whoami-v2.
+    """
+    token = huggingface_hub.get_token()
+    return _cached_whoami(token)["name"]
+
+
+@lru_cache(maxsize=32)
+def _cached_whoami(token: str | None) -> dict:
+    return huggingface_hub.whoami(token=token)