Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+trackio/assets/trackio_logo_old.png filter=lfs diff=lfs merge=lfs -text

trackio/CHANGELOG.md

@@ -0,0 +1,152 @@
+# trackio
+
+## 0.16.1
+
+### Features
+
+- [#431](https://github.com/gradio-app/trackio/pull/431) [`c7ce55b`](https://github.com/gradio-app/trackio/commit/c7ce55b14dd5eb0c2165fb15df17dd60721c9325) - Lazy load the UI when trackio is imported.  Thanks @abidlabs!
+
+## 0.16.0
+
+### Features
+
+- [#426](https://github.com/gradio-app/trackio/pull/426) [`ead4dc8`](https://github.com/gradio-app/trackio/commit/ead4dc8e74ee2d8e47d61bca0a7668456acf49be) - Fix redundant double rendering of group checkboxes.  Thanks @abidlabs!
+- [#413](https://github.com/gradio-app/trackio/pull/413) [`39c4750`](https://github.com/gradio-app/trackio/commit/39c4750951d554ba6eb4d58847c6bb444b2891a8) - Check `dist-packages` when checking for source installation.  Thanks @sergiopaniego!
+- [#423](https://github.com/gradio-app/trackio/pull/423) [`2e52ab3`](https://github.com/gradio-app/trackio/commit/2e52ab303e3041718a6a56fbf84d0848aca9ad67) - Fix legend outline visibility issue.  Thanks @Raghunath-Balaji!
+- [#407](https://github.com/gradio-app/trackio/pull/407) [`c8a384d`](https://github.com/gradio-app/trackio/commit/c8a384ddfe5a295cecf862a26178d40e48acb424) - Fix pytests that were failling locally on MacOS.  Thanks @abidlabs!
+- [#405](https://github.com/gradio-app/trackio/pull/405) [`35aae4e`](https://github.com/gradio-app/trackio/commit/35aae4e3aa3e2b2888887528478b9dc6a9808bda) - Add conditional padding for HF Space dashboard when not in iframe.  Thanks @znation!
+
+## 0.15.0
+
+### Features
+
+- [#397](https://github.com/gradio-app/trackio/pull/397) [`6b38ad0`](https://github.com/gradio-app/trackio/commit/6b38ad02e5d73a0df49c4eede7e91331282ece04) - Adds `--host` cli option support.  Thanks @abidlabs!
+- [#396](https://github.com/gradio-app/trackio/pull/396) [`4a4d1ab`](https://github.com/gradio-app/trackio/commit/4a4d1ab85e63d923132a3fa7afa5d90e16431bec) - Fix run selection issue.  Thanks @abidlabs!
+- [#394](https://github.com/gradio-app/trackio/pull/394) [`c47a3a3`](https://github.com/gradio-app/trackio/commit/c47a3a31f8c4b83bce1aa7fc22eeba3d9021ad3d) - Add wandb-compatible API for trackio.  Thanks @abidlabs!
+- [#378](https://github.com/gradio-app/trackio/pull/378) [`b02046a`](https://github.com/gradio-app/trackio/commit/b02046a5b0dad7c9854e099a87f884afba4aecb2) - Add JSON export button for line plots and upgrade gradio dependency.  Thanks @JamshedAli18!
+
+## 0.14.2
+
+### Features
+
+- [#386](https://github.com/gradio-app/trackio/pull/386) [`f9452cd`](https://github.com/gradio-app/trackio/commit/f9452cdb8f0819368f3610f7ac0ed08957305275) - Fixing some issues related to deployed Trackio Spaces.  Thanks @abidlabs!
+
+## 0.14.1
+
+### Features
+
+- [#382](https://github.com/gradio-app/trackio/pull/382) [`44fe9bb`](https://github.com/gradio-app/trackio/commit/44fe9bb264fb2aafb0ec302ff15227c045819a2c) - Fix app file path when Trackio is not installed from source.  Thanks @abidlabs!
+- [#380](https://github.com/gradio-app/trackio/pull/380) [`c3f4cff`](https://github.com/gradio-app/trackio/commit/c3f4cff74bc5676e812773d8571454894fcdc7cc) - Add CLI commands for querying projects, runs, and metrics.  Thanks @abidlabs!
+
+## 0.14.0
+
+### Features
+
+- [#377](https://github.com/gradio-app/trackio/pull/377) [`5c5015b`](https://github.com/gradio-app/trackio/commit/5c5015b68c85c5de51111dad983f735c27b9a05f) - fixed wrapping issue in Runs table.  Thanks @gaganchapa!
+- [#374](https://github.com/gradio-app/trackio/pull/374) [`388e26b`](https://github.com/gradio-app/trackio/commit/388e26b9e9f24cd7ad203affe9b709be885b3d24) - Save Optimized Parquet files.  Thanks @lhoestq!
+- [#371](https://github.com/gradio-app/trackio/pull/371) [`fbace9c`](https://github.com/gradio-app/trackio/commit/fbace9cd7732c166f34d268f54b05bb06846cc5d) - Add GPU metrics logging.  Thanks @kashif!
+- [#367](https://github.com/gradio-app/trackio/pull/367) [`862840c`](https://github.com/gradio-app/trackio/commit/862840c13e30fc960cbee5b9eac4d3c25beba9de) - Add option to only show latest run, and fix the double logo issue.  Thanks @abidlabs!
+
+## 0.13.1
+
+### Features
+
+- [#369](https://github.com/gradio-app/trackio/pull/369) [`767e9fe`](https://github.com/gradio-app/trackio/commit/767e9fe095d7c6ed102016caf927c1517fb8618c) - tiny pr removing unnecessary code.  Thanks @abidlabs!
+
+## 0.13.0
+
+### Features
+
+- [#358](https://github.com/gradio-app/trackio/pull/358) [`073715d`](https://github.com/gradio-app/trackio/commit/073715d1caf8282f68890117f09c3ac301205312) - Improvements to `trackio.sync()`.  Thanks @abidlabs!
+
+## 0.12.0
+
+### Features
+
+- [#357](https://github.com/gradio-app/trackio/pull/357) [`02ba815`](https://github.com/gradio-app/trackio/commit/02ba815358060f1966052de051a5bdb09702920e) - Redesign media and tables to show up on separate page.  Thanks @abidlabs!
+- [#359](https://github.com/gradio-app/trackio/pull/359) [`08fe9c9`](https://github.com/gradio-app/trackio/commit/08fe9c9ddd7fe99ee811555fdfb62df9ab88e939) - docs: Improve docstrings.  Thanks @qgallouedec!
+
+## 0.11.0
+
+### Features
+
+- [#355](https://github.com/gradio-app/trackio/pull/355) [`ea51f49`](https://github.com/gradio-app/trackio/commit/ea51f4954922f21be76ef828700420fe9a912c4b) - Color code run checkboxes and match with plot lines.  Thanks @abidlabs!
+- [#353](https://github.com/gradio-app/trackio/pull/353) [`8abe691`](https://github.com/gradio-app/trackio/commit/8abe6919aeefe21fc7a23af814883efbb037c21f) - Remove show_api from demo.launch.  Thanks @sergiopaniego!
+- [#351](https://github.com/gradio-app/trackio/pull/351) [`8a8957e`](https://github.com/gradio-app/trackio/commit/8a8957e530dd7908d1fef7f2df030303f808101f) - Add `trackio.save()`.  Thanks @abidlabs!
+
+## 0.10.0
+
+### Features
+
+- [#305](https://github.com/gradio-app/trackio/pull/305) [`e64883a`](https://github.com/gradio-app/trackio/commit/e64883a51f7b8b93f7d48b8afe55acdb62238b71) - bump to gradio 6.0, make `trackio` compatible, and fix related issues.  Thanks @abidlabs!
+
+## 0.9.1
+
+### Features
+
+- [#344](https://github.com/gradio-app/trackio/pull/344) [`7e01024`](https://github.com/gradio-app/trackio/commit/7e010241d9a34794e0ce0dc19c1a6f0cf94ba856) - Avoid redundant calls to /whoami-v2.  Thanks @Wauplin!
+
+## 0.9.0
+
+### Features
+
+- [#343](https://github.com/gradio-app/trackio/pull/343) [`51bea30`](https://github.com/gradio-app/trackio/commit/51bea30f2877adff8e6497466d3a799400a0a049) - Sync offline projects to Hugging Face spaces.  Thanks @candemircan!
+- [#341](https://github.com/gradio-app/trackio/pull/341) [`4fd841f`](https://github.com/gradio-app/trackio/commit/4fd841fa190e15071b02f6fba7683ef4f393a654) - Adds a basic UI test to `trackio`.  Thanks @abidlabs!
+- [#339](https://github.com/gradio-app/trackio/pull/339) [`011d91b`](https://github.com/gradio-app/trackio/commit/011d91bb6ae266516fd250a349285670a8049d05) - Allow customzing the trackio color palette.  Thanks @abidlabs!
+
+## 0.8.1
+
+### Features
+
+- [#336](https://github.com/gradio-app/trackio/pull/336) [`5f9f51d`](https://github.com/gradio-app/trackio/commit/5f9f51dac8677f240d7c42c3e3b2660a22aee138) - Support a list of `Trackio.Image` in a `trackio.Table` cell.  Thanks @abidlabs!
+
+## 0.8.0
+
+### Features
+
+- [#331](https://github.com/gradio-app/trackio/pull/331) [`2c02d0f`](https://github.com/gradio-app/trackio/commit/2c02d0fd0a5824160528782402bb0dd4083396d5) - Truncate table string values that are greater than 250 characters (configuirable via env variable).  Thanks @abidlabs!
+- [#324](https://github.com/gradio-app/trackio/pull/324) [`50b2122`](https://github.com/gradio-app/trackio/commit/50b2122e7965ac82a72e6cb3b7d048bc10a2a6b1) - Add log y-axis functionality to UI.  Thanks @abidlabs!
+- [#326](https://github.com/gradio-app/trackio/pull/326) [`61dc1f4`](https://github.com/gradio-app/trackio/commit/61dc1f40af2f545f8e70395ddf0dbb8aee6b60d5) - Fix: improve table rendering for metrics in Trackio Dashboard.  Thanks @vigneshwaran!
+- [#328](https://github.com/gradio-app/trackio/pull/328) [`6857cbb`](https://github.com/gradio-app/trackio/commit/6857cbbe557a59a4642f210ec42566d108294e63) - Support trackio.Table with trackio.Image columns.  Thanks @abidlabs!
+- [#323](https://github.com/gradio-app/trackio/pull/323) [`6857cbb`](https://github.com/gradio-app/trackio/commit/6857cbbe557a59a4642f210ec42566d108294e63) - add Trackio client implementations in Go, Rust, and JS.  Thanks @vaibhav-research!
+
+## 0.7.0
+
+### Features
+
+- [#277](https://github.com/gradio-app/trackio/pull/277) [`db35601`](https://github.com/gradio-app/trackio/commit/db35601b9c023423c4654c9909b8ab73e58737de) - fix: make grouped runs view reflect live updates.  Thanks @Saba9!
+- [#320](https://github.com/gradio-app/trackio/pull/320) [`24ae739`](https://github.com/gradio-app/trackio/commit/24ae73969b09fb3126acd2f91647cdfbf8cf72a1) - Add additional query parms for xmin, xmax, and smoothing.  Thanks @abidlabs!
+- [#270](https://github.com/gradio-app/trackio/pull/270) [`cd1dfc3`](https://github.com/gradio-app/trackio/commit/cd1dfc3dc641b4499ac6d4a1b066fa8e2b52c57b) - feature: add support for logging audio.  Thanks @Saba9!
+
+## 0.6.0
+
+### Features
+
+- [#309](https://github.com/gradio-app/trackio/pull/309) [`1df2353`](https://github.com/gradio-app/trackio/commit/1df23534d6c01938c8db9c0f584ffa23e8d6021d) - Add histogram support with wandb-compatible API.  Thanks @abidlabs!
+- [#315](https://github.com/gradio-app/trackio/pull/315) [`76ba060`](https://github.com/gradio-app/trackio/commit/76ba06055dc43ca8f03b79f3e72d761949bd19a8) - Add guards to avoid silent fails.  Thanks @Xmaster6y!
+- [#313](https://github.com/gradio-app/trackio/pull/313) [`a606b3e`](https://github.com/gradio-app/trackio/commit/a606b3e1c5edf3d4cf9f31bd50605226a5a1c5d0) - No longer prevent certain keys from being used. Instead, dunderify them to prevent collisions with internal usage.  Thanks @abidlabs!
+- [#317](https://github.com/gradio-app/trackio/pull/317) [`27370a5`](https://github.com/gradio-app/trackio/commit/27370a595d0dbdf7eebbe7159d2ba778f039da44) - quick fixes for trackio.histogram.  Thanks @abidlabs!
+- [#312](https://github.com/gradio-app/trackio/pull/312) [`aa0f3bf`](https://github.com/gradio-app/trackio/commit/aa0f3bf372e7a0dd592a38af699c998363830eeb) - Fix video logging by adding TRACKIO_DIR to allowed_paths.  Thanks @abidlabs!
+
+## 0.5.3
+
+### Features
+
+- [#300](https://github.com/gradio-app/trackio/pull/300) [`5e4cacf`](https://github.com/gradio-app/trackio/commit/5e4cacf2e7ce527b4ce60de3a5bc05d2c02c77fb) - Adds more environment variables to allow customization of Trackio dashboard.  Thanks @abidlabs!
+
+## 0.5.2
+
+### Features
+
+- [#293](https://github.com/gradio-app/trackio/pull/293) [`64afc28`](https://github.com/gradio-app/trackio/commit/64afc28d3ea1dfd821472dc6bf0b8ed35a9b74be) - Ensures that the TRACKIO_DIR environment variable is respected.  Thanks @abidlabs!
+- [#287](https://github.com/gradio-app/trackio/pull/287) [`cd3e929`](https://github.com/gradio-app/trackio/commit/cd3e9294320949e6b8b829239069a43d5d7ff4c1) - fix(sqlite): unify .sqlite extension, allow export when DBs exist, clean WAL sidecars on import.  Thanks @vaibhav-research!
+
+### Fixes
+
+- [#291](https://github.com/gradio-app/trackio/pull/291) [`3b5adc3`](https://github.com/gradio-app/trackio/commit/3b5adc3d1f452dbab7a714d235f4974782f93730) - Fix the wheel build.  Thanks @pngwn!
+
+## 0.5.1
+
+### Fixes
+
+- [#278](https://github.com/gradio-app/trackio/pull/278) [`314c054`](https://github.com/gradio-app/trackio/commit/314c05438007ddfea3383e06fd19143e27468e2d) - Fix row orientation of metrics plots.  Thanks @abidlabs!

trackio/__init__.py

@@ -0,0 +1,601 @@
+import atexit
+import glob
+import json
+import logging
+import os
+import shutil
+import warnings
+import webbrowser
+from pathlib import Path
+from typing import Any
+
+import huggingface_hub
+from gradio.themes import ThemeClass
+from gradio.utils import TupleNoPrint
+from gradio_client import Client, handle_file
+from huggingface_hub import SpaceStorage
+from huggingface_hub.errors import LocalTokenNotFoundError
+
+from trackio import context_vars, deploy, utils
+from trackio.api import Api
+from trackio.deploy import sync
+from trackio.gpu import gpu_available, log_gpu
+from trackio.histogram import Histogram
+from trackio.imports import import_csv, import_tf_events
+from trackio.media import (
+    TrackioAudio,
+    TrackioImage,
+    TrackioVideo,
+    get_project_media_path,
+)
+from trackio.run import Run
+from trackio.sqlite_storage import SQLiteStorage
+from trackio.table import Table
+from trackio.typehints import UploadEntry
+from trackio.utils import TRACKIO_DIR, TRACKIO_LOGO_DIR
+
+logging.getLogger("httpx").setLevel(logging.WARNING)
+
+warnings.filterwarnings(
+    "ignore",
+    message="Empty session being created. Install gradio\\[oauth\\]",
+    category=UserWarning,
+    module="gradio.helpers",
+)
+
+__version__ = json.loads(Path(__file__).parent.joinpath("package.json").read_text())[
+    "version"
+]
+
+__all__ = [
+    "init",
+    "log",
+    "log_system",
+    "log_gpu",
+    "finish",
+    "show",
+    "sync",
+    "delete_project",
+    "import_csv",
+    "import_tf_events",
+    "save",
+    "Image",
+    "Video",
+    "Audio",
+    "Table",
+    "Histogram",
+    "Api",
+]
+
+Image = TrackioImage
+Video = TrackioVideo
+Audio = TrackioAudio
+
+
+config = {}
+
+_atexit_registered = False
+
+
+def _cleanup_current_run():
+    run = context_vars.current_run.get()
+    if run is not None:
+        try:
+            run.finish()
+        except Exception:
+            pass
+
+
+def _get_demo():
+    # Lazy import to avoid initializing Gradio Blocks (and FastAPI) at import time,
+    # which causes import lock errors for libraries that just `import trackio`.
+    from trackio.ui.main import CSS, HEAD, demo
+
+    return demo, CSS, HEAD
+
+
+def init(
+    project: str,
+    name: str | None = None,
+    group: str | None = None,
+    space_id: str | None = None,
+    space_storage: SpaceStorage | None = None,
+    dataset_id: str | None = None,
+    config: dict | None = None,
+    resume: str = "never",
+    settings: Any = None,
+    private: bool | None = None,
+    embed: bool = True,
+    auto_log_gpu: bool | None = None,
+    gpu_log_interval: float = 10.0,
+) -> Run:
+    """
+    Creates a new Trackio project and returns a [`Run`] object.
+
+    Args:
+        project (`str`):
+            The name of the project (can be an existing project to continue tracking or
+            a new project to start tracking from scratch).
+        name (`str`, *optional*):
+            The name of the run (if not provided, a default name will be generated).
+        group (`str`, *optional*):
+            The name of the group which this run belongs to in order to help organize
+            related runs together. You can toggle the entire group's visibilitiy in the
+            dashboard.
+        space_id (`str`, *optional*):
+            If provided, the project will be logged to a Hugging Face Space instead of
+            a local directory. Should be a complete Space name like
+            `"username/reponame"` or `"orgname/reponame"`, or just `"reponame"` in which
+            case the Space will be created in the currently-logged-in Hugging Face
+            user's namespace. If the Space does not exist, it will be created. If the
+            Space already exists, the project will be logged to it.
+        space_storage ([`~huggingface_hub.SpaceStorage`], *optional*):
+            Choice of persistent storage tier.
+        dataset_id (`str`, *optional*):
+            If a `space_id` is provided, a persistent Hugging Face Dataset will be
+            created and the metrics will be synced to it every 5 minutes. Specify a
+            Dataset with name like `"username/datasetname"` or `"orgname/datasetname"`,
+            or `"datasetname"` (uses currently-logged-in Hugging Face user's namespace),
+            or `None` (uses the same name as the Space but with the `"_dataset"`
+            suffix). If the Dataset does not exist, it will be created. If the Dataset
+            already exists, the project will be appended to it.
+        config (`dict`, *optional*):
+            A dictionary of configuration options. Provided for compatibility with
+            `wandb.init()`.
+        resume (`str`, *optional*, defaults to `"never"`):
+            Controls how to handle resuming a run. Can be one of:
+
+            - `"must"`: Must resume the run with the given name, raises error if run
+              doesn't exist
+            - `"allow"`: Resume the run if it exists, otherwise create a new run
+            - `"never"`: Never resume a run, always create a new one
+        private (`bool`, *optional*):
+            Whether to make the Space private. If None (default), the repo will be
+            public unless the organization's default is private. This value is ignored
+            if the repo already exists.
+        settings (`Any`, *optional*):
+            Not used. Provided for compatibility with `wandb.init()`.
+        embed (`bool`, *optional*, defaults to `True`):
+            If running inside a jupyter/Colab notebook, whether the dashboard should
+            automatically be embedded in the cell when trackio.init() is called.
+        auto_log_gpu (`bool` or `None`, *optional*, defaults to `None`):
+            Controls automatic GPU metrics logging. If `None` (default), GPU logging
+            is automatically enabled when `nvidia-ml-py` is installed and an NVIDIA
+            GPU is detected. Set to `True` to force enable or `False` to disable.
+        gpu_log_interval (`float`, *optional*, defaults to `10.0`):
+            The interval in seconds between automatic GPU metric logs.
+            Only used when `auto_log_gpu=True`.
+
+    Returns:
+        `Run`: A [`Run`] object that can be used to log metrics and finish the run.
+    """
+    if settings is not None:
+        warnings.warn(
+            "* Warning: settings is not used. Provided for compatibility with wandb.init(). Please create an issue at: https://github.com/gradio-app/trackio/issues if you need a specific feature implemented."
+        )
+
+    if space_id is None and dataset_id is not None:
+        raise ValueError("Must provide a `space_id` when `dataset_id` is provided.")
+    try:
+        space_id, dataset_id = utils.preprocess_space_and_dataset_ids(
+            space_id, dataset_id
+        )
+    except LocalTokenNotFoundError as e:
+        raise LocalTokenNotFoundError(
+            f"You must be logged in to Hugging Face locally when `space_id` is provided to deploy to a Space. {e}"
+        ) from e
+
+    url = context_vars.current_server.get()
+
+    if space_id is not None:
+        if url is None:
+            url = space_id
+            context_vars.current_server.set(url)
+            context_vars.current_space_id.set(space_id)
+
+    if (
+        context_vars.current_project.get() is None
+        or context_vars.current_project.get() != project
+    ):
+        print(f"* Trackio project initialized: {project}")
+
+        if dataset_id is not None:
+            os.environ["TRACKIO_DATASET_ID"] = dataset_id
+            print(
+                f"* Trackio metrics will be synced to Hugging Face Dataset: {dataset_id}"
+            )
+        if space_id is None:
+            print(f"* Trackio metrics logged to: {TRACKIO_DIR}")
+            utils.print_dashboard_instructions(project)
+        else:
+            deploy.create_space_if_not_exists(
+                space_id, space_storage, dataset_id, private
+            )
+            user_name, space_name = space_id.split("/")
+            space_url = deploy.SPACE_HOST_URL.format(
+                user_name=user_name, space_name=space_name
+            )
+            print(f"* View dashboard by going to: {space_url}")
+            if utils.is_in_notebook() and embed:
+                utils.embed_url_in_notebook(space_url)
+    context_vars.current_project.set(project)
+
+    if resume == "must":
+        if name is None:
+            raise ValueError("Must provide a run name when resume='must'")
+        if name not in SQLiteStorage.get_runs(project):
+            raise ValueError(f"Run '{name}' does not exist in project '{project}'")
+        resumed = True
+    elif resume == "allow":
+        resumed = name is not None and name in SQLiteStorage.get_runs(project)
+    elif resume == "never":
+        if name is not None and name in SQLiteStorage.get_runs(project):
+            warnings.warn(
+                f"* Warning: resume='never' but a run '{name}' already exists in "
+                f"project '{project}'. Generating a new name and instead. If you want "
+                "to resume this run, call init() with resume='must' or resume='allow'."
+            )
+            name = None
+        resumed = False
+    else:
+        raise ValueError("resume must be one of: 'must', 'allow', or 'never'")
+
+    if auto_log_gpu is None:
+        auto_log_gpu = gpu_available()
+        if auto_log_gpu:
+            print("* GPU detected, enabling automatic GPU metrics logging")
+
+    run = Run(
+        url=url,
+        project=project,
+        client=None,
+        name=name,
+        group=group,
+        config=config,
+        space_id=space_id,
+        auto_log_gpu=auto_log_gpu,
+        gpu_log_interval=gpu_log_interval,
+    )
+
+    if space_id is not None:
+        SQLiteStorage.set_project_metadata(project, "space_id", space_id)
+        if SQLiteStorage.has_pending_data(project):
+            run._has_local_buffer = True
+
+    global _atexit_registered
+    if not _atexit_registered:
+        atexit.register(_cleanup_current_run)
+        _atexit_registered = True
+
+    if resumed:
+        print(f"* Resumed existing run: {run.name}")
+    else:
+        print(f"* Created new run: {run.name}")
+
+    context_vars.current_run.set(run)
+    globals()["config"] = run.config
+    return run
+
+
+def log(metrics: dict, step: int | None = None) -> None:
+    """
+    Logs metrics to the current run.
+
+    Args:
+        metrics (`dict`):
+            A dictionary of metrics to log.
+        step (`int`, *optional*):
+            The step number. If not provided, the step will be incremented
+            automatically.
+    """
+    run = context_vars.current_run.get()
+    if run is None:
+        raise RuntimeError("Call trackio.init() before trackio.log().")
+    run.log(
+        metrics=metrics,
+        step=step,
+    )
+
+
+def log_system(metrics: dict) -> None:
+    """
+    Logs system metrics (GPU, etc.) to the current run using timestamps instead of steps.
+
+    Args:
+        metrics (`dict`):
+            A dictionary of system metrics to log.
+    """
+    run = context_vars.current_run.get()
+    if run is None:
+        raise RuntimeError("Call trackio.init() before trackio.log_system().")
+    run.log_system(metrics=metrics)
+
+
+def finish():
+    """
+    Finishes the current run.
+    """
+    run = context_vars.current_run.get()
+    if run is None:
+        raise RuntimeError("Call trackio.init() before trackio.finish().")
+    run.finish()
+
+
+def delete_project(project: str, force: bool = False) -> bool:
+    """
+    Deletes a project by removing its local SQLite database.
+
+    Args:
+        project (`str`):
+            The name of the project to delete.
+        force (`bool`, *optional*, defaults to `False`):
+            If `True`, deletes the project without prompting for confirmation.
+            If `False`, prompts the user to confirm before deleting.
+
+    Returns:
+        `bool`: `True` if the project was deleted, `False` otherwise.
+    """
+    db_path = SQLiteStorage.get_project_db_path(project)
+
+    if not db_path.exists():
+        print(f"* Project '{project}' does not exist.")
+        return False
+
+    if not force:
+        response = input(
+            f"Are you sure you want to delete project '{project}'? "
+            f"This will permanently delete all runs and metrics. (y/N): "
+        )
+        if response.lower() not in ["y", "yes"]:
+            print("* Deletion cancelled.")
+            return False
+
+    try:
+        db_path.unlink()
+
+        for suffix in ("-wal", "-shm"):
+            sidecar = Path(str(db_path) + suffix)
+            if sidecar.exists():
+                sidecar.unlink()
+
+        print(f"* Project '{project}' has been deleted.")
+        return True
+    except Exception as e:
+        print(f"* Error deleting project '{project}': {e}")
+        return False
+
+
+def save(
+    glob_str: str | Path,
+    project: str | None = None,
+) -> str:
+    """
+    Saves files to a project (not linked to a specific run). If Trackio is running
+    locally, the file(s) will be copied to the project's files directory. If Trackio is
+    running in a Space, the file(s) will be uploaded to the Space's files directory.
+
+    Args:
+        glob_str (`str` or `Path`):
+            The file path or glob pattern to save. Can be a single file or a pattern
+            matching multiple files (e.g., `"*.py"`, `"models/**/*.pth"`).
+        project (`str`, *optional*):
+            The name of the project to save files to. If not provided, uses the current
+            project from `trackio.init()`. If no project is initialized, raises an
+            error.
+
+    Returns:
+        `str`: The path where the file(s) were saved (project's files directory).
+
+    Example:
+        ```python
+        import trackio
+
+        trackio.init(project="my-project")
+        trackio.save("config.yaml")
+        trackio.save("models/*.pth")
+        ```
+    """
+    if project is None:
+        project = context_vars.current_project.get()
+        if project is None:
+            raise RuntimeError(
+                "No project specified. Either call trackio.init() first or provide a "
+                "project parameter to trackio.save()."
+            )
+
+    glob_str = Path(glob_str)
+    base_path = Path.cwd().resolve()
+
+    matched_files = []
+    if glob_str.is_file():
+        matched_files = [glob_str.resolve()]
+    else:
+        pattern = str(glob_str)
+        if not glob_str.is_absolute():
+            pattern = str((Path.cwd() / glob_str).resolve())
+        matched_files = [
+            Path(f).resolve()
+            for f in glob.glob(pattern, recursive=True)
+            if Path(f).is_file()
+        ]
+
+    if not matched_files:
+        raise ValueError(f"No files found matching pattern: {glob_str}")
+
+    current_run = context_vars.current_run.get()
+    is_local = (
+        current_run._is_local
+        if current_run is not None
+        else (context_vars.current_space_id.get() is None)
+    )
+
+    if is_local:
+        for file_path in matched_files:
+            try:
+                relative_to_base = file_path.relative_to(base_path)
+            except ValueError:
+                relative_to_base = Path(file_path.name)
+
+            if current_run is not None:
+                current_run._queue_upload(
+                    file_path,
+                    step=None,
+                    relative_path=str(relative_to_base.parent),
+                    use_run_name=False,
+                )
+            else:
+                media_path = get_project_media_path(
+                    project=project,
+                    run=None,
+                    step=None,
+                    relative_path=str(relative_to_base),
+                )
+                shutil.copy(str(file_path), str(media_path))
+    else:
+        url = context_vars.current_server.get()
+
+        upload_entries = []
+        for file_path in matched_files:
+            try:
+                relative_to_base = file_path.relative_to(base_path)
+            except ValueError:
+                relative_to_base = Path(file_path.name)
+
+            if current_run is not None:
+                current_run._queue_upload(
+                    file_path,
+                    step=None,
+                    relative_path=str(relative_to_base.parent),
+                    use_run_name=False,
+                )
+            else:
+                upload_entry: UploadEntry = {
+                    "project": project,
+                    "run": None,
+                    "step": None,
+                    "relative_path": str(relative_to_base),
+                    "uploaded_file": handle_file(file_path),
+                }
+                upload_entries.append(upload_entry)
+
+        if upload_entries:
+            if url is None:
+                raise RuntimeError(
+                    "No server available. Call trackio.init() before trackio.save() to start the server."
+                )
+
+            try:
+                client = Client(url, verbose=False, httpx_kwargs={"timeout": 90})
+                client.predict(
+                    api_name="/bulk_upload_media",
+                    uploads=upload_entries,
+                    hf_token=huggingface_hub.utils.get_token(),
+                )
+            except Exception as e:
+                warnings.warn(
+                    f"Failed to upload files: {e}. "
+                    "Files may not be available in the dashboard."
+                )
+
+    return str(utils.MEDIA_DIR / project / "files")
+
+
+def show(
+    project: str | None = None,
+    *,
+    theme: str | ThemeClass | None = None,
+    mcp_server: bool | None = None,
+    footer: bool = True,
+    color_palette: list[str] | None = None,
+    open_browser: bool = True,
+    block_thread: bool | None = None,
+    host: str | None = None,
+):
+    """
+    Launches the Trackio dashboard.
+
+    Args:
+        project (`str`, *optional*):
+            The name of the project whose runs to show. If not provided, all projects
+            will be shown and the user can select one.
+        theme (`str` or `ThemeClass`, *optional*):
+            A Gradio Theme to use for the dashboard instead of the default Gradio theme,
+            can be a built-in theme (e.g. `'soft'`, `'citrus'`), a theme from the Hub
+            (e.g. `"gstaff/xkcd"`), or a custom Theme class. If not provided, the
+            `TRACKIO_THEME` environment variable will be used, or if that is not set,
+            the default Gradio theme will be used.
+        mcp_server (`bool`, *optional*):
+            If `True`, the Trackio dashboard will be set up as an MCP server and certain
+            functions will be added as MCP tools. If `None` (default behavior), then the
+            `GRADIO_MCP_SERVER` environment variable will be used to determine if the
+            MCP server should be enabled (which is `"True"` on Hugging Face Spaces).
+        footer (`bool`, *optional*, defaults to `True`):
+            Whether to show the Gradio footer. When `False`, the footer will be hidden.
+            This can also be controlled via the `footer` query parameter in the URL.
+        color_palette (`list[str]`, *optional*):
+            A list of hex color codes to use for plot lines. If not provided, the
+            `TRACKIO_COLOR_PALETTE` environment variable will be used (comma-separated
+            hex codes), or if that is not set, the default color palette will be used.
+            Example: `['#FF0000', '#00FF00', '#0000FF']`
+        open_browser (`bool`, *optional*, defaults to `True`):
+            If `True` and not in a notebook, a new browser tab will be opened with the
+            dashboard. If `False`, the browser will not be opened.
+        block_thread (`bool`, *optional*):
+            If `True`, the main thread will be blocked until the dashboard is closed.
+            If `None` (default behavior), then the main thread will not be blocked if the
+            dashboard is launched in a notebook, otherwise the main thread will be blocked.
+        host (`str`, *optional*):
+            The host to bind the server to. If not provided, defaults to `'127.0.0.1'`
+            (localhost only). Set to `'0.0.0.0'` to allow remote access.
+
+        Returns:
+            `app`: The Gradio app object corresponding to the dashboard launched by Trackio.
+            `url`: The local URL of the dashboard.
+            `share_url`: The public share URL of the dashboard.
+            `full_url`: The full URL of the dashboard including the write token (will use the public share URL if launched publicly, otherwise the local URL).
+    """
+    demo, CSS, HEAD = _get_demo()
+
+    if color_palette is not None:
+        os.environ["TRACKIO_COLOR_PALETTE"] = ",".join(color_palette)
+
+    theme = theme or os.environ.get("TRACKIO_THEME")
+
+    _mcp_server = (
+        mcp_server
+        if mcp_server is not None
+        else os.environ.get("GRADIO_MCP_SERVER", "False") == "True"
+    )
+
+    app, url, share_url = demo.launch(
+        css=CSS,
+        head=HEAD,
+        footer_links=["gradio", "settings"] + (["api"] if _mcp_server else []),
+        quiet=True,
+        inline=False,
+        prevent_thread_lock=True,
+        favicon_path=TRACKIO_LOGO_DIR / "trackio_logo_light.png",
+        allowed_paths=[TRACKIO_LOGO_DIR, TRACKIO_DIR],
+        mcp_server=_mcp_server,
+        theme=theme,
+        ssr_mode=False,
+        server_name=host,
+    )
+
+    base_url = share_url + "/" if share_url else url
+    full_url = utils.get_full_url(
+        base_url, project=project, write_token=demo.write_token, footer=footer
+    )
+
+    if not utils.is_in_notebook():
+        print(f"* Trackio UI launched at: {full_url}")
+        if open_browser:
+            webbrowser.open(full_url)
+        block_thread = block_thread if block_thread is not None else True
+    else:
+        utils.embed_url_in_notebook(full_url)
+        block_thread = block_thread if block_thread is not None else False
+
+    if block_thread:
+        utils.block_main_thread_until_keyboard_interrupt()
+    return TupleNoPrint((demo, url, share_url, full_url))

trackio/api.py

@@ -0,0 +1,66 @@
+from typing import Iterator
+
+from trackio.sqlite_storage import SQLiteStorage
+
+
+class Run:
+    def __init__(self, project: str, name: str):
+        self.project = project
+        self.name = name
+        self._config = None
+
+    @property
+    def id(self) -> str:
+        return self.name
+
+    @property
+    def config(self) -> dict | None:
+        if self._config is None:
+            self._config = SQLiteStorage.get_run_config(self.project, self.name)
+        return self._config
+
+    def delete(self) -> bool:
+        return SQLiteStorage.delete_run(self.project, self.name)
+
+    def move(self, new_project: str) -> bool:
+        success = SQLiteStorage.move_run(self.project, self.name, new_project)
+        if success:
+            self.project = new_project
+        return success
+
+    def __repr__(self) -> str:
+        return f"<Run {self.name} in project {self.project}>"
+
+
+class Runs:
+    def __init__(self, project: str):
+        self.project = project
+        self._runs = None
+
+    def _load_runs(self):
+        if self._runs is None:
+            run_names = SQLiteStorage.get_runs(self.project)
+            self._runs = [Run(self.project, name) for name in run_names]
+
+    def __iter__(self) -> Iterator[Run]:
+        self._load_runs()
+        return iter(self._runs)
+
+    def __getitem__(self, index: int) -> Run:
+        self._load_runs()
+        return self._runs[index]
+
+    def __len__(self) -> int:
+        self._load_runs()
+        return len(self._runs)
+
+    def __repr__(self) -> str:
+        self._load_runs()
+        return f"<Runs project={self.project} count={len(self._runs)}>"
+
+
+class Api:
+    def runs(self, project: str) -> Runs:
+        if not SQLiteStorage.get_project_db_path(project).exists():
+            raise ValueError(f"Project '{project}' does not exist")
+        return Runs(project)

trackio/cli.py

@@ -0,0 +1,514 @@
+import argparse
+
+from trackio import show, sync
+from trackio.cli_helpers import (
+    error_exit,
+    format_json,
+    format_list,
+    format_metric_values,
+    format_project_summary,
+    format_run_summary,
+    format_system_metric_names,
+    format_system_metrics,
+)
+from trackio.sqlite_storage import SQLiteStorage
+from trackio.ui.main import get_project_summary, get_run_summary
+
+
+def _handle_status():
+    print("Reading local Trackio projects...\n")
+    projects = SQLiteStorage.get_projects()
+    if not projects:
+        print("No Trackio projects found.")
+        return
+
+    local_projects = []
+    synced_projects = []
+    unsynced_projects = []
+
+    for project in projects:
+        space_id = SQLiteStorage.get_space_id(project)
+        if space_id is None:
+            local_projects.append(project)
+        elif SQLiteStorage.has_pending_data(project):
+            unsynced_projects.append(project)
+        else:
+            synced_projects.append(project)
+
+    print("Finished reading Trackio projects")
+    if local_projects:
+        print(f"  * {len(local_projects)} local trackio project(s) [OK]")
+    if synced_projects:
+        print(f"  * {len(synced_projects)} trackio project(s) synced to Spaces [OK]")
+    if unsynced_projects:
+        print(
+            f"  * {len(unsynced_projects)} trackio project(s) with unsynced changes [WARNING]:"
+        )
+        for p in unsynced_projects:
+            print(f"    - {p}")
+
+    if unsynced_projects:
+        print(
+            f"\nRun `trackio sync --project {unsynced_projects[0]}` to sync. "
+            "Or run `trackio sync --all` to sync all unsynced changes."
+        )
+
+
+def _handle_sync(args):
+    from trackio.deploy import sync_incremental
+
+    if args.sync_all and args.project:
+        error_exit("Cannot use --all and --project together.")
+    if not args.sync_all and not args.project:
+        error_exit("Must provide either --project or --all.")
+
+    if args.sync_all:
+        projects = SQLiteStorage.get_projects()
+        synced_any = False
+        for project in projects:
+            space_id = SQLiteStorage.get_space_id(project)
+            if space_id and SQLiteStorage.has_pending_data(project):
+                sync_incremental(
+                    project, space_id, private=args.private, pending_only=True
+                )
+                synced_any = True
+        if not synced_any:
+            print("No projects with unsynced data found.")
+    else:
+        space_id = args.space_id
+        if space_id is None:
+            space_id = SQLiteStorage.get_space_id(args.project)
+        sync(
+            project=args.project,
+            space_id=space_id,
+            private=args.private,
+            force=args.force,
+        )
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Trackio CLI")
+    subparsers = parser.add_subparsers(dest="command")
+
+    ui_parser = subparsers.add_parser(
+        "show", help="Show the Trackio dashboard UI for a project"
+    )
+    ui_parser.add_argument(
+        "--project", required=False, help="Project name to show in the dashboard"
+    )
+    ui_parser.add_argument(
+        "--theme",
+        required=False,
+        default="default",
+        help="A Gradio Theme to use for the dashboard instead of the default, can be a built-in theme (e.g. 'soft', 'citrus'), or a theme from the Hub (e.g. 'gstaff/xkcd').",
+    )
+    ui_parser.add_argument(
+        "--mcp-server",
+        action="store_true",
+        help="Enable MCP server functionality. The Trackio dashboard will be set up as an MCP server and certain functions will be exposed as MCP tools.",
+    )
+    ui_parser.add_argument(
+        "--footer",
+        action="store_true",
+        default=True,
+        help="Show the Gradio footer. Use --no-footer to hide it.",
+    )
+    ui_parser.add_argument(
+        "--no-footer",
+        dest="footer",
+        action="store_false",
+        help="Hide the Gradio footer.",
+    )
+    ui_parser.add_argument(
+        "--color-palette",
+        required=False,
+        help="Comma-separated list of hex color codes for plot lines (e.g. '#FF0000,#00FF00,#0000FF'). If not provided, the TRACKIO_COLOR_PALETTE environment variable will be used, or the default palette if not set.",
+    )
+    ui_parser.add_argument(
+        "--host",
+        required=False,
+        help="Host to bind the server to (e.g. '0.0.0.0' for remote access). If not provided, defaults to '127.0.0.1' (localhost only).",
+    )
+
+    subparsers.add_parser(
+        "status",
+        help="Show the status of all local Trackio projects, including sync status.",
+    )
+
+    sync_parser = subparsers.add_parser(
+        "sync",
+        help="Sync a local project's database to a Hugging Face Space. If the Space does not exist, it will be created.",
+    )
+    sync_parser.add_argument(
+        "--project",
+        required=False,
+        help="The name of the local project.",
+    )
+    sync_parser.add_argument(
+        "--space-id",
+        required=False,
+        help="The Hugging Face Space ID where the project will be synced (e.g. username/space_id). If not provided, uses the previously-configured Space.",
+    )
+    sync_parser.add_argument(
+        "--all",
+        action="store_true",
+        dest="sync_all",
+        help="Sync all projects that have unsynced data to their configured Spaces.",
+    )
+    sync_parser.add_argument(
+        "--private",
+        action="store_true",
+        help="Make the Hugging Face Space private if creating a new Space. By default, the repo will be public unless the organization's default is private. This value is ignored if the repo already exists.",
+    )
+    sync_parser.add_argument(
+        "--force",
+        action="store_true",
+        help="Overwrite the existing database without prompting for confirmation.",
+    )
+
+    list_parser = subparsers.add_parser(
+        "list",
+        help="List projects, runs, or metrics",
+    )
+    list_subparsers = list_parser.add_subparsers(dest="list_type", required=True)
+
+    list_projects_parser = list_subparsers.add_parser(
+        "projects",
+        help="List all projects",
+    )
+    list_projects_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    list_runs_parser = list_subparsers.add_parser(
+        "runs",
+        help="List runs for a project",
+    )
+    list_runs_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    list_runs_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    list_metrics_parser = list_subparsers.add_parser(
+        "metrics",
+        help="List metrics for a run",
+    )
+    list_metrics_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    list_metrics_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    list_metrics_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    list_system_metrics_parser = list_subparsers.add_parser(
+        "system-metrics",
+        help="List system metrics for a run",
+    )
+    list_system_metrics_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    list_system_metrics_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    list_system_metrics_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_parser = subparsers.add_parser(
+        "get",
+        help="Get project, run, or metric information",
+    )
+    get_subparsers = get_parser.add_subparsers(dest="get_type", required=True)
+
+    get_project_parser = get_subparsers.add_parser(
+        "project",
+        help="Get project summary",
+    )
+    get_project_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_project_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_run_parser = get_subparsers.add_parser(
+        "run",
+        help="Get run summary",
+    )
+    get_run_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_run_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    get_run_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_metric_parser = get_subparsers.add_parser(
+        "metric",
+        help="Get metric values for a run",
+    )
+    get_metric_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_metric_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    get_metric_parser.add_argument(
+        "--metric",
+        required=True,
+        help="Metric name",
+    )
+    get_metric_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    get_system_metric_parser = get_subparsers.add_parser(
+        "system-metric",
+        help="Get system metric values for a run",
+    )
+    get_system_metric_parser.add_argument(
+        "--project",
+        required=True,
+        help="Project name",
+    )
+    get_system_metric_parser.add_argument(
+        "--run",
+        required=True,
+        help="Run name",
+    )
+    get_system_metric_parser.add_argument(
+        "--metric",
+        required=False,
+        help="System metric name (optional, if not provided returns all system metrics)",
+    )
+    get_system_metric_parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output in JSON format",
+    )
+
+    args = parser.parse_args()
+
+    if args.command == "show":
+        color_palette = None
+        if args.color_palette:
+            color_palette = [color.strip() for color in args.color_palette.split(",")]
+        show(
+            project=args.project,
+            theme=args.theme,
+            mcp_server=args.mcp_server,
+            footer=args.footer,
+            color_palette=color_palette,
+            host=args.host,
+        )
+    elif args.command == "status":
+        _handle_status()
+    elif args.command == "sync":
+        _handle_sync(args)
+    elif args.command == "list":
+        if args.list_type == "projects":
+            projects = SQLiteStorage.get_projects()
+            if args.json:
+                print(format_json({"projects": projects}))
+            else:
+                print(format_list(projects, "Projects"))
+        elif args.list_type == "runs":
+            db_path = SQLiteStorage.get_project_db_path(args.project)
+            if not db_path.exists():
+                error_exit(f"Project '{args.project}' not found.")
+            runs = SQLiteStorage.get_runs(args.project)
+            if args.json:
+                print(format_json({"project": args.project, "runs": runs}))
+            else:
+                print(format_list(runs, f"Runs in '{args.project}'"))
+        elif args.list_type == "metrics":
+            db_path = SQLiteStorage.get_project_db_path(args.project)
+            if not db_path.exists():
+                error_exit(f"Project '{args.project}' not found.")
+            runs = SQLiteStorage.get_runs(args.project)
+            if args.run not in runs:
+                error_exit(f"Run '{args.run}' not found in project '{args.project}'.")
+            metrics = SQLiteStorage.get_all_metrics_for_run(args.project, args.run)
+            if args.json:
+                print(
+                    format_json(
+                        {"project": args.project, "run": args.run, "metrics": metrics}
+                    )
+                )
+            else:
+                print(
+                    format_list(
+                        metrics, f"Metrics for '{args.run}' in '{args.project}'"
+                    )
+                )
+        elif args.list_type == "system-metrics":
+            db_path = SQLiteStorage.get_project_db_path(args.project)
+            if not db_path.exists():
+                error_exit(f"Project '{args.project}' not found.")
+            runs = SQLiteStorage.get_runs(args.project)
+            if args.run not in runs:
+                error_exit(f"Run '{args.run}' not found in project '{args.project}'.")
+            system_metrics = SQLiteStorage.get_all_system_metrics_for_run(
+                args.project, args.run
+            )
+            if args.json:
+                print(
+                    format_json(
+                        {
+                            "project": args.project,
+                            "run": args.run,
+                            "system_metrics": system_metrics,
+                        }
+                    )
+                )
+            else:
+                print(format_system_metric_names(system_metrics))
+    elif args.command == "get":
+        if args.get_type == "project":
+            db_path = SQLiteStorage.get_project_db_path(args.project)
+            if not db_path.exists():
+                error_exit(f"Project '{args.project}' not found.")
+            summary = get_project_summary(args.project)
+            if args.json:
+                print(format_json(summary))
+            else:
+                print(format_project_summary(summary))
+        elif args.get_type == "run":
+            db_path = SQLiteStorage.get_project_db_path(args.project)
+            if not db_path.exists():
+                error_exit(f"Project '{args.project}' not found.")
+            runs = SQLiteStorage.get_runs(args.project)
+            if args.run not in runs:
+                error_exit(f"Run '{args.run}' not found in project '{args.project}'.")
+            summary = get_run_summary(args.project, args.run)
+            if args.json:
+                print(format_json(summary))
+            else:
+                print(format_run_summary(summary))
+        elif args.get_type == "metric":
+            db_path = SQLiteStorage.get_project_db_path(args.project)
+            if not db_path.exists():
+                error_exit(f"Project '{args.project}' not found.")
+            runs = SQLiteStorage.get_runs(args.project)
+            if args.run not in runs:
+                error_exit(f"Run '{args.run}' not found in project '{args.project}'.")
+            metrics = SQLiteStorage.get_all_metrics_for_run(args.project, args.run)
+            if args.metric not in metrics:
+                error_exit(
+                    f"Metric '{args.metric}' not found in run '{args.run}' of project '{args.project}'."
+                )
+            values = SQLiteStorage.get_metric_values(
+                args.project, args.run, args.metric
+            )
+            if args.json:
+                print(
+                    format_json(
+                        {
+                            "project": args.project,
+                            "run": args.run,
+                            "metric": args.metric,
+                            "values": values,
+                        }
+                    )
+                )
+            else:
+                print(format_metric_values(values))
+        elif args.get_type == "system-metric":
+            db_path = SQLiteStorage.get_project_db_path(args.project)
+            if not db_path.exists():
+                error_exit(f"Project '{args.project}' not found.")
+            runs = SQLiteStorage.get_runs(args.project)
+            if args.run not in runs:
+                error_exit(f"Run '{args.run}' not found in project '{args.project}'.")
+            if args.metric:
+                system_metrics = SQLiteStorage.get_system_logs(args.project, args.run)
+                all_system_metric_names = SQLiteStorage.get_all_system_metrics_for_run(
+                    args.project, args.run
+                )
+                if args.metric not in all_system_metric_names:
+                    error_exit(
+                        f"System metric '{args.metric}' not found in run '{args.run}' of project '{args.project}'."
+                    )
+                filtered_metrics = [
+                    {
+                        k: v
+                        for k, v in entry.items()
+                        if k == "timestamp" or k == args.metric
+                    }
+                    for entry in system_metrics
+                    if args.metric in entry
+                ]
+                if args.json:
+                    print(
+                        format_json(
+                            {
+                                "project": args.project,
+                                "run": args.run,
+                                "metric": args.metric,
+                                "values": filtered_metrics,
+                            }
+                        )
+                    )
+                else:
+                    print(format_system_metrics(filtered_metrics))
+            else:
+                system_metrics = SQLiteStorage.get_system_logs(args.project, args.run)
+                if args.json:
+                    print(
+                        format_json(
+                            {
+                                "project": args.project,
+                                "run": args.run,
+                                "system_metrics": system_metrics,
+                            }
+                        )
+                    )
+                else:
+                    print(format_system_metrics(system_metrics))
+    else:
+        parser.print_help()
+
+
+if __name__ == "__main__":
+    main()

trackio/cli_helpers.py

@@ -0,0 +1,118 @@
+import json
+import sys
+from typing import Any
+
+
+def format_json(data: Any) -> str:
+    """Format data as JSON."""
+    return json.dumps(data, indent=2)
+
+
+def format_list(items: list[str], title: str | None = None) -> str:
+    """Format a list of items in human-readable format."""
+    if not items:
+        return f"No {title.lower() if title else 'items'} found."
+
+    output = []
+    if title:
+        output.append(f"{title}:")
+
+    for item in items:
+        output.append(f"  - {item}")
+
+    return "\n".join(output)
+
+
+def format_project_summary(summary: dict) -> str:
+    """Format project summary in human-readable format."""
+    output = [f"Project: {summary['project']}"]
+    output.append(f"Number of runs: {summary['num_runs']}")
+
+    if summary["runs"]:
+        output.append("\nRuns:")
+        for run in summary["runs"]:
+            output.append(f"  - {run}")
+    else:
+        output.append("\nNo runs found.")
+
+    if summary.get("last_activity"):
+        output.append(f"\nLast activity (max step): {summary['last_activity']}")
+
+    return "\n".join(output)
+
+
+def format_run_summary(summary: dict) -> str:
+    """Format run summary in human-readable format."""
+    output = [f"Project: {summary['project']}"]
+    output.append(f"Run: {summary['run']}")
+    output.append(f"Number of logs: {summary['num_logs']}")
+
+    if summary.get("last_step") is not None:
+        output.append(f"Last step: {summary['last_step']}")
+
+    if summary.get("metrics"):
+        output.append("\nMetrics:")
+        for metric in summary["metrics"]:
+            output.append(f"  - {metric}")
+    else:
+        output.append("\nNo metrics found.")
+
+    config = summary.get("config")
+    if config:
+        output.append("\nConfig:")
+        config_display = {k: v for k, v in config.items() if not k.startswith("_")}
+        if config_display:
+            for key, value in config_display.items():
+                output.append(f"  {key}: {value}")
+        else:
+            output.append("  (no config)")
+    else:
+        output.append("\nConfig: (no config)")
+
+    return "\n".join(output)
+
+
+def format_metric_values(values: list[dict]) -> str:
+    """Format metric values in human-readable format."""
+    if not values:
+        return "No metric values found."
+
+    output = [f"Found {len(values)} value(s):\n"]
+    output.append("Step | Timestamp | Value")
+    output.append("-" * 50)
+
+    for value in values:
+        step = value.get("step", "N/A")
+        timestamp = value.get("timestamp", "N/A")
+        val = value.get("value", "N/A")
+        output.append(f"{step} | {timestamp} | {val}")
+
+    return "\n".join(output)
+
+
+def format_system_metrics(metrics: list[dict]) -> str:
+    """Format system metrics in human-readable format."""
+    if not metrics:
+        return "No system metrics found."
+
+    output = [f"Found {len(metrics)} system metric entry/entries:\n"]
+
+    for i, entry in enumerate(metrics):
+        timestamp = entry.get("timestamp", "N/A")
+        output.append(f"\nEntry {i + 1} (Timestamp: {timestamp}):")
+        for key, value in entry.items():
+            if key != "timestamp":
+                output.append(f"  {key}: {value}")
+
+    return "\n".join(output)
+
+
+def format_system_metric_names(names: list[str]) -> str:
+    """Format system metric names in human-readable format."""
+    return format_list(names, "System Metrics")
+
+
+def error_exit(message: str, code: int = 1) -> None:
+    """Print error message and exit."""
+    print(f"Error: {message}", file=sys.stderr)
+    sys.exit(code)

trackio/commit_scheduler.py

@@ -0,0 +1,310 @@
+# Originally copied from https://github.com/huggingface/huggingface_hub/blob/d0a948fc2a32ed6e557042a95ef3e4af97ec4a7c/src/huggingface_hub/_commit_scheduler.py
+
+import atexit
+import logging
+import time
+from concurrent.futures import Future
+from dataclasses import dataclass
+from pathlib import Path
+from threading import Lock, Thread
+from typing import Callable, Dict, List, Union
+
+from huggingface_hub.hf_api import (
+    DEFAULT_IGNORE_PATTERNS,
+    CommitInfo,
+    CommitOperationAdd,
+    HfApi,
+)
+from huggingface_hub.utils import filter_repo_objects
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass(frozen=True)
+class _FileToUpload:
+    """Temporary dataclass to store info about files to upload. Not meant to be used directly."""
+
+    local_path: Path
+    path_in_repo: str
+    size_limit: int
+    last_modified: float
+
+
+class CommitScheduler:
+    """
+    Scheduler to upload a local folder to the Hub at regular intervals (e.g. push to hub every 5 minutes).
+
+    The recommended way to use the scheduler is to use it as a context manager. This ensures that the scheduler is
+    properly stopped and the last commit is triggered when the script ends. The scheduler can also be stopped manually
+    with the `stop` method. Checkout the [upload guide](https://huggingface.co/docs/huggingface_hub/guides/upload#scheduled-uploads)
+    to learn more about how to use it.
+
+    Args:
+        repo_id (`str`):
+            The id of the repo to commit to.
+        folder_path (`str` or `Path`):
+            Path to the local folder to upload regularly.
+        every (`int` or `float`, *optional*):
+            The number of minutes between each commit. Defaults to 5 minutes.
+        path_in_repo (`str`, *optional*):
+            Relative path of the directory in the repo, for example: `"checkpoints/"`. Defaults to the root folder
+            of the repository.
+        repo_type (`str`, *optional*):
+            The type of the repo to commit to. Defaults to `model`.
+        revision (`str`, *optional*):
+            The revision of the repo to commit to. Defaults to `main`.
+        private (`bool`, *optional*):
+            Whether to make the repo private. If `None` (default), the repo will be public unless the organization's default is private. This value is ignored if the repo already exists.
+        token (`str`, *optional*):
+            The token to use to commit to the repo. Defaults to the token saved on the machine.
+        allow_patterns (`List[str]` or `str`, *optional*):
+            If provided, only files matching at least one pattern are uploaded.
+        ignore_patterns (`List[str]` or `str`, *optional*):
+            If provided, files matching any of the patterns are not uploaded.
+        squash_history (`bool`, *optional*):
+            Whether to squash the history of the repo after each commit. Defaults to `False`. Squashing commits is
+            useful to avoid degraded performances on the repo when it grows too large.
+        hf_api (`HfApi`, *optional*):
+            The [`HfApi`] client to use to commit to the Hub. Can be set with custom settings (user agent, token,...).
+        on_before_commit (`Callable[[], None]`, *optional*):
+            If specified, a function that will be called before the CommitScheduler lists files to create a commit.
+
+    Example:
+    ```py
+    >>> from pathlib import Path
+    >>> from huggingface_hub import CommitScheduler
+
+    # Scheduler uploads every 10 minutes
+    >>> csv_path = Path("watched_folder/data.csv")
+    >>> CommitScheduler(repo_id="test_scheduler", repo_type="dataset", folder_path=csv_path.parent, every=10)
+
+    >>> with csv_path.open("a") as f:
+    ...     f.write("first line")
+
+    # Some time later (...)
+    >>> with csv_path.open("a") as f:
+    ...     f.write("second line")
+    ```
+
+    Example using a context manager:
+    ```py
+    >>> from pathlib import Path
+    >>> from huggingface_hub import CommitScheduler
+
+    >>> with CommitScheduler(repo_id="test_scheduler", repo_type="dataset", folder_path="watched_folder", every=10) as scheduler:
+    ...     csv_path = Path("watched_folder/data.csv")
+    ...     with csv_path.open("a") as f:
+    ...         f.write("first line")
+    ...     (...)
+    ...     with csv_path.open("a") as f:
+    ...         f.write("second line")
+
+    # Scheduler is now stopped and last commit have been triggered
+    ```
+    """
+
+    def __init__(
+        self,
+        *,
+        repo_id: str,
+        folder_path: Union[str, Path],
+        every: Union[int, float] = 5,
+        path_in_repo: str | None = None,
+        repo_type: str | None = None,
+        revision: str | None = None,
+        private: bool | None = None,
+        token: str | None = None,
+        allow_patterns: list[str] | str | None = None,
+        ignore_patterns: list[str] | str | None = None,
+        squash_history: bool = False,
+        hf_api: HfApi | None = None,
+        on_before_commit: Callable[[], None] | None = None,
+    ) -> None:
+        self.api = hf_api or HfApi(token=token)
+        self.on_before_commit = on_before_commit
+
+        # Folder
+        self.folder_path = Path(folder_path).expanduser().resolve()
+        self.path_in_repo = path_in_repo or ""
+        self.allow_patterns = allow_patterns
+
+        if ignore_patterns is None:
+            ignore_patterns = []
+        elif isinstance(ignore_patterns, str):
+            ignore_patterns = [ignore_patterns]
+        self.ignore_patterns = ignore_patterns + DEFAULT_IGNORE_PATTERNS
+
+        if self.folder_path.is_file():
+            raise ValueError(
+                f"'folder_path' must be a directory, not a file: '{self.folder_path}'."
+            )
+        self.folder_path.mkdir(parents=True, exist_ok=True)
+
+        # Repository
+        repo_url = self.api.create_repo(
+            repo_id=repo_id, private=private, repo_type=repo_type, exist_ok=True
+        )
+        self.repo_id = repo_url.repo_id
+        self.repo_type = repo_type
+        self.revision = revision
+        self.token = token
+
+        self.last_uploaded: Dict[Path, float] = {}
+        self.last_push_time: float | None = None
+
+        if not every > 0:
+            raise ValueError(f"'every' must be a positive integer, not '{every}'.")
+        self.lock = Lock()
+        self.every = every
+        self.squash_history = squash_history
+
+        logger.info(
+            f"Scheduled job to push '{self.folder_path}' to '{self.repo_id}' every {self.every} minutes."
+        )
+        self._scheduler_thread = Thread(target=self._run_scheduler, daemon=True)
+        self._scheduler_thread.start()
+        atexit.register(self._push_to_hub)
+
+        self.__stopped = False
+
+    def stop(self) -> None:
+        """Stop the scheduler.
+
+        A stopped scheduler cannot be restarted. Mostly for tests purposes.
+        """
+        self.__stopped = True
+
+    def __enter__(self) -> "CommitScheduler":
+        return self
+
+    def __exit__(self, exc_type, exc_value, traceback) -> None:
+        # Upload last changes before exiting
+        self.trigger().result()
+        self.stop()
+        return
+
+    def _run_scheduler(self) -> None:
+        """Dumb thread waiting between each scheduled push to Hub."""
+        while True:
+            self.last_future = self.trigger()
+            time.sleep(self.every * 60)
+            if self.__stopped:
+                break
+
+    def trigger(self) -> Future:
+        """Trigger a `push_to_hub` and return a future.
+
+        This method is automatically called every `every` minutes. You can also call it manually to trigger a commit
+        immediately, without waiting for the next scheduled commit.
+        """
+        return self.api.run_as_future(self._push_to_hub)
+
+    def _push_to_hub(self) -> CommitInfo | None:
+        if self.__stopped:  # If stopped, already scheduled commits are ignored
+            return None
+
+        logger.info("(Background) scheduled commit triggered.")
+        try:
+            value = self.push_to_hub()
+            if self.squash_history:
+                logger.info("(Background) squashing repo history.")
+                self.api.super_squash_history(
+                    repo_id=self.repo_id, repo_type=self.repo_type, branch=self.revision
+                )
+            return value
+        except Exception as e:
+            logger.error(
+                f"Error while pushing to Hub: {e}"
+            )  # Depending on the setup, error might be silenced
+            raise
+
+    def push_to_hub(self) -> CommitInfo | None:
+        """
+        Push folder to the Hub and return the commit info.
+
+        <Tip warning={true}>
+
+        This method is not meant to be called directly. It is run in the background by the scheduler, respecting a
+        queue mechanism to avoid concurrent commits. Making a direct call to the method might lead to concurrency
+        issues.
+
+        </Tip>
+
+        The default behavior of `push_to_hub` is to assume an append-only folder. It lists all files in the folder and
+        uploads only changed files. If no changes are found, the method returns without committing anything. If you want
+        to change this behavior, you can inherit from [`CommitScheduler`] and override this method. This can be useful
+        for example to compress data together in a single file before committing. For more details and examples, check
+        out our [integration guide](https://huggingface.co/docs/huggingface_hub/main/en/guides/upload#scheduled-uploads).
+        """
+        # Check files to upload (with lock)
+        with self.lock:
+            if self.on_before_commit is not None:
+                self.on_before_commit()
+
+            logger.debug("Listing files to upload for scheduled commit.")
+
+            # List files from folder (taken from `_prepare_upload_folder_additions`)
+            relpath_to_abspath = {
+                path.relative_to(self.folder_path).as_posix(): path
+                for path in sorted(
+                    self.folder_path.glob("**/*")
+                )  # sorted to be deterministic
+                if path.is_file()
+            }
+            prefix = f"{self.path_in_repo.strip('/')}/" if self.path_in_repo else ""
+
+            # Filter with pattern + filter out unchanged files + retrieve current file size
+            files_to_upload: List[_FileToUpload] = []
+            for relpath in filter_repo_objects(
+                relpath_to_abspath.keys(),
+                allow_patterns=self.allow_patterns,
+                ignore_patterns=self.ignore_patterns,
+            ):
+                local_path = relpath_to_abspath[relpath]
+                stat = local_path.stat()
+                if (
+                    self.last_uploaded.get(local_path) is None
+                    or self.last_uploaded[local_path] != stat.st_mtime
+                ):
+                    files_to_upload.append(
+                        _FileToUpload(
+                            local_path=local_path,
+                            path_in_repo=prefix + relpath,
+                            size_limit=stat.st_size,
+                            last_modified=stat.st_mtime,
+                        )
+                    )
+
+        # Return if nothing to upload
+        if len(files_to_upload) == 0:
+            logger.debug("Dropping schedule commit: no changed file to upload.")
+            return None
+
+        # Convert `_FileToUpload` as `CommitOperationAdd` (=> compute file shas + limit to file size)
+        logger.debug("Removing unchanged files since previous scheduled commit.")
+        add_operations = [
+            CommitOperationAdd(
+                # TODO: Cap the file to its current size, even if the user append data to it while a scheduled commit is happening
+                # (requires an upstream fix for XET-535: `hf_xet` should support `BinaryIO` for upload)
+                path_or_fileobj=file_to_upload.local_path,
+                path_in_repo=file_to_upload.path_in_repo,
+            )
+            for file_to_upload in files_to_upload
+        ]
+
+        # Upload files (append mode expected - no need for lock)
+        logger.debug("Uploading files for scheduled commit.")
+        commit_info = self.api.create_commit(
+            repo_id=self.repo_id,
+            repo_type=self.repo_type,
+            operations=add_operations,
+            commit_message="Scheduled Commit",
+            revision=self.revision,
+        )
+
+        for file in files_to_upload:
+            self.last_uploaded[file.local_path] = file.last_modified
+
+        self.last_push_time = time.time()
+
+        return commit_info

trackio/context_vars.py

@@ -0,0 +1,18 @@
+import contextvars
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from trackio.run import Run
+
+current_run: contextvars.ContextVar["Run | None"] = contextvars.ContextVar(
+    "current_run", default=None
+)
+current_project: contextvars.ContextVar[str | None] = contextvars.ContextVar(
+    "current_project", default=None
+)
+current_server: contextvars.ContextVar[str | None] = contextvars.ContextVar(
+    "current_server", default=None
+)
+current_space_id: contextvars.ContextVar[str | None] = contextvars.ContextVar(
+    "current_space_id", default=None
+)

trackio/deploy.py

@@ -0,0 +1,433 @@
+import importlib.metadata
+import io
+import os
+import sys
+import threading
+import time
+from importlib.resources import files
+from pathlib import Path
+
+if sys.version_info >= (3, 11):
+    import tomllib
+else:
+    import tomli as tomllib
+
+import gradio
+import huggingface_hub
+from gradio_client import Client, handle_file
+from httpx import ReadTimeout
+from huggingface_hub.errors import HfHubHTTPError, RepositoryNotFoundError
+
+import trackio
+from trackio.sqlite_storage import SQLiteStorage
+from trackio.utils import get_or_create_project_hash, preprocess_space_and_dataset_ids
+
+SPACE_HOST_URL = "https://{user_name}-{space_name}.hf.space/"
+SPACE_URL = "https://huggingface.co/spaces/{space_id}"
+
+
+def _get_source_install_dependencies() -> str:
+    """Get trackio dependencies from pyproject.toml for source installs."""
+    trackio_path = files("trackio")
+    pyproject_path = Path(trackio_path).parent / "pyproject.toml"
+    with open(pyproject_path, "rb") as f:
+        pyproject = tomllib.load(f)
+    deps = pyproject["project"]["dependencies"]
+    spaces_deps = (
+        pyproject["project"].get("optional-dependencies", {}).get("spaces", [])
+    )
+    return "\n".join(deps + spaces_deps)
+
+
+def _is_trackio_installed_from_source() -> bool:
+    """Check if trackio is installed from source/editable install vs PyPI."""
+    try:
+        trackio_file = trackio.__file__
+        if "site-packages" not in trackio_file and "dist-packages" not in trackio_file:
+            return True
+
+        dist = importlib.metadata.distribution("trackio")
+        if dist.files:
+            files = list(dist.files)
+            has_pth = any(".pth" in str(f) for f in files)
+            if has_pth:
+                return True
+
+        return False
+    except (
+        AttributeError,
+        importlib.metadata.PackageNotFoundError,
+        importlib.metadata.MetadataError,
+        ValueError,
+        TypeError,
+    ):
+        return True
+
+
+def deploy_as_space(
+    space_id: str,
+    space_storage: huggingface_hub.SpaceStorage | None = None,
+    dataset_id: str | None = None,
+    private: bool | None = None,
+):
+    if (
+        os.getenv("SYSTEM") == "spaces"
+    ):  # in case a repo with this function is uploaded to spaces
+        return
+
+    trackio_path = files("trackio")
+
+    hf_api = huggingface_hub.HfApi()
+
+    try:
+        huggingface_hub.create_repo(
+            space_id,
+            private=private,
+            space_sdk="gradio",
+            space_storage=space_storage,
+            repo_type="space",
+            exist_ok=True,
+        )
+    except HfHubHTTPError as e:
+        if e.response.status_code in [401, 403]:  # unauthorized or forbidden
+            print("Need 'write' access token to create a Spaces repo.")
+            huggingface_hub.login(add_to_git_credential=False)
+            huggingface_hub.create_repo(
+                space_id,
+                private=private,
+                space_sdk="gradio",
+                space_storage=space_storage,
+                repo_type="space",
+                exist_ok=True,
+            )
+        else:
+            raise ValueError(f"Failed to create Space: {e}")
+
+    # We can assume pandas, gradio, and huggingface-hub are already installed in a Gradio Space.
+    # Make sure necessary dependencies are installed by creating a requirements.txt.
+    is_source_install = _is_trackio_installed_from_source()
+
+    with open(Path(trackio_path, "README.md"), "r") as f:
+        readme_content = f.read()
+        readme_content = readme_content.replace("{GRADIO_VERSION}", gradio.__version__)
+        if is_source_install:
+            readme_content = readme_content.replace("{APP_FILE}", "trackio/ui/main.py")
+        else:
+            readme_content = readme_content.replace("{APP_FILE}", "app.py")
+        readme_buffer = io.BytesIO(readme_content.encode("utf-8"))
+        hf_api.upload_file(
+            path_or_fileobj=readme_buffer,
+            path_in_repo="README.md",
+            repo_id=space_id,
+            repo_type="space",
+        )
+
+    if is_source_install:
+        requirements_content = _get_source_install_dependencies()
+    else:
+        requirements_content = f"trackio[spaces]=={trackio.__version__}"
+
+    requirements_buffer = io.BytesIO(requirements_content.encode("utf-8"))
+    hf_api.upload_file(
+        path_or_fileobj=requirements_buffer,
+        path_in_repo="requirements.txt",
+        repo_id=space_id,
+        repo_type="space",
+    )
+
+    huggingface_hub.utils.disable_progress_bars()
+
+    if is_source_install:
+        hf_api.upload_folder(
+            repo_id=space_id,
+            repo_type="space",
+            folder_path=trackio_path,
+            path_in_repo="trackio",
+            ignore_patterns=["README.md"],
+        )
+
+    app_file_content = """import trackio
+trackio.show()"""
+    app_file_buffer = io.BytesIO(app_file_content.encode("utf-8"))
+    hf_api.upload_file(
+        path_or_fileobj=app_file_buffer,
+        path_in_repo="app.py",
+        repo_id=space_id,
+        repo_type="space",
+    )
+
+    if hf_token := huggingface_hub.utils.get_token():
+        huggingface_hub.add_space_secret(space_id, "HF_TOKEN", hf_token)
+    if dataset_id is not None:
+        huggingface_hub.add_space_variable(space_id, "TRACKIO_DATASET_ID", dataset_id)
+    if logo_light_url := os.environ.get("TRACKIO_LOGO_LIGHT_URL"):
+        huggingface_hub.add_space_variable(
+            space_id, "TRACKIO_LOGO_LIGHT_URL", logo_light_url
+        )
+    if logo_dark_url := os.environ.get("TRACKIO_LOGO_DARK_URL"):
+        huggingface_hub.add_space_variable(
+            space_id, "TRACKIO_LOGO_DARK_URL", logo_dark_url
+        )
+    if plot_order := os.environ.get("TRACKIO_PLOT_ORDER"):
+        huggingface_hub.add_space_variable(space_id, "TRACKIO_PLOT_ORDER", plot_order)
+    if theme := os.environ.get("TRACKIO_THEME"):
+        huggingface_hub.add_space_variable(space_id, "TRACKIO_THEME", theme)
+    huggingface_hub.add_space_variable(space_id, "GRADIO_MCP_SERVER", "True")
+
+
+def create_space_if_not_exists(
+    space_id: str,
+    space_storage: huggingface_hub.SpaceStorage | None = None,
+    dataset_id: str | None = None,
+    private: bool | None = None,
+) -> None:
+    """
+    Creates a new Hugging Face Space if it does not exist.
+
+    Args:
+        space_id (`str`):
+            The ID of the Space to create.
+        space_storage ([`~huggingface_hub.SpaceStorage`], *optional*):
+            Choice of persistent storage tier for the Space.
+        dataset_id (`str`, *optional*):
+            The ID of the Dataset to add to the Space as a space variable.
+        private (`bool`, *optional*):
+            Whether to make the Space private. If `None` (default), the repo will be
+            public unless the organization's default is private. This value is ignored
+            if the repo already exists.
+    """
+    if "/" not in space_id:
+        raise ValueError(
+            f"Invalid space ID: {space_id}. Must be in the format: username/reponame or orgname/reponame."
+        )
+    if dataset_id is not None and "/" not in dataset_id:
+        raise ValueError(
+            f"Invalid dataset ID: {dataset_id}. Must be in the format: username/datasetname or orgname/datasetname."
+        )
+    try:
+        huggingface_hub.repo_info(space_id, repo_type="space")
+        print(f"* Found existing space: {SPACE_URL.format(space_id=space_id)}")
+        return
+    except RepositoryNotFoundError:
+        pass
+    except HfHubHTTPError as e:
+        if e.response.status_code in [401, 403]:  # unauthorized or forbidden
+            print("Need 'write' access token to create a Spaces repo.")
+            huggingface_hub.login(add_to_git_credential=False)
+        else:
+            raise ValueError(f"Failed to create Space: {e}")
+
+    print(f"* Creating new space: {SPACE_URL.format(space_id=space_id)}")
+    deploy_as_space(space_id, space_storage, dataset_id, private)
+
+
+def wait_until_space_exists(
+    space_id: str,
+) -> None:
+    """
+    Blocks the current thread until the Space exists.
+
+    Args:
+        space_id (`str`):
+            The ID of the Space to wait for.
+
+    Raises:
+        `TimeoutError`: If waiting for the Space takes longer than expected.
+    """
+    hf_api = huggingface_hub.HfApi()
+    delay = 1
+    for _ in range(30):
+        try:
+            hf_api.space_info(space_id)
+            return
+        except (huggingface_hub.utils.HfHubHTTPError, ReadTimeout):
+            time.sleep(delay)
+            delay = min(delay * 2, 60)
+    raise TimeoutError("Waiting for space to exist took longer than expected")
+
+
+def upload_db_to_space(project: str, space_id: str, force: bool = False) -> None:
+    """
+    Uploads the database of a local Trackio project to a Hugging Face Space.
+
+    This uses the Gradio Client to upload since we do not want to trigger a new build of
+    the Space, which would happen if we used `huggingface_hub.upload_file`.
+
+    Args:
+        project (`str`):
+            The name of the project to upload.
+        space_id (`str`):
+            The ID of the Space to upload to.
+        force (`bool`, *optional*, defaults to `False`):
+            If `True`, overwrites the existing database without prompting. If `False`,
+            prompts for confirmation.
+    """
+    db_path = SQLiteStorage.get_project_db_path(project)
+    client = Client(space_id, verbose=False, httpx_kwargs={"timeout": 90})
+
+    if not force:
+        try:
+            existing_projects = client.predict(api_name="/get_all_projects")
+            if project in existing_projects:
+                response = input(
+                    f"Database for project '{project}' already exists on Space '{space_id}'. "
+                    f"Overwrite it? (y/N): "
+                )
+                if response.lower() not in ["y", "yes"]:
+                    print("* Upload cancelled.")
+                    return
+        except Exception as e:
+            print(f"* Warning: Could not check if project exists on Space: {e}")
+            print("* Proceeding with upload...")
+
+    client.predict(
+        api_name="/upload_db_to_space",
+        project=project,
+        uploaded_db=handle_file(db_path),
+        hf_token=huggingface_hub.utils.get_token(),
+    )
+
+
+SYNC_BATCH_SIZE = 500
+
+
+def sync_incremental(
+    project: str,
+    space_id: str,
+    private: bool | None = None,
+    pending_only: bool = False,
+) -> None:
+    """
+    Syncs a local Trackio project to a Space via the bulk_log API endpoints
+    instead of uploading the entire DB file. Supports incremental sync.
+
+    Args:
+        project: The name of the project to sync.
+        space_id: The HF Space ID to sync to.
+        private: Whether to make the Space private if creating.
+        pending_only: If True, only sync rows tagged with space_id (pending data).
+    """
+    print(
+        f"* Syncing project '{project}' to: {SPACE_URL.format(space_id=space_id)} (please wait...)"
+    )
+    create_space_if_not_exists(space_id, private=private)
+    wait_until_space_exists(space_id)
+
+    client = Client(space_id, verbose=False, httpx_kwargs={"timeout": 90})
+    hf_token = huggingface_hub.utils.get_token()
+
+    if pending_only:
+        pending_logs = SQLiteStorage.get_pending_logs(project)
+        if pending_logs:
+            logs = pending_logs["logs"]
+            for i in range(0, len(logs), SYNC_BATCH_SIZE):
+                batch = logs[i : i + SYNC_BATCH_SIZE]
+                print(
+                    f"  Syncing metrics: {min(i + SYNC_BATCH_SIZE, len(logs))}/{len(logs)}..."
+                )
+                client.predict(api_name="/bulk_log", logs=batch, hf_token=hf_token)
+            SQLiteStorage.clear_pending_logs(project, pending_logs["ids"])
+
+        pending_sys = SQLiteStorage.get_pending_system_logs(project)
+        if pending_sys:
+            logs = pending_sys["logs"]
+            for i in range(0, len(logs), SYNC_BATCH_SIZE):
+                batch = logs[i : i + SYNC_BATCH_SIZE]
+                print(
+                    f"  Syncing system metrics: {min(i + SYNC_BATCH_SIZE, len(logs))}/{len(logs)}..."
+                )
+                client.predict(
+                    api_name="/bulk_log_system", logs=batch, hf_token=hf_token
+                )
+            SQLiteStorage.clear_pending_system_logs(project, pending_sys["ids"])
+
+        pending_uploads = SQLiteStorage.get_pending_uploads(project)
+        if pending_uploads:
+            upload_entries = []
+            for u in pending_uploads["uploads"]:
+                fp = u["file_path"]
+                if os.path.exists(fp):
+                    upload_entries.append(
+                        {
+                            "project": u["project"],
+                            "run": u["run"],
+                            "step": u["step"],
+                            "relative_path": u["relative_path"],
+                            "uploaded_file": handle_file(fp),
+                        }
+                    )
+            if upload_entries:
+                print(f"  Syncing {len(upload_entries)} media files...")
+                client.predict(
+                    api_name="/bulk_upload_media",
+                    uploads=upload_entries,
+                    hf_token=hf_token,
+                )
+            SQLiteStorage.clear_pending_uploads(project, pending_uploads["ids"])
+    else:
+        all_logs = SQLiteStorage.get_all_logs_for_sync(project)
+        if all_logs:
+            for i in range(0, len(all_logs), SYNC_BATCH_SIZE):
+                batch = all_logs[i : i + SYNC_BATCH_SIZE]
+                print(
+                    f"  Syncing metrics: {min(i + SYNC_BATCH_SIZE, len(all_logs))}/{len(all_logs)}..."
+                )
+                client.predict(api_name="/bulk_log", logs=batch, hf_token=hf_token)
+
+        all_sys_logs = SQLiteStorage.get_all_system_logs_for_sync(project)
+        if all_sys_logs:
+            for i in range(0, len(all_sys_logs), SYNC_BATCH_SIZE):
+                batch = all_sys_logs[i : i + SYNC_BATCH_SIZE]
+                print(
+                    f"  Syncing system metrics: {min(i + SYNC_BATCH_SIZE, len(all_sys_logs))}/{len(all_sys_logs)}..."
+                )
+                client.predict(
+                    api_name="/bulk_log_system", logs=batch, hf_token=hf_token
+                )
+
+    SQLiteStorage.set_project_metadata(project, "space_id", space_id)
+    print(f"* Synced successfully to space: {SPACE_URL.format(space_id=space_id)}")
+
+
+def sync(
+    project: str,
+    space_id: str | None = None,
+    private: bool | None = None,
+    force: bool = False,
+    run_in_background: bool = False,
+) -> str:
+    """
+    Syncs a local Trackio project's database to a Hugging Face Space.
+    If the Space does not exist, it will be created.
+
+    Args:
+        project (`str`): The name of the project to upload.
+        space_id (`str`, *optional*): The ID of the Space to upload to (e.g., `"username/space_id"`).
+            If not provided, checks project metadata first, then generates a random space_id.
+        private (`bool`, *optional*):
+            Whether to make the Space private. If None (default), the repo will be
+            public unless the organization's default is private. This value is ignored
+            if the repo already exists.
+        force (`bool`, *optional*, defaults to `False`):
+            If `True`, overwrite the existing database without prompting for confirmation.
+            If `False`, prompt the user before overwriting an existing database.
+        run_in_background (`bool`, *optional*, defaults to `False`):
+            If `True`, the Space creation and database upload will be run in a background thread.
+            If `False`, all the steps will be run synchronously.
+    Returns:
+        `str`: The Space ID of the synced project.
+    """
+    if space_id is None:
+        space_id = SQLiteStorage.get_space_id(project)
+    if space_id is None:
+        space_id = f"{project}-{get_or_create_project_hash(project)}"
+    space_id, _ = preprocess_space_and_dataset_ids(space_id, None)
+
+    def _do_sync(space_id: str, private: bool | None = None):
+        sync_incremental(project, space_id, private=private, pending_only=False)
+
+    if run_in_background:
+        threading.Thread(target=_do_sync, args=(space_id, private)).start()
+    else:
+        _do_sync(space_id, private)
+    return space_id