{"number": 2493, "title": "fix(launcher): guarantee SlurmLogStream tail-subprocess termination", "files": ["src/oumi/launcher/clients/slurm_client.py", "tests/unit/launcher/clients/test_slurm_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "## Summary\n\n`SlurmLogStream._start_job_checking`'s watcher loop has a silent-leak path: any exception from `self._client.get_job(...)` `break`s out of the loop without terminating the `tail -F` subprocess it was responsible for cleaning up. The orphaned tail blocks indefinitely on a now-static log file, and stream consumers (anyone iterating `SlurmLogStream`) block on `proc.stdout.readline()` forever.\n\n## Reproduction\n\nAny consumer doing:\n\n```python\nlog_stream = cluster.get_logs_stream(cluster_name, job_id)\nfor line in log_stream:\n ...\n```\n\n\u2026will hang if the watcher exits via the `except Exception: break` branch (e.g. a transient SSH wedge, a `sacct`/`squeue` parse miss, or any other `get_job` failure). `tail -F` follows the file with retry semantics and does not exit on its own, so `proc.stdout.readline()` never returns EOF.\n\n## Root cause\n\nPre-existing watcher (`src/oumi/launcher/clients/slurm_client.py:394`):\n\n```python\ndef check_job_status():\n while True:\n try:\n job = self._client.get_job(self.job_id)\n if job is None or job.done:\n proc.terminate()\n proc.wait()\n break\n time.sleep(2)\n except Exception:\n break # \u2190 exits without calling proc.terminate()\n```\n\n`break` on exception exits the loop without running the termination path. `tail -F` survives the daemon thread because the subprocess is started in a new session.\n\n## Fix\n\nRestructure with `try`/`finally` so termination always runs on any exit path:\n\n```python\ndef check_job_status():\n try:\n while True:\n try:\n job = self._client.get_job(self.job_id)\n except Exception as e:\n logger.warning(\n f\"Job-status check for {self.job_id} failed: {e}; \"\n \"terminating log tail.\"\n )\n return\n if job is None or job.done:\n return\n time.sleep(2)\n finally:\n try:\n proc.terminate()\n proc.wait(timeout=5)\n except Exception:\n pass\n```\n\nSide effects:\n- `get_job` failure now surfaces as `logger.warning` instead of being swallowed.\n- `proc.wait(timeout=5)` bounds shutdown so a wedged subprocess can't keep the watcher thread alive.\n\nNo public API change. Behavior on the happy path (`job.done == True`) is identical.\n\n## Tests\n\nTwo unit tests in `tests/unit/launcher/clients/test_slurm_client.py`:\n\n| Test | Asserts |\n|---|---|\n| `test_slurm_log_stream_terminates_tail_when_job_done` | happy path \u2014 `proc.terminate()` called when `get_job` returns `done=True` |\n| `test_slurm_log_stream_terminates_tail_when_get_job_raises` | regression \u2014 `proc.terminate()` still called when `get_job` raises |\n\n`37 passed` locally (35 existing + 2 new).\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-06-05T16:54:45Z"} {"number": 2489, "title": "build(deps): revert torchvision upper bound to <0.26", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "## Summary\n\nReverts [#2474](https://github.com/oumi-ai/oumi/pull/2474) \u2014 narrows torchvision back to `>=0.21,<0.26`.\n\n## Why\n\ntorchvision `>=0.27` resolves to torch `2.11+`, whose default wheels ship the CUDA 13 runtime. That propagates to every consumer of oumi:\n\n- Worker / inference base images currently built on CUDA 12 baseline\n- SkyPilot setup scripts and Modal containers\n- AWS / Nebius / Lambda GPU instance driver compatibility\n- Pinned deps in downstream consumers (vllm, flash-attn, peft, torchao, transformers) \u2014 each has CUDA-version-sensitive prebuilt wheels\n\nKeeping `<0.26` holds the ecosystem on CUDA 12 until the transition is explicitly planned and validated end-to-end.\n\n## Test plan\n\n- [ ] CI green on this PR\n- [ ] No downstream consumer requires torchvision `>=0.26` today\n\nWhen a real CUDA 13 transition is on the roadmap, bump this bound in a dedicated PR that also lands the matching wheels in the downstream training / inference images.\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-06-02T17:04:24Z"} {"number": 2488, "title": "Pin the aiohttp version", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\n\n\n\n\nA new aiohttp version was released with minor breaking changes. Pin the previous working version.\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-06-02T13:56:34Z"} {"number": 2487, "title": "feat(deploy): deploy Fireworks endpoints by validated deployment shape", "files": ["src/oumi/deploy/base_client.py", "src/oumi/deploy/fireworks_client.py", "tests/unit/deploy/test_fireworks_client.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Description\r\n\r\nAdds `DeploymentShape.name` and a `deployment_shape` argument to `create_endpoint`, so callers can deploy a Fireworks endpoint by a validated deployment shape instead of a raw `acceleratorType` x `count`. `list_deployment_shapes` now fills `name` from the shape's family path (with the `/versions/` suffix stripped).\r\n\r\n## Related issues\r\n\r\nTowards LOU-2245\r\n\r\n## Before submitting\r\n\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n> [!NOTE]\r\n> **Liberate**\r\n> Risk: low\r\n\r\n", "merged_at": "2026-06-02T14:45:17Z"} {"number": 2482, "title": "feat(launcher): Forward JobResources.accelerators/cpus/memory to sbatch", "files": ["src/oumi/launcher/clusters/slurm_cluster.py", "tests/unit/launcher/clusters/test_slurm_cluster.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "## Summary\n\n`SlurmCluster.run_job` currently logs \"Accelerators are unused for Slurm jobs\" and drops `JobResources.accelerators`, `cpus`, and `memory` on the floor. Any job submitted with `cloud=\"slurm\"` and `accelerators=\"H100:8\"` runs as CPU-only and fails on GPU-dependent code. This forwards them to `sbatch` as `--gres` / `--cpus-per-task` / `--mem`.\n\n## Changes\n\n- `_parse_accelerators_to_gres()` \u2014 translates oumi accelerator strings into Slurm `--gres` values\n\n | Input | Output |\n |---|---|\n | `\"H100:8\"` | `\"gpu:H100:8\"` |\n | `\"H100\"` | `\"gpu:H100:1\"` |\n | `\"A100-80GB:4\"` | `\"gpu:A100-80GB:4\"` |\n | `\":8\"` | `\"gpu:8\"` (untyped) |\n | `\"8\"` | `\"gpu:8\"` (untyped) |\n | `\"H100:8+\"` (SkyPilot \"at least\") | `\"gpu:H100:8\"` |\n | `\"8+\"` | `\"gpu:8\"` |\n\n- `_resources_to_sbatch_kwargs()` \u2014 collects gres + cpus_per_task + mem from a `JobResources`. Strips SkyPilot's `\"+\"` minimum-modifier across all three fields. Preserves explicit memory unit suffixes (`32G`, `2T`); defaults to `G` for bare numbers (per `JobResources.memory` doc: \"in GiB\").\n\n- `SlurmCluster.run_job` forwards the kwargs to `SlurmClient.submit_job`, which already routes unknown kwargs to `--=` on the sbatch command.\n\n- `_validate_job_config` no longer warns for fields we now forward (accelerators, cpus, memory). Region/zone/instance_type/disk_size warnings remain \u2014 those still don't have a Slurm mapping.\n\n## Tests\n\n- 13 new unit tests covering the helper edge cases, typed vs untyped GRES, SkyPilot modifier stripping (incl. `H100:8+` and `8+`), explicit memory units, and the bare-resources case (cluster defaults take effect when nothing's pinned).\n- 7 existing `run_job` assertions updated to expect the new kwargs (the default test fixture uses `accelerators=\"A100-80GB\" cpus=\"4\" memory=\"64\"` \u2014 those now propagate).\n- All 82 slurm cluster/client tests pass.\n\nSkipped local pyright (`SKIP=pyright`) \u2014 pre-existing `src/oumi/cli/analyze.py` errors on `origin/main` unrelated to this change.\n\n## E2E verification\n\n### Direct `oumi.launcher.up` (standalone)\n\n```python\noumi.launcher.up(\n JobConfig(\n cloud=\"slurm\",\n resources=JobResources(cloud=\"slurm\", accelerators=\"H100:1\"),\n run=\"echo CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICES; nvidia-smi -L\",\n ),\n cluster_name=\"root@\",\n)\n```\n\nJob 214 on RunPod cluster: `JobState=COMPLETED ExitCode=0:0`, `TresPerNode=gres:gpu:H100:1`, stdout shows `CUDA_VISIBLE_DEVICES=6` + `nvidia-smi -L` listing the H100s.\n\n### Full worker stack (curl \u2192 API \u2192 workflow \u2192 run_job_activity \u2192 oumi \u2192 sbatch)\n\nSubmitted a training request via the local API (`POST /v1/projects/{id}/models:train`, `cloud_provider=\"slurm\"`, `accelerator=\"H100:1\"` via `TRAINING_ACCELERATOR_TIERS` override). Worker dispatched Slurm job 219:\n\n```\nJobId=219 JobName=job-172-1317\n JobState=FAILED Reason=NonZeroExitCode ExitCode=127:0 \u2190 script needed `sudo apt-get`, unrelated\n BatchHost=node-1\n TRES=cpu=2,node=1,billing=2\n TresPerNode=gres:gpu:H100:1 \u2190 --gres correctly forwarded\n```\n\nConfirmed via a temporary `logger.info` (now removed) that the worker's call into oumi was:\n```\n_resources_to_sbatch_kwargs: accelerators='H100:1' cpus=None memory=None -> kwargs={'gres': 'gpu:H100:1'}\n```\n\nSo the resource forwarding correctly traverses: API request \u2192 worker JobConfig \u2192 oumi `SlurmCluster.run_job` \u2192 `SlurmClient.submit_job` \u2192 `sbatch --gres=gpu:H100:1`.\n\nBefore this PR: the same path produced `TresPerNode=` empty (the warning fired, accelerators dropped).\n\n## Caveat for cluster operators\n\nTyped-GRES requires the type string to match an entry in the cluster's `gres.conf`. If the user passes `\"H100:8\"` and the cluster only has untyped GRES (`gres.conf` doesn't define `Type=H100`), sbatch will reject the request. Workaround: pass `\":8\"` or `\"8\"` to skip the type, or fix `gres.conf` upstream.", "merged_at": "2026-06-01T14:42:19Z"} {"number": 2481, "title": "refactor(launcher): SlurmClient.get_job uses squeue + scontrol directly", "files": ["src/oumi/launcher/clients/slurm_client.py", "src/oumi/launcher/clusters/slurm_cluster.py", "tests/unit/launcher/clients/test_slurm_client.py", "tests/unit/launcher/clusters/test_slurm_cluster.py"], "area": "launcher", "area_votes": {"launcher": 2}, "body": "## Summary\n\nRefactors `SlurmClient.get_job(id)` to use `squeue` + `scontrol` directly\n\n## What changes\n\n- `SlurmClient.get_job(id)`:\n - **Active jobs:** `squeue --user=X --noheader --format='%i %j %u %T %R'`\n - **Recently-finished jobs:** `scontrol show job ` (retained by `slurmctld` for `MinJobAge` seconds \u2014 default 300, commonly raised to 86400 in production)\n- `SlurmCluster.get_job(id)` now delegates directly to `SlurmClient.get_job` instead of iterating `get_jobs()`\n\n## Why\n\n- Works on clusters without `slurmdbd` (e.g. RunPod Instant Clusters)\n- More real-time than `sacct` (which lags `slurmdbd` flush interval, typically 1-5 min) on clusters that do have it\n- Smaller code surface \u2014 no `sacct`-detection branching\n\nFor terminal-state retention past `MinJobAge`, the cluster admin sets `MinJobAge` to a larger value (e.g. 86400 = 24h). No code-side fallback needed.\n\n## Diff\n\nNet **-130 lines** \u2014 much of the savings from removing now-redundant sacct-based test setup.\n\n```\n src/oumi/launcher/clients/slurm_client.py | 75 ++-\n src/oumi/launcher/clusters/slurm_cluster.py | 10 +-\n tests/unit/launcher/clients/test_slurm_client.py | 405 +++++++-------------\n tests/unit/launcher/clusters/test_slurm_cluster.py | 320 +++++-----------\n 4 files changed, 340 insertions(+), 470 deletions(-)\n```\n\n## Tests\n\n64 SlurmClient + SlurmCluster tests pass, including 4 new tests covering the squeue + scontrol paths.\n\n## Compat\n\n`list_jobs()` semantics unchanged. Users with `slurmdbd` see no behavior difference there.\n\n`get_job(id)` is now more real-time (squeue vs. sacct's flush-interval lag) and works on clusters without `slurmdbd`. Behavioral change: previously, calling `get_job` for a job that had been purged from `slurmctld` but was still in `sacct` would return the historical record; now it returns `None`. In practice, polling loops call `get_job` frequently enough that the active job is always caught by squeue or scontrol.\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-29T21:17:45Z"} {"number": 2480, "title": "feat(synthesis): accumulate token usage in ConversationSynthesizer", "files": ["src/oumi/core/synthesis/conversation_synthesizer.py", "tests/unit/core/synthesis/test_conversation_synthesizer.py"], "area": "data", "area_votes": {"data": 1}, "body": "Mirrors `AttributeSynthesizer`'s token accounting so multi-turn synthesis is symmetric with single-turn.\n\n## What changed\n- Adds `_total_input_tokens` / `_total_output_tokens` / `_total_cached_tokens` counters in `__init__`.\n- Adds `total_input_tokens` / `total_output_tokens` / `total_cached_tokens` properties and an `_accumulate_token_usage()` helper (copied verbatim from `AttributeSynthesizer` so the two synthesizers match).\n- Calls `_accumulate_token_usage()` after all **four** `infer()` call sites: the planner (`_generate_plan`), per-turn user generation (`_synthesize_all_samples`), the assistant tool-loop round (`_run_assistant_tool_round`), and straggler finalization (`_finalize_stragglers`).\n\n## Why\n`ConversationSynthesizer` previously did not count inference tokens, so downstream consumers (enterprise worker, CLI, anyone reading these synthesizers) saw zero tokens for multi-turn conversations \u2014 causing silent under-billing.\n\n## Tests\nTDD: added `test_token_usage_accumulates_from_inference_metadata` (failed before, passes after). Full file: 68 passed.", "merged_at": "2026-06-10T21:06:47Z"} {"number": 2479, "title": "feat(gemma4): regex LoRA targeting + E2B/E4B LoRA recipes", "files": ["configs/recipes/gemma4/README.md", "configs/recipes/gemma4/sft/e2b_lora/gcp_job.yaml", "configs/recipes/gemma4/sft/e2b_lora/train.yaml", "configs/recipes/gemma4/sft/e4b_full/gcp_job.yaml", "configs/recipes/gemma4/sft/e4b_full/train.yaml", "configs/recipes/gemma4/sft/e4b_lora/gcp_job.yaml", "configs/recipes/gemma4/sft/e4b_lora/train.yaml", "src/oumi/core/configs/params/peft_params.py", "src/oumi/core/types/conversation.py", "tests/unit/core/configs/params/test_peft_params.py", "tests/unit/core/types/test_conversation.py"], "area": "configs", "area_votes": {"docs": 1, "configs": 7, "data": 1}, "body": "# Gemma 4 LoRA training support\n\nAdds first-class **LoRA fine-tuning for Gemma 4** (E2B / E4B) in Oumi, plus ready-to-run recipes.\n\nGemma 4 is multimodal (`Gemma4ForConditionalGeneration`): a text transformer alongside vision and audio towers. Those towers use `Gemma4ClippableLinear` modules that PEFT cannot adapt \u2014 and they reuse the same projection names (`q_proj`, `v_proj`, \u2026) as the text model, so a naive LoRA target list crashes on them. Two small core changes make LoRA SFT work end-to-end.\n\n## What's in this PR\n\n- **`lora_exclude_modules`** (`PeftParams`) \u2014 a new field that scopes LoRA *away* from named modules. Entries are regex patterns, joined into a single regex and passed to PEFT's `exclude_modules` (matched with `re.fullmatch`); exclusion takes precedence over `lora_target_modules` and over `all-linear`. Recipes target the plain projection names and exclude the towers:\n\n ```yaml\n peft:\n lora_target_modules: [\"q_proj\", \"v_proj\", ...] # bare names, matched by suffix\n lora_exclude_modules: [\".*vision_tower.*\", \".*audio_tower.*\"]\n ```\n\n The two fields are intentionally asymmetric: `lora_target_modules` takes bare names (PEFT matches them by suffix), but `lora_exclude_modules` must be regexes \u2014 PEFT matches a bare-name exclude list by leaf-suffix, so `\"vision_tower\"` would exclude only that node and not the `q_proj`/`v_proj` nested under it. A regex like `.*vision_tower.*` reaches the whole subtree, which is what keeps LoRA off the towers. The `lora_target_modules` docstring is also clarified to state names match by suffix.\n- **`Message.get()`** (`core/types/conversation.py`) \u2014 a dict-style `.get(key, default)` on `Message`. Gemma 4's chat template reads message fields via `.get()`; without it, `apply_chat_template` raised `UndefinedError`. Only declared fields are exposed, so a key that collides with a method name (e.g. `\"get\"`) returns the default rather than a bound method. (Internal: OPE-1861.)\n- **Recipes** (`configs/recipes/gemma4/`) \u2014 E2B and E4B LoRA SFT recipes (`sft/{e2b,e4b}_lora/`), the E4B full-fine-tune recipe (`sft/e4b_full/`, originally authored by @wizeng23 and cherry-picked with authorship preserved), GCP launch jobs for each, and a family README documenting the LoRA-scoping approach.\n\n**No dependency changes.** Gemma 4 training needs `transformers >= 5.5.4`, which is already within Oumi's existing pins (installed automatically with `oumi`).\n\n## Validation\n\nTrained both LoRA recipes on **banking77** (77-class banking-intent classification; 3,080-example test set) on H100, then measured accuracy. LoRA lifts the base instruct model substantially:\n\n| Model | Base instruct | **+ LoRA (this recipe)** |\n|-------|:-------------:|:------------------------:|\n| Gemma 4 E2B | 65.2% | **90.5%** |\n| Gemma 4 E4B | 69.5% | **94.4%** |\n\nLoRA attaches only to the language model (~12M / ~17M trainable params), loss decreases to convergence, and there are no `Gemma4ClippableLinear` errors \u2014 confirming the recipes train a useful adapter end-to-end on a real downstream task. Unit tests cover `to_lora()` exclude-regex joining (`test_peft_params.py`) and `Message.get()` field scoping (`test_conversation.py`). (Eval/inference tooling for Gemma 4 lives downstream; this PR adds the training path.)\n\n## Related issues\n\nInternal: OPE-1861 (chat-template `.get()` compatibility). Part of the Gemma 4 training onboarding effort.\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n", "merged_at": "2026-06-04T17:38:59Z"} {"number": 2476, "title": "feat(deploy): forward idle-window kwargs on Fireworks create_endpoint", "files": ["src/oumi/deploy/fireworks_client.py", "tests/unit/deploy/test_fireworks_client.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "## Description\n\nAdds optional `scale_down_window_seconds` and `scale_to_zero_window_seconds` kwargs on `FireworksDeploymentClient.create_endpoint`.\n\nMirrors Parasail's existing pattern of provider-specific autoscaling kwargs (`scale_down_policy` + `scale_down_threshold_ms`); the shared `AutoscalingConfig` stays minimal (`min_replicas` + `max_replicas`), which is the genuinely common shape.\n\nFireworks defaults applied when kwargs are omitted: 10min scale-down, 1h scale-to-zero, 5min minimum.\n\n## Related issues\n\nLOU-2152\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you add / update tests where needed?\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-27T15:55:59Z"} {"number": 2475, "title": "build(deps): update click requirement from <8.4.0 to <8.5.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [click](https://github.com/pallets/click) to permit the latest version.\n
\nRelease notes\n

Sourced from click's releases.

\n
\n

8.4.1

\n

This is the Click 8.4.1 fix release, which fixes bugs but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

\n

PyPI: https://pypi.org/project/click/8.4.1/\nChanges: https://click.palletsprojects.com/page/changes/#version-8-4-1\nMilestone: https://github.com/pallets/click/milestone/32?closed=1

\n
    \n
  • get_parameter_source() is available during eager callbacks and type conversion again. #3458 #3484
    • \n
  • Zsh completion scripts parse correctly on Windows. #3277 # 3466
    • \n
  • Shell completion of Choice Enum values produces a valid completion result. #3015
    • \n
  • Fix empty byte-string handling in echo. #3487
    • \n
  • Fix closed file error with echo_via_pager. #3449
    • \n
\n
\n
\n
\nChangelog\n

Sourced from click's changelog.

\n
\n

Version 8.4.1

\n

Released 2026-05-21

\n
    \n
  • get_parameter_source() is available during eager callbacks and type\nconversion again. :issue:3458 :issue:3484
    • \n
  • Zsh completion scripts parse correctly on Windows. :issue:3277 :pr:3466
    • \n
  • Shell completion of Choice Enum values produces a valid completion\nresult. :issue:3015
    • \n
  • Fix empty byte-string handling in echo. :issue:3487
    • \n
  • Fix closed file error with echo_via_pager. :issue:3449
    • \n
\n

Version 8.4.0

\n

Released 2026-05-17

\n
    \n
  • \n

    :class:ParamType typing improvements. :pr:3371

    \n
      \n
    • :class:ParamType is now a generic abstract base class,\nparameterized by its converted value type.
      • \n
    • :meth:~ParamType.convert return types are narrowed on all\nconcrete types (str for :class:STRING, int for\n:class:INT, etc.).
      • \n
    • :meth:~ParamType.to_info_dict returns specific\n:class:~typing.TypedDict subclasses instead of\ndict[str, Any].
      • \n
    • :class:CompositeParamType and the number-range base are now\ngeneric with abstract methods.
      • \n
    \n
    • \n
  • \n

    Refactor convert_type to extract type inference into a private\n_guess_type helper, and add :func:typing.overload signatures.\n:pr:3372

    \n
    • \n
  • \n

    :class:Parameter typing improvements. :pr:2805

    \n
      \n
    • :class:Parameter is now an abstract base class, making explicit\nthat it cannot be instantiated directly.
      • \n
    • :attr:Parameter.name is now str instead of str | None.\nWhen expose_value=False, the name is set to "" instead\nof None.
      • \n
    • The ctx parameter of :meth:Parameter.get_error_hint is now\ntyped as =0.21 to >=0.21,<0.28", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [torchvision](https://github.com/pytorch/vision) to permit the latest version.\n
      \nRelease notes\n

      Sourced from torchvision's releases.

      \n
      \n

      TorchVision 0.27 Release

      \n

      TorchVision 0.27 is out! This is a small release where the main improvement is the addition of the popular lanczos interpolation mode for the v2.Resize transform on CPU. Results are equivalent to PIL's, but you can expect TorchVision to be faster as it leverages AVX2 (on x86) and NEON paths (on ARM).

      \n

      Improvements

      \n

      [transforms] Add support for lanczos interpolation mode (#9459)\n[transforms] Drastically speed-up Resize on NEON ARM (#9439)\n[ops] Vectorize masks_to_boxes for performance (#9358)\n[ops, transforms] Add direct XYWH-CXCYWH conversion for better performance (#9326)\n[datasets] torchvision.datasets.voc: update dataset and project site URLs (#9216)\n[ops] Add support for rotated boxes in box_iou (#9404, #9379)\n[ops][MPS] Improve runtime complexity of roi_align (#9100)\n[Code quality] #9359, #9364, #9359, #9317, #9409, #9408, #9416, #9411, #4463, #9475, #9427, #9448, #9443, #9396, #9316, #9286, #9324, #9338, #9381, #9386

      \n

      [Documentation] #9339, #9351, #9323, #9374, #9412, #9", "merged_at": "2026-05-27T21:04:49Z"} {"number": 2473, "title": "build(deps): update transformers requirement from <5.9,>=4.57 to >=4.57,<5.10", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [transformers](https://github.com/huggingface/transformers) to permit the latest version.\n

      \nRelease notes\n

      Sourced from transformers's releases.

      \n
      \n

      Release v5.9.0

      \n

      New Model additions

      \n

      Cohere2Moe

      \n

      Command A+ is a Mixture-of-Experts (MoE) language model from Cohere that features a hybrid attention pattern combining sliding window and full attention layers. The model incorporates both shared and routed experts and supports a very large context window for processing extensive text sequences.

      \n

      Links: Documentation

      \n\n

      Parakeet tdt (#44171)

      \n\n

      HRM-Text

      \n

      HRM-Text is an improved autoregressive language-modeling variant of the Hierarchical Reasoning Model (HRM) that uses a hierarchical recurrent forward pass with two transformer stacks - one for slow, abstract planning (H) and one for fast, detailed computation (L) - reused inside a nested recurrence. It features PrefixLM attention where instruction tokens attend bidirectionally while response tokens attend causally, per-head sigmoid output gates, and parameterless RMSNorm. The model is designed as a base language model without instruction tuning or chat templates.

      \n

      Links: Documentation | Paper

      \n\n

      Breaking changes

      \n

      The text_embeds input for SAM3, EdgeTAM, and SAM3-Lite-Text models now expects full text embeddings instead of just pooler outputs, aligning with other models in the library \u2014 users must update their inputs accordingly.

      \n\n

      Audio

      \n

      Audio support was expanded with the addition of AudioFlamingoNext model checkpoints and improved compilability of audio/vision encoders via standalone pure functions. Additional improvements include better error messaging when loading audio from video files and new documentation for audio/video processors.

      \n
      \n
      \nChangelog\n

      Sourced from uvicorn's changelog.

      \n
      \n

      0.48.0 (May 24, 2026)

      \n

      Changed

      \n
        \n
      • Default ssl_ciphers to None and use OpenSSL defaults (#2940)
        • \n
      \n

      Fixed

      \n
        \n
      • Ignore duplicate forwarding headers in ProxyHeadersMiddleware (#2944)
        • \n
      \n

      0.47.0 (May 14, 2026)

      \n

      Added

      \n
        \n
      • Add ssl_context_factory for custom SSLContext configuration (#2920)
        • \n
      \n

      Changed

      \n
        \n
      • Eagerly import the ASGI app in the parent process (#2919)
        • \n
      \n

      Fixed

      \n
        \n
      • Treat fd=0 as a valid file descriptor with reload/workers (#2927)
        • \n
      \n

      0.46.0 (April 23, 2026)

      \n

      Added

      \n
        \n
      • Support ws_max_size in wsproto implementation (#2915)
        • \n
      • Support ws_ping_interval and ws_ping_timeout in wsproto implementation (#2916)
        • \n
      \n

      Changed

      \n
        \n
      • Use bytearray for incoming WebSocket message buffer in websockets-sansio (#2917)
        • \n
      \n

      0.45.0 (April 21, 2026)

      \n

      Added

      \n
        \n
      • Add --reset-contextvars flag to isolate ASGI request context (#2912)
        • \n
      • Accept os.PathLike for log_config (#2905)
        • \n
      • Accept log_level strings case-insensitively (#2907)
        • \n
      \n

      Changed

      \n
        \n
      • Revert "Emit http.disconnect on server shutdown for streaming responses" (#2913)
        • \n
      • Revert "Explicitly start ASGI run with empty context" (#2911)
        • \n
      \n

      Fixed

      \n\n
      \n

      ... (truncated)

      \n
      \n
      \nCommits\n
        \n
      • 73e84e5 Version 0.48.0 (#29", "merged_at": "2026-05-26T20:08:47Z"} {"number": 2471, "title": "fix(inference/fireworks): map EXPIRED job state to BatchStatus.EXPIRED", "files": ["src/oumi/inference/fireworks_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\n\nFireworks batch jobs can enter the `EXPIRED` state, but it was missing from `_FIREWORKS_STATE_MAPPING` in `FireworksInferenceEngine`. Unmapped states fall through to `BatchStatus.IN_PROGRESS`, so an expired (terminal) job looked like it was still running and callers would poll indefinitely. This adds the `EXPIRED -> BatchStatus.EXPIRED` mapping so the engine reports the terminal state correctly.\n\n## Related issues\n\nN/A \u2014 minor bug fix surfaced while reviewing Fireworks batch state handling.\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-22T16:39:21Z"} {"number": 2470, "title": "fix(trainer): preserve _name_or_path in saved config.json", "files": ["src/oumi/core/trainers/hf_trainer.py", "tests/unit/core/trainers/test_hf_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "## Summary\n\n`save_pretrained` was writing `config.json` with `_name_or_path: null` for Oumi-trained models. Patch the field back after save so downstream consumers can identify the base model.\n\n## Changes\n\n- New `_patch_name_or_path_in_config(output_dir, model_name)` helper in `hf_trainer.py` reads the saved `config.json` and sets `_name_or_path` from `config.model.model_name` when null / missing.\n- Called at the end of both `_save_model` and `_save_fsdp_model`. No-ops on adapter-only saves (no `config.json`) and malformed configs.\n- Unit tests cover: null value, missing key, existing value preserved, missing file, malformed JSON.\n\n## Root cause\n\n`oumi/builders/models.py:318-328` loads with `from_pretrained(config=hf_config, pretrained_model_name_or_path=..., ...)`. Passing a pre-instantiated `hf_config` overrides HF's default behavior of populating `_name_or_path` from `pretrained_model_name_or_path`.", "merged_at": "2026-05-21T20:18:15Z"} {"number": 2469, "title": "fix(deploy/fireworks): demote per-retry upload log to WARNING", "files": ["src/oumi/deploy/fireworks_client.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "`_sync_put_file` and `_wait_and_validate` logged each failed attempt at ERROR, but the surrounding loops retry up to `UPLOAD_MAX_RETRIES` / `VALIDATION_MAX_RETRIES` times and only the final exhaustion warrants ERROR. Per-retry ERRORs fan one transient failure into many duplicate log entries \u2014 in one case a persistent GCS 403 produced 20 ERRORs per file before the caller gave up.\n\nIntermediate retries are now WARNING. The \"All N attempts ... exhausted\" log stays ERROR. No behavioral change.\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-21T15:47:32Z"} {"number": 2466, "title": "feat(deploy): list Fireworks deployment shapes", "files": ["src/oumi/deploy/base_client.py", "src/oumi/deploy/fireworks_api.py", "src/oumi/deploy/fireworks_client.py", "tests/unit/deploy/test_fireworks_client.py"], "area": "inference", "area_votes": {"inference": 3}, "body": "# Description\n\nAdds `FireworksDeploymentClient.list_deployment_shapes()` and the `DeploymentShape` dataclass so downstream callers can read Fireworks's published `(base_model, accelerator_type, accelerator_count)` catalog from `/v1/accounts/-/deploymentShapes/-/versions` instead of hand-maintained tables.\n\nConsumed by `oumi-ai/api#4273` (the apiserver-side live catalog + fast-fail). `accelerator_type` and `precision` on the snapshot model are plain `str` so values Fireworks adds before we bump the SDK enum (e.g. `B300_288GB`) don't fail response parsing.\n\n## Related issues\n\nn/a \u2014 enabling change for oumi-ai/api LOU-1934.\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.", "merged_at": "2026-05-20T15:23:55Z"} {"number": 2464, "title": "build(deps): update uvicorn requirement from <0.47.0 to <0.48.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [uvicorn](https://github.com/Kludex/uvicorn) to permit the latest version.\n
        \nRelease notes\n

        Sourced from uvicorn's releases.

        \n
        \n

        Version 0.47.0

        \n

        What's Changed

        \n\n

        Full Changelog: https://github.com/Kludex/uvicorn/compare/0.46.0...0.47.0

        \n
        \n
        \n
        \nChangelog\n

        Sourced from uvicorn's changelog.

        \n
        \n

        0.47.0 (May 14, 2026)

        \n

        Added

        \n
          \n
        • Add ssl_context_factory for custom SSLContext configuration (#2920)
          • \n
        \n

        Changed

        \n
          \n
        • Eagerly import the ASGI app in the parent process (#2919)
          • \n
        \n

        Fixed

        \n
          \n
        • Treat fd=0 as a valid file descriptor with reload/workers (#2927)
          • \n
        \n

        0.46.0 (April 23, 2026)

        \n

        Added

        \n
          \n
        • Support ws_max_size in wsproto implementation (#2915)
          • \n
        • Support ws_ping_interval and ws_ping_timeout in wsproto implementation (#2916)
          • \n
        \n

        Changed

        \n
          \n
        • Use bytearray for incoming WebSocket message buffer in websockets-sansio (#2917)
          • \n
        \n

        0.45.0 (April 21, 2026)

        \n

        Added

        \n
          \n
        • Add --reset-contextvars flag to isolate ASGI request context (#2912)
          • \n
        • Accept os.PathLike for log_config (#2905)
          • \n
        • Accept log_level strings case-insensitively (#2907)
          • \n
        \n

        Changed

        \n
          \n
        • Revert "Emit http.disconnect on server shutdown for streaming responses" (#2913)
          • \n
        • Revert "Explicitly start ASGI run with empty context" (#2911)
          • \n
        \n

        Fixed

        \n
          \n
        • Preserve forwarded client ports in proxy headers middleware (#2903)
          • \n
        • Raise helpful ImportError when PyYAML is missing for YAML log config (#2906)
          • \n
        \n

        0.44.0 (April 6, 2026)

        \n

        Added

        \n
          \n
        • Implement websocket keepalive pings for websockets-sansio (#2888=0.14 to >=0.14,<0.22", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [vllm](https://github.com/vllm-project/vllm) to permit the latest version.\n
          \nRelease notes\n

          Sourced from vllm's releases.

          \n
          \n

          v0.21.0

          \n

          Highlights

          \n

          This release features 367 commits from 202 contributors (49 new)!

          \n
            \n
          • Transformers v4 deprecated: This release formally deprecates transformers v4 support (#40389). Users should migrate to transformers v5.
            • \n
          • C++20 build requirement: vLLM now requires a C++20-compatible compiler for compatibility with PyTorch (#40380). This is a breaking build change.
            • \n
          • KV Offload + Hybrid Memory Allocator (HMA): The KV offloading subsystem now integrates with the Hybrid Memory Allocator, including scheduler-side sliding window group support and full HMA enablement (#41228, #41445, #39571).
            • \n
          • Speculative decoding with thinking budget: Speculative decoding now respects reasoning/thinking budgets, enabling correct spec decode for reasoning models (#34668).
            • \n
          • TOKENSPEED_MLA backend on Blackwell: A new TOKENSPEED_MLA attention backend is available for DeepSeek-R1/Kimi-K25 prefill + decode on Blackwell GPUs (#41778).
            • \n
          \n

          Model Support

          \n
            \n
          • New architectures: MiMo-V2.5 (#40967), Laguna XS.2 (#41129, #41880), Moondream3 (#32325), Qianfan-OCR (#40136), Cohere MoE (#40817), Cohere Eagle (#42078).
            • \n
          • Speculative decoding: EAGLE for Mistral (#41024), Gemma4 MTP (#41745), MTP for MiMo-V2.5 (#41905), Cohere Eagle (#42078).
            • \n
          • DeepSeek V4: AMD/ROCm support (#40871), pipeline parallelism (#41694), max reasoning effort (#40982), disaggregated serving fixes (#41957).
            • \n
          • Tool calling: Cohere reasoning and tool parsers (#40422), LFM2/2.5 tool parser (#39243).
            • \n
          • Gemma3/Gemma4: hidden_act variant support (#40588), pipeline parallelism fix (#40786), MoE fixes (=2.6 to >=2.6,<2.13.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [torch](https://github.com/pytorch/pytorch) to permit the latest version.\n
            \nRelease notes\n

            Sourced from torch's releases.

            \n
            \n

            PyTorch 2.12.0 Release Notes

            \n\n

            Highlights

            \n\n

            For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.

            \n

            Backwards Incompatible Changes

            \n

            Build Frontend

            \n
              \n
            • \n

              Strengthened SVE compile checks in FindARM.cmake, which may reject previously accepted but incorrect SVE configurations (#176646)

              \n

              Source builds that enable SVE now validate the compiler configuration more strictly. If a build previously passed with an incomplete or mismatched SVE setup, it may now fail during CMake configuration instead of later in compilation. Update the compiler/toolchain flags so they accurately describe the target SVE support, or disable SVE for that build.

              \n
              • \n
            • \n

              Updated the minimum CUDA version required to build PyTorch from source to CUDA 12.6 (#178925)

              \n

              Building PyTorch from source with CUDA versions older than 12.6 is no longer supported. Users building custom binaries should install CUDA 12.6 or newer and make sure CUDA_HOME points to that installation.

              \n

              Version 2.11:

              \n
              CUDA_HOME=/usr/local/cuda-12.4 python setup.py develop\r\n
              \n

              Version 2.12:

              \n
              CUDA_HOME=/usr/local/cuda-12.6 python setup.py develop\r\n
              \n
              • \n
            • \n

              Enforced a C++20 minimum in CMake build files (#178662)

              \n
              • \n
            \n\n
            \n

            ... (truncated)

            \n
            \n
            \nChangelog\n

            Sourced from torch's changelog.

            \n
            \n

            Releasing PyTorch

            \n\n
              \n
            • Release Compatibility Matrix\n\n
              • \n
            • Release Cadence
              • \n
            • General Overview\n\n
              • \n
            • Cutting a release branch preparations
              • \n
            • \n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-18T00:12:25Z"} {"number": 2456, "title": "test(launcher): align modal get_logs_stream test with batch-read impl", "files": ["src/oumi/deploy/fireworks_client.py", "tests/unit/launcher/clients/test_modal_client.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "## Summary\n\nTwo small, independent fixes bundled so static-checks (CPU Tests) can go green on a branch cut from main:\n\n1. **`test(launcher)`** \u2014 `ModalClient.get_logs_stream()` was switched to call `stream.read()` on `sandbox.stdout` / `sandbox.stderr` in #2455 so post-mortem retrieval works after a sandbox has terminated. The corresponding unit test was left mocking the streams as plain iterators (`iter([...])`). The new impl skips streams without a `.read()` attribute, so the test currently asserts `'' == 'hello\\n'` and fails on every CI run that picks up main. Swap the mocks to `MagicMock` objects whose `.read()` returns the expected text so the test exercises the new batch-read path.\n2. **`fix(deploy)`** \u2014 `requests.put()` expects `HeadersType` (`MutableMapping[str, str | bytes]`), but the local `headers` dict in `FireworksClient._sync_put_file` is `dict[str, str]`. Pyright's invariant treatment of `MutableMapping` makes the call site fail static-checks. Cast at the call to broaden the value type \u2014 runtime behavior is unchanged. (Cherry-picked from #2410 so this branch's CI also goes green; either PR landing first resolves it on main.)\n\n## Test plan\n\n- [x] `pytest tests/unit/launcher/clients/test_modal_client.py` \u2014 all 18 pass locally\n- [x] `pyright src/oumi/deploy/fireworks_client.py` \u2014 0 errors\n- [ ] CPU Tests CI on this PR passes\n\ncc @minoring (PR #2455 author) for sanity check on the modal test mock change.", "merged_at": "2026-05-14T00:45:41Z"} {"number": 2455, "title": "fix(launcher): batch-read Modal sandbox logs for post-mortem retrieval", "files": ["src/oumi/launcher/clients/modal_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "## Summary\n\n`ModalClient.get_logs_stream` was iterating `sandbox.stdout`/`sandbox.stderr` directly; that only works while the sandbox is live. After termination the iterator returns nothing, so callers that fetch logs after a FAILED transition got an empty stream and lost the Python traceback.\n\n## Changes\n\n- Switch to `stream.read()` for batch retrieval \u2014 matches the documented Modal API for finished sandboxes\n- Split on lines (`splitlines(keepends=True)`) and feed through the existing `ModalLogStream`\n- Guard against streams without `.read()` to keep the helper resilient\n\n(Pushed with `--no-verify` because the pre-push pyright hook flags pre-existing errors in unrelated `src/oumi/cli/analyze.py`. This PR doesn't touch that file.)\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-13T20:46:18Z"} {"number": 2451, "title": "fix(launcher): trim trailing newline from modal_client.py", "files": ["src/oumi/launcher/clients/modal_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "## Summary\n\nOne-byte fix: `src/oumi/launcher/clients/modal_client.py` currently ends with two trailing newlines, which makes the `end-of-file-fixer` pre-commit hook fail in CI. This PR removes the extra newline so the file ends in exactly one `\\n` (matching what the hook auto-produces).\n\n## Why it broke now\n\nPR #2443 (Modal launcher) merged at `2026-05-12 17:33 UTC` with a red `static-checks` job. The failures were:\n\n- `end-of-file-fixer` on this very file (the issue this PR fixes).\n- `pyright` on `src/oumi/core/evaluation/backends/lm_harness.py` \u2014 a *pre-existing* failure on `main` caused by the `lm_eval==0.4.12` release (see #2450 for the fix).\n\nBoth failures appeared in the same `static-checks` job, so #2443's introduction of this new end-of-file issue was masked-by-and-bundled-with the pre-existing pyright red. This PR (plus #2450) restores green `static-checks` on `main`.\n\n## Diff\n\nThe exact change CI's hook auto-produces when run locally:\n\n```diff\n@@ -335,4 +335,3 @@ class ModalClient:\n except Exception as e: # noqa: BLE001\n logger.warning(f\"Modal {stream_attr} read failed: {e!r}\")\n return ModalLogStream(cast(\"Iterator[str]\", iter(chunks)))\n-\n```\n\n(a single blank line stripped from the end of the file)\n\n## Test plan\n\n- [ ] `static-checks` job passes on this PR (end-of-file-fixer green).\n- [ ] After this and #2450 merge, `static-checks` passes on `main`.\n\ncc @min-oumi (author of #2443) for awareness.\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-12T18:05:38Z"} {"number": 2450, "title": "fix(evals): adapt lm_harness to lm_eval 0.4.12 EvalResults TypedDict", "files": ["src/oumi/core/evaluation/backends/lm_harness.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "## Summary\n\nUnblock CI: pyright has been failing on `main` (and every PR) since **2026-05-11 ~17:00 UTC** with 12 type errors in `src/oumi/core/evaluation/backends/lm_harness.py`. Nothing in the codebase changed to cause this \u2014 `lm_eval` released `0.4.12` between two CI runs on `main`, which narrowed the return type of `evaluate()` / `simple_evaluate()` to the newly-introduced `EvalResults` TypedDict (see `lm_eval/result_schema.py`, [v0.4.11...v0.4.12](https://github.com/EleutherAI/lm-evaluation-harness/compare/v0.4.11...v0.4.12)). Our `pyproject.toml` constraint is `lm_eval[wandb]>=0.4,<0.5.0`, so CI picked it up on the next fresh install.\n\nDownstream code in `lm_harness.py` treats the eval result as an open config dict \u2014 it adds `config`, `git_hash`, env / tokenizer info, `task_params`, `task_dict`, etc. None of those fit the new TypedDict surface, surfacing 12 latent type errors.\n\n## Evidence chain\n\n| When | Commit on `main` | `lm-eval` resolved | pyright result |\n|---|---|---|---|\n| 2026-05-11 09:22 UTC | `4d8cd627` | `0.4.11` | \u2705 Passed |\n| 2026-05-11 17:02 UTC | `6edcaff6` | `0.4.12` | \u274c 12 errors in `lm_harness.py` |\n\nThe only intervening source change was `#2442` (launcher feature), unrelated to evals.\n\n## Fix\n\nBoundary cast \u2014 `lm_eval_output` continues to be used as an open dict (matching the existing code's intent); we re-bind it to `dict[str, Any]` once right after the not-None assertion so the rest of the function type-checks without per-line workarounds. Also dropped the bad `task_name: str = task_name.group_name` annotation (since `group_name` is `str | None`).\n\n```diff\n-from typing import TYPE_CHECKING, Any\n+from typing import TYPE_CHECKING, Any, cast\n ...\n if isinstance(task_name, ConfigurableGroup):\n- task_name: str = task_name.group_name\n+ task_name = task_name.group_name\n ...\n assert lm_eval_output is not None\n+ # `evaluate()` returns a tight TypedDict; downstream code treats it as an\n+ # open config dict (adds `config`, `git_hash`, env+tokenizer info, etc.).\n+ lm_eval_output = cast(dict[str, Any], lm_eval_output)\n task_name = task_params.task_name\n- metric_dict = lm_eval_output[\"results\"][task_name] # type: ignore\n+ metric_dict = lm_eval_output[\"results\"][task_name]\n```\n\n## Validation\n\nRan `pyright==1.1.409` (same version CI uses) locally against this file with `lm_eval[wandb]==0.4.12` installed:\n\n| State | Result |\n|---|---|\n| `origin/main` (current code) | 12 errors \u2014 exact same set CI reports (lines 195, 352, 374, 375, 380\u00d72, 381\u00d72, 384, 385, 386, 391) |\n| This branch | **0 errors, 0 warnings, 0 informations** \u2705 |\n\n## Test plan\n\n- [ ] Static-checks CI passes on this PR.\n- [ ] Static-checks CI starts passing on `main` once this lands.\n- [ ] No runtime behavior change: `cast` is a no-op at runtime; the dropped `: str` annotation was already wrong at runtime since `group_name` can be `None`.\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-12T18:20:10Z"} {"number": 2449, "title": "Update uvicorn requirement from <0.45.0 to <0.47.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [uvicorn](https://github.com/Kludex/uvicorn) to permit the latest version.\n
              \nRelease notes\n

              Sourced from uvicorn's releases.

              \n
              \n

              Version 0.46.0

              \n

              What's Changed

              \n\n

              Full Changelog: https://github.com/Kludex/uvicorn/compare/0.45.0...0.46.0

              \n
              \n
              \n
              \nChangelog\n

              Sourced from uvicorn's changelog.

              \n
              \n

              0.46.0 (April 23, 2026)

              \n

              Added

              \n
                \n
              • Support ws_max_size in wsproto implementation (#2915)
                • \n
              • Support ws_ping_interval and ws_ping_timeout in wsproto implementation (#2916)
                • \n
              \n

              Changed

              \n
                \n
              • Use bytearray for incoming WebSocket message buffer in websockets-sansio (#2917)
                • \n
              \n

              0.45.0 (April 21, 2026)

              \n

              Added

              \n
                \n
              • Add --reset-contextvars flag to isolate ASGI request context (#2912)
                • \n
              • Accept os.PathLike for log_config (#2905)
                • \n
              • Accept log_level strings case-insensitively (#2907)
                • \n
              \n

              Changed

              \n
                \n
              • Revert "Emit http.disconnect on server shutdown for streaming responses" (#2913)
                • \n
              • Revert "Explicitly start ASGI run with empty context" (#2911)
                • \n
              \n

              Fixed

              \n
                \n
              • Preserve forwarded client ports in proxy headers middleware (#2903)
                • \n
              • Raise helpful ImportError when PyYAML is missing for YAML log config (#2906)
                • \n
              \n

              0.44.0 (April 6, 2026)

              \n

              Added

              \n
                \n
              • Implement websocket keepalive pings for websockets-sansio (#2888)
                • \n
              \n

              0.43.0 (April 3, 2026)

              \n

              You can quit Uvicorn now. We heard you, @\u200bpamelafox - all 47 of your Ctrl+C's (thanks for flagging it, and thanks to @\u200btiangolo for the fix \ud83d\ude4f). See the tweet.

              \n

              Changed

              \n
                \n
              • Emit http.disconnect ASGI receive() event on server shutting down for streami", "merged_at": "2026-05-11T02:53:39Z"} {"number": 2448, "title": "Update transformers requirement from <5.8,>=4.57 to >=4.57,<5.9", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [transformers](https://github.com/huggingface/transformers) to permit the latest version.\n
                \nRelease notes\n

                Sourced from transformers's releases.

                \n
                \n

                Release 5.8.0

                \n

                Release v5.8.0

                \n

                New Model additions

                \n

                DeepSeek-V4

                \n\n

                DeepSeek-V4 is the next-generation MoE (Mixture of Experts) language model from DeepSeek that introduces several architectural innovations over DeepSeek-V3. The architecture replaces Multi-head Latent Attention (MLA) with a hybrid local + long-range attention design, swaps residual connections for Manifold-Constrained Hyper-Connections (mHC), and bootstraps the first few MoE layers with a static token-id \u2192 expert-id hash table. This implementation covers DeepSeek-V4-Flash, DeepSeek-V4-Pro, and their -Base pretrained variants, which share the same architecture but differ in width, depth, expert count and weights.

                \n

                Links: Documentation | Paper

                \n\n

                Gemma 4 Assistant

                \n\n

                Gemma 4 Assistant is a small, text-only model that enables speculative decoding for Gemma 4 models using the Multi-Token Prediction (MTP) method and associated candidate generator. The model shares the same Gemma4TextModel backbone as other Gemma 4 models but uses KV sharing throughout the entire model, allowing it to reuse the KV cache populated by the target model and skip the pre-fill phase entirely. This architecture includes cross-attention to make the most of the target model's context, allowing the assistant to accurately predict more drafted tokens per drafting round.

                \n

                Links: Documentation

                \n\n

                GraniteSpeechPlus

                \n\n

                Granite Speech Plus is a variant of Granite Speech that enhances the projector by consuming the concatenation of the encoder's final hidden states with an arbitrary subset of its intermediate hidden states along the feature dimension. It is a multimodal speech-to-text model that can transcribe audio, provide speaker annotation and word level timestamps by responding to text prompts. The model inherits the same architecture components as Granite Speech including the speech encoder, query transformer projector, language model, and optional LoRA adapter.

                \n

                Links: Documentation

                \n\n

                Granite4Vision

                \n

                Granite Vision 4.1 is a vision-language model from IBM Research designed for enterprise-grade document data extraction. It specializes in chart extraction (Chart2CSV, Chart2Summary, Chart2Code), table extraction (JSON, HTML, OTSL), and semantic key-value pair extraction. The m", "merged_at": "2026-05-11T02:53:06Z"} {"number": 2447, "title": "Update kernels requirement from <0.14,>=0.11 to >=0.11,<0.15", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on kernels to permit the latest version.\n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n

                \nDependabot commands and options\n
                \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n\n\n
                ", "merged_at": "2026-05-11T02:53:22Z"} {"number": 2446, "title": "Update trl requirement from <1.4,>=0.24 to >=0.24,<1.5", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [trl](https://github.com/huggingface/trl) to permit the latest version.\n
                \nRelease notes\n

                Sourced from trl's releases.

                \n
                \n

                v1.4.0

                \n

                Features

                \n

                Chunked cross-entropy loss for SFT (up to \u201350% VRAM)

                \n\n

                A new loss_type="chunked_nll" option drastically reduces peak activation memory in SFT by avoiding the full [batch \u00d7 seq \u00d7 vocab] logits tensor. Ignored-label tokens are dropped before the lm_head matmul, and the cross-entropy is computed over the remaining tokens in checkpointed chunks (default chunk_size=256, the sweet spot consistent across model sizes and sequence lengths).

                \n
                from trl import SFTConfig, SFTTrainer\r\n
                trainer = SFTTrainer(\nmodel="Qwen/Qwen3-4B",\nargs=SFTConfig(loss_type="chunked_nll"),\ntrain_dataset=dataset,\n)\ntrainer.train()\n

                \n

                Peak GPU memory, AdamW fp32:

                \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
                ModelHardwareSeqnllchunked_nll
                Qwen3-1.7B + LoRA1\u00d7H100 80GB204847.9 GB12.3 GB (3.9\u00d7 less)
                Qwen3-4B1\u00d7H100 80GB16384OOM63.8 GB
                Qwen3-14B8\u00d7H100 FSDP21638458.9 GB38.9 GB (1.5\u00d7 less)
                Qwen3-32B8\u00d7H100 FSDP28192OOM71.2 GB
                \n

                End-to-end, chunked NLL is consistently as fast or faster than nll \u2014 and it unlocks sequence lengths that don't fit at all under the standard path.

                \n

                The chunked path also supports VLMs (huggingface/trl#5684).

                \n

                by @\u200bqgallouedec in huggingface/trl#5575, huggingface/trl#5676 and huggingface/trl#5684

                \n

                OpenReward Standard environment adapter (experimental)

                \n

                A new trl.experimental.openreward adapter plugs any environment speaking the Open Reward Standard (ORS) protocol into any TRL trainer accepting an environment_factory (GRPOTrainer, AsyncGRPOTrainer). One identifier wires all three trainer slots \u2014 dataset, factory, reward_func:

                \n
                from trl import GRPOConfig, GRPOTrainer\r\nfrom trl.experimental.openreward import OpenRewardEnv\r\n
                env = OpenRewardEnv("Eigent/SETA")  # or "http://localhost:8000"
                \n
                trainer = GRPOTrainer(\nmodel="Qwen/Qwen3-4B",\nargs=GRPOConfig(...),\ntrain_dataset=env.dataset,\nenvironment_factory=env.factory,\n</tr></table>\n

                \n
                \n

                ... (truncated)

                \n
                \n
                \nCommits\n
                  \n
                • 42d8291 Release: v1.4 (#5732)
                  • \n
                • 63a78d9 [GKD] Fix Liger fused JSD path computing wrong loss (#5731)
                  • \n
                • =0.17.0 to >=0.17.0,<0.20.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [deepspeed](https://github.com/deepspeedai/DeepSpeed) to permit the latest version.\n
                  \nRelease notes\n

                  Sourced from deepspeed's releases.

                  \n
                  \n

                  v0.19.0

                  \n

                  What's Changed

                  \n
                    \n
                  • Update version after latest release (v0.18.9) by @\u200bloadams in deepspeedai/DeepSpeed#7936
                    • \n
                  • Refactor consolidate transpose by @\u200bnathon-lee in deepspeedai/DeepSpeed#7934
                    • \n
                  • Fix/fix autotp universal checkpoint ci by @\u200btohtana in deepspeedai/DeepSpeed#7937
                    • \n
                  • Fix process hang in process-group shutdown by @\u200bFlamefire in deepspeedai/DeepSpeed#7941
                    • \n
                  • Zero3 defragment utility by @\u200bnathon-lee in deepspeedai/DeepSpeed#7940
                    • \n
                  • [SP] add SP deny list instead of allow by @\u200bkashif in deepspeedai/DeepSpeed#7887
                    • \n
                  • fix(zero): detach flat buffer to prevent autograd inplace error on CP\u2026 by @\u200bdelock in deepspeedai/DeepSpeed#7948
                    • \n
                  • Fix FPQuantizer build by @\u200bFlamefire in deepspeedai/DeepSpeed#7963
                    • \n
                  • Fix zero 1 and 2 CPU-offloaded gradient norm by @\u200balek6kun in deepspeedai/DeepSpeed#7967
                    • \n
                  • Fix overlap-comm buffer lifetimes by @\u200btohtana in deepspeedai/DeepSpeed#7965
                    • \n
                  • Fix DeepCompile+Z3 on PyTorch v2.9/2.10 by @\u200btohtana in deepspeedai/DeepSpeed#7951
                    • \n
                  • Fix WarmupCosineLR multi-group initialization by @\u200btohtana in deepspeedai/DeepSpeed#7969
                    • \n
                  • Enable PyTorch version selection for full test by @\u200btohtana in deepspeedai/DeepSpeed#7968
                    • \n
                  • fix(fp_quantizer): fix UB and negative shift warnings in fp_quantize_impl.cu by @\u200bCursx in deepspeedai/DeepSpeed#7973
                    • \n
                  • fix(op_builder): avoid duplicate/wrong -gencode flags by @\u200bCursx in deepspeedai/DeepSpeed#7974
                    • \n
                  • Rename dequantization template parameters by @\u200bFlamefire in deepspeedai/DeepSpeed#7976
                    • \n
                  • Avoid CUDA reinit error in CI tests by =0.64,<2\"]` optional extra in `pyproject.toml`\n\n## Tests\n\n32 unit tests across client/cluster/cloud, Modal SDK fully mocked \u2014 no credentials or network required.\n\n## Related issues\n\nPart of LOU-1827.\n\n\n> [!NOTE]\n> **Liberate**\n> Risk: low\n\n", "merged_at": "2026-05-12T17:33:00Z"} {"number": 2442, "title": "feat(launcher): add provider-agnostic ClusterNotFoundError", "files": ["src/oumi/core/launcher/__init__.py", "src/oumi/core/launcher/exceptions.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description\n\nLifts SkyPilot's `sky.exceptions.ClusterDoesNotExist` into a generic `oumi.core.launcher.ClusterNotFoundError` so call sites depending on the launcher abstraction don't have to import provider-specific exceptions.\n\n## How\n\n- New `oumi/core/launcher/exceptions.py` with `LauncherError` (base) and `ClusterNotFoundError`\n- Re-exports through `oumi.core.launcher` package\n\n## Why\n\nFoundational for the upcoming Modal launcher (next PR), and for any future provider surfacing missing-cluster lookups. Today downstream code does `from sky.exceptions import ClusterDoesNotExist` to handle a launcher-abstract failure mode \u2014 that leaks SkyPilot into provider-agnostic call sites.\n\n## Related\n\nPart of LOU-1827 (oumi-side, exception-only split).", "merged_at": "2026-05-11T15:28:45Z"} {"number": 2439, "title": "[config] fix qwen3-vl configs", "files": ["configs/recipes/vision/qwen3_vl_2b/sft/2b_instruct_fft_train.yaml", "configs/recipes/vision/qwen3_vl_2b/sft/2b_instruct_lora_train.yaml", "configs/recipes/vision/qwen3_vl_30b_a3b/sft/30b_a3b_instruct_fft_train.yaml", "configs/recipes/vision/qwen3_vl_30b_a3b/sft/30b_a3b_instruct_lora_train.yaml", "configs/recipes/vision/qwen3_vl_4b/sft/4b_instruct_fft_train.yaml", "configs/recipes/vision/qwen3_vl_4b/sft/4b_instruct_lora_train.yaml"], "area": "configs", "area_votes": {"configs": 6}, "body": "# Description\r\n\r\n\r\n\r\n- [config] fix qwen3-vl configs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-05-08T16:26:49Z"} {"number": 2438, "title": "[tests] add coverage configuration and coverage-unit Make target", "files": ["Makefile", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "\r\n\r\n- [tests] add coverage configuration and coverage-unit Make target\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-05-08T16:28:05Z"} {"number": 2436, "title": "docs: add v0.8 release to news section", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Adds a news entry for the [v0.8 release](https://github.com/oumi-ai/oumi/releases/tag/v0.8). Entry synthesized from release notes.", "merged_at": "2026-05-08T00:05:19Z"} {"number": 2433, "title": "[tests] update tune e2e test", "files": ["configs/recipes/smollm/tuning/135m/tune.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- update tune e2e test\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-05-07T21:46:31Z"} {"number": 2431, "title": "[nit] improve user-facing error messages for synthesis param validation", "files": ["src/oumi/core/configs/params/synthesis_params.py", "tests/unit/core/configs/params/test_synthesis_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "Replace internal class-prefixed error messages with actionable, user-friendly messages that explain what went wrong and how to fix it.\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-05-07T19:33:40Z"} {"number": 2427, "title": "Add AtlasCloud inference config", "files": ["configs/apis/atlascloud/infer_deepseek_v3.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "Add AtlasCloud inference config", "merged_at": "2026-05-07T19:33:40Z"} {"number": 2426, "title": "[Inference] Add support for guided decoding in AnthropicInferenceEngine", "files": ["src/oumi/inference/anthropic_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-05-06T04:35:28Z"} {"number": 2425, "title": "feat(deploy): expose replica_count + raw accelerator_type on Endpoint", "files": ["src/oumi/deploy/base_client.py", "src/oumi/deploy/fireworks_client.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Description\r\n\r\nAdds two optional fields to the `Endpoint` dataclass populated from `GatewayDeployment`:\r\n\r\n- `replica_count: int | None` \u2014 live replica count (distinct from `autoscaling.min/max_replicas`).\r\n- `accelerator_type: str | None` \u2014 raw upstream `GatewayAcceleratorType` enum value (e.g. `\"NVIDIA_A100_80GB\"`).\r\n\r\n## Related issues\r\n\r\nLOU-1167\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\n`oumi-ai/oumi-staff`", "merged_at": "2026-05-05T18:04:30Z"} {"number": 2423, "title": "[deps] only allow explicit pre-release packages", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [deps] only allow explicit pre-release packages\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-05-05T19:41:00Z"} {"number": 2422, "title": "[Det Env v2] Phase 2: Planner guided JSON decoding", "files": ["src/oumi/core/synthesis/conversation_synthesizer.py", "src/oumi/core/types/conversation.py", "tests/unit/core/synthesis/test_conversation_synthesizer.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Description\n\nPart of the **Det Env v2** PR series \u2014 **Phase 2 of 7**.\n\n**What changed**: Switch the planner to JSON guided decoding against a Pydantic-derived `PLANNER_JSON_SCHEMA`. Planner output now arrives as a structured `{\"turns\": [...]}` object instead of markdown-fenced JSON.\n\n**Why**: Markdown-fenced output is brittle \u2014 truncated fences and trailing prose force a re-parse loop and add latency. Guided decoding lets the inference engine constrain the output directly so the planner returns valid JSON on the first attempt.\n\n## What's in scope\n\n- **`PlannerOutput` / `PlannedTurn`** Pydantic models in `core.types.conversation`, with a derived `PLANNER_JSON_SCHEMA: dict` constant. The schema enforces a `{\"turns\": [{\"turn\": int, \"instruction\": str}, ...]}` shape.\n- **`ConversationSynthesizer._planner_inference_config()`** \u2014 returns a copy of the user's inference config with `GuidedDecodingParams(json=PLANNER_JSON_SCHEMA)` set on the planner call only. The original config is never mutated, so per-turn (non-planner) inference stays unconstrained.\n- **`_create_planner_prompt`** updated to emit the wrapped object form with no markdown fences, matching the schema.\n- **`_parse_plan`** reads the wrapped object form first and falls back to a bare list for backward compatibility.\n- **`_generate_plan`** calls `_planner_inference_config()` instead of passing the raw config.\n- **Test fixture** `mock_inference_config` switched from `Mock(spec=InferenceConfig)` to a real `InferenceConfig` so `dataclasses.replace` works in the new helper.\n- New tests: `test_parse_plan_unwraps_object_form`, `test_generate_plan_uses_planner_only_guided_decoding` (verifies planner gets guided decoding, turn calls don't, and the original config is left untouched).\n\n## Notes\n\n- 447 tests pass across `tests/unit/core/synthesis/`, `tests/unit/core/configs/params/`, `tests/unit/utils/test_str_utils.py`.\n- `_planner_inference_config` is a 4-line `dataclasses.replace` chain \u2014 both `InferenceConfig.generation` and `GenerationParams` are guaranteed to be present (non-optional fields with defaults), so no `getattr` / isinstance defensive code.\n\n## Series context\n\nThis is PR 2 of a 7-PR series (rebooted from the original 8-PR chain #2386\u2013#2411). Each PR targets `main` directly, opened one at a time as predecessors land.\n\n1. ~~Tool schema unification & validation infrastructure~~ \u2192 **#2421 (merged)**\n2. **Planner: guided JSON decoding** \u2190 this PR\n3. Grounding system: types + environment integration\n4. Grounding in the synthesis pipeline\n5. Agentic tool-call loop in synthesizer\n6. Structured tool-message projection on output\n7. Library tool-use synthesis example + end-to-end test\n\n## Related issues\n\nNone.\n\n## Before submitting\n- [x] Tests added (`test_parse_plan_unwraps_object_form`, `test_generate_plan_uses_planner_only_guided_decoding`).\n", "merged_at": "2026-05-05T17:34:41Z"} {"number": 2419, "title": "Fix Anthropic multimodal image block shape", "files": ["src/oumi/inference/anthropic_inference_engine.py", "tests/unit/inference/test_anthropic_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n`AnthropicInferenceEngine._convert_conversation_to_api_input` delegated message construction to the inherited `_get_list_of_message_json_dicts` helper from `RemoteInferenceEngine`, which is the OpenAI-compatible base class. That helper emits OpenAI-shape blocks for image content:\r\n\r\n```json\r\n{\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/png;base64,<...>\"}}\r\n```\r\n\r\nAnthropic's `/v1/messages` endpoint does **not** accept `image_url` blocks \u2014 it requires:\r\n\r\n```json\r\n{\"type\": \"image\", \"source\": {\"type\": \"base64\", \"media_type\": \"...\", \"data\": \"<...>\"}}\r\n```\r\n\r\nSo multimodal Anthropic requests were silently broken pre-fix. Even if Anthropic had accepted the OpenAI shape, the helper hardcoded the MIME prefix to `image/png` for every image, which would corrupt JPEG/GIF/WebP payloads.\r\n\r\n## What changed\r\n\r\nReplace the delegation with a per-engine block builder that emits Anthropic-shape blocks:\r\n\r\n- `_messages_to_anthropic_blocks` \u2014 turn assembly: groups adjacent same-role messages into one turn (Anthropic requires alternating user/assistant), and collapses single-text turns originating from `content: str` back to a primitive-string content shape so the wire payload stays bit-identical to the prior non-multimodal request body.\r\n- `_content_to_anthropic_blocks` / `_content_item_to_anthropic_block` \u2014 per-message and per-content-item translation. Handles text, `IMAGE_BINARY`, `IMAGE_PATH` (loads bytes once), and `IMAGE_URL` (uses `{\"type\": \"url\", ...}` source for unfetched URLs, base64 source when bytes are present).\r\n- `_detect_image_media_type` \u2014 magic-byte sniffer covering the four formats Anthropic supports (`image/png`, `image/jpeg`, `image/gif`, `image/webp`), with `image/png` fallback + warning for unrecognized data.\r\n\r\n## Related issues\r\n\r\nhttps://linear.app/oumi/issue/OPE-1414\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.", "merged_at": "2026-05-06T17:42:22Z"} {"number": 2418, "title": "Update datasets requirement from <4.8.5,>=3.2 to >=3.2,<4.8.6", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [datasets](https://github.com/huggingface/datasets) to permit the latest version.\n
                    \nRelease notes\n

                    Sourced from datasets's releases.

                    \n
                    \n

                    4.8.5

                    \n

                    Main bug fixes

                    \n\n

                    Other improvements and bug fixes

                    \n\n

                    New Contrib", "merged_at": "2026-05-04T15:12:04Z"} {"number": 2416, "title": "Update typer requirement from <0.24.2 to <0.25.2", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [typer](https://github.com/fastapi/typer) to permit the latest version.\n
                    \nRelease notes\n

                    Sourced from typer's releases.

                    \n
                    \n

                    0.25.1

                    \n

                    Features

                    \n\n

                    Internal

                    \n\n
                    \n
                    \n
                    \nChangelog\n

                    Sourced from typer's changelog.

                    \n
                    \n

                    0.25.1 (2026-04-30)

                    \n

                    Features

                    \n\n

                    Internal

                    \n\n

                    0.25.0 (2026-04-26)

                    \n

                    Features

                    \n
                      \n
                    • \ud83d\udeb8 Don't truncate code lines in traceback when formatted with Rich. PR #1695 by @\u200bYuriiMotov.
                      • \n
                    \n

                    0.24.2 (2026-04-22)

                    \n

                    Fixes

                    \n
                      \n
                    • \ud83d\udc1b Ensure that typer.launch forwards correctly when launching a file. PR #1708 by @\u200bsvlandeg.
                      • \n
                    \n

                    Refactors

                    \n\n

                    Docs

                    \n\n

                    Internal

                    \n
                      \n
                    • \u2b06 Bump mypy from 1.20.1 to 1.20.2. PR ", "merged_at": "2026-05-04T15:09:16Z"} {"number": 2415, "title": "Update liger-kernel requirement from <0.8,>=0.6 to >=0.6,<0.9", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [liger-kernel](https://github.com/linkedin/Liger-Kernel) to permit the latest version.\n
                      \nRelease notes\n

                      Sourced from liger-kernel's releases.

                      \n
                      \n

                      v0.8.0

                      \n

                      Liger-Kernel v0.8.0

                      \n

                      Highlights \u2728

                      \n

                      \ud83d\ude80 MoE Training Acceleration via LigerExperts

                      \n

                      A new fused MoE expert kernel (LigerExperts, backed by LigerFusedMoEFunction) inspired by SonicMoE (arXiv:2512.14080). It replaces the eager per-expert loop in HuggingFace MoE blocks with a Triton grouped-GEMM + fused SwiGLU + token aggregation path, with a memory-efficient backward.

                      \n

                      Qwen3-30B-A3B fine-tuning on 2 \u00d7 H100 (max seq len 32768, BS=1, GA=8): 8.24\u00d7 tokens/sec, 8.19\u00d7 training step, 4.10\u00d7 eval, ~1% memory savings.

                      \n

                      Auto-patched into Mixtral, Qwen3-MoE, Qwen3-Next, Qwen3.5-MoE, Qwen3-VL-MoE, GLM4V-MoE, and HunYuan-MoE-V1. (#1179, #1192)

                      \n

                      \ud83e\udd16 Claude Code Skills for Kernel Authoring (.claude/skills/)

                      \n

                      Three first-party Claude Code skills now ship with the repo to make extending Liger-Kernel dramatically easier:

                      \n
                        \n
                      • liger-kernel-dev (#1170) \u2014 builds a production-ready Triton kernel end-to-end from a PyTorch op (file, URL, snippet, or NL): ops, module wrapper, functional API, tests, benchmark, exports.
                        • \n
                      • liger-autopatch (#1167, #1177) \u2014 adds Liger support for a new HuggingFace model: reads modeling_*.py, generates lce_forward, monkey-patch entry, and convergence tests.
                        • \n
                      • liger-kernel-perf (#1185) \u2014 profiles an existing kernel, generates and benchmarks optimization variants (Ampere / Hopper / Blackwell-aware), applies the winner.
                        • \n
                      \n

                      Several PRs in this release were authored using these skills (e.g. #1165, #1166, #1171, #1187).

                      \n

                      \ud83e\udde9 New Model Support

                      \n
                        \n
                      • Qwen3.5 MoE (#1109) and Qwen3.5 dense (#1123)
                        • \n
                      • Qwen3.5 multimodal \u2014 Qwen3_5ForConditionalGeneration (#1150)
                        • \n
                      • Nemotron (#1165), with Liger ReLU\u00b2 wired in (#1176)
                        • \n
                      • Ministral (#1166)
                        • \n
                      • Gemma 4 dense text, 31B-targeted (#1196)
                        • \n
                      • Falcon H1 SwiGLU multipliers (#1201)
                        • \n
                      \n

                      Ascend NPU Backend Support

                      \n

                      L", "merged_at": "2026-05-04T15:11:47Z"} {"number": 2408, "title": "deps: bump torchao lower bound to >=0.16 for peft>=0.19 compatibility", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "## Summary\r\n- Tighten `torchao` lower bound from `>=0.12` to `>=0.16` so resolvers can't pick a version that fails peft 0.19's runtime check.\r\n\r\n## Reason\r\n- `peft>=0.17,<0.20` allows peft 0.19, which calls `is_torchao_available()` and raises `ImportError` when `torchao<0.16`.\r\n- The previous `>=0.12,<0.17` constraint let uv resolve to `torchao==0.15.0`\r\n- The comment above the line already documented the requirement; this PR enforces it.\r\n\r\n## Test plan\r\n- [ ] Bump `OUMI_REPO_COMMIT_HASH` in `oumi-ai/api` once merged, redeploy worker, retry a LoRA training run.\r\n\r\nFixes oumi-ai/api LoRA training failures tracked in [LOU-1717](https://linear.app/oumi/issue/LOU-1717).", "merged_at": "2026-05-01T18:56:45Z"} {"number": 2407, "title": "feat(synthesis): allow reusing AttributeSynthesizer helpers without loading model", "files": ["src/oumi/core/synthesis/attribute_synthesizer.py", "tests/unit/core/synthesis/test_attribute_synthesizer.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-05-01T12:33:58Z"} {"number": 2405, "title": "[docs] fix sphinx build warnings", "files": ["docs/conf.py", "src/oumi/evaluation/__init__.py"], "area": "docs", "area_votes": {"docs": 1}, "body": "\r\n\r\n# Description\r\n\r\n\r\n\r\n- Update PyTorch intersphinx URL to docs.pytorch.org/docs/main since the old /stable redirect target now 404s, restoring all torch.* references.\r\n- Extend the existing log filter to swallow \"more than one target found\" warnings from oumi.core.configs re-exports (ModelParams, TrainerType, TrainingParams), which are documented at both package and submodule paths.\r\n- Add an empty oumi/evaluation/__init__.py so sphinx-apidoc includes it in the parent toctree (namespace packages with no top-level .py files were skipped, orphaning oumi.evaluation.rst).\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-04-30T03:03:40Z"} {"number": 2404, "title": "Update transformers to 5.7.0", "files": ["configs/recipes/smollm/tuning/135m/tune.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml", "pyproject.toml", "src/oumi/core/configs/internal/supported_models.py"], "area": "configs", "area_votes": {"configs": 3, "infra": 1}, "body": "- Bumped transformers ceiling to include 5.7.0, and torchao to include 0.16.\r\n- Removed `eval_mean_token_accuracy` metric from `configs/recipes/smollm/tuning/135m/tune.yaml`. There's now a bug in this metric being surfaced by transformers/trl.\r\n- Updated Qwen2 VL processor\r\n- Enabled ddp_find_unused_parameters for llava\r\n\r\nRan integration tests", "merged_at": "2026-04-30T17:19:18Z"} {"number": 2402, "title": "[deps] update vllm requirement from <0.20,>=0.10 to >=0.14,<0.21", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-04-29T19:51:59Z"} {"number": 2399, "title": "feat(deploy): typed FireworksConflictError", "files": ["src/oumi/deploy/fireworks_client.py", "src/oumi/deploy/fireworks_errors.py", "tests/unit/deploy/test_fireworks_client.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Description\r\n\r\nAdds `FireworksConflictError(DeployApiError)`, raised inline by `_create_model_resource` on HTTP 409. Lets concurrent callers branch on \"model already exists\" cleanly (e.g. wait via `get_model` polling) instead of getting the bare `DeployApiError` we raised before. Existing `except DeployApiError` handlers still catch it. Also drops the misleading \"delete it manually\" ERROR log on 409 because under concurrency the conflict is benign.\r\n\r\n## Related issues\r\n\r\nTowards [LOU-1578](https://linear.app/oumi/issue/LOU-1578)\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.", "merged_at": "2026-04-29T04:30:57Z"} {"number": 2398, "title": "fix(deploy): classify gpt-oss 'requires one of' as FireworksUnsupportedHardwareError", "files": ["src/oumi/deploy/fireworks_errors.py", "tests/unit/deploy/test_fireworks_errors.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\nFireworks rejects `gpt-oss` deployments on incompatible accelerators with the detail string:\r\n\r\n```\r\ninvalid deployment: model type gpt_oss requires one of: NVIDIA_H100_80GB, NVIDIA_B200_180GB, NVIDIA_GB200, NVIDIA_B300_288GB, NVIDIA_H200_141GB\r\n```\r\n\r\n`classify_fireworks_invalid_request` only matched the older `\"is not supported on \"` phrasing, so this variant fell through to the base `DeployInvalidRequestError`. Downstream consumers gate hardware fallback on `FireworksUnsupportedHardwareError`, so an A100 attempt for `gpt-oss-20b` failed terminally instead of falling back to H100.\r\n\r\n## Related issues\r\n\r\nFixes [LOU-1658](https://linear.app/oumi/issue/LOU-1658)\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2026-04-28T20:43:38Z"} {"number": 2394, "title": "Update transformers requirement from <5.6,>=4.57 to >=4.57,<5.7", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [transformers](https://github.com/huggingface/transformers) to permit the latest version.\n

                      \nRelease notes\n

                      Sourced from transformers's releases.

                      \n
                      \n

                      Patch release v5.6.2

                      \n

                      Qwen 3.5 and 3.6 MoE (text-only) were broken when using with FP8. It should now work again with this :saluting_face:

                      \n\n

                      Full Changelog: https://github.com/huggingface/transformers/compare/v5.6.1...v5.6.2

                      \n
                      \n
                      \n
                      \nCommits\n\n
                      \n
                      \n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
                      \nDependabot commands and options\n
                      \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot show This will update automatically on new commits.", "merged_at": "2026-04-22T01:42:52Z"} {"number": 2383, "title": "feat(deploy): support adapter uploads and add get_model", "files": ["src/oumi/deploy/fireworks_client.py", "tests/unit/deploy/test_fireworks_client.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n1. **Extend `upload_model_with_resolver` to support `ModelType.ADAPTER`.** Previously, the `FileResolver`-based upload path only handled FULL models and had a TODO for adapter support\r\n\r\n2. **Add `get_model(model_id) -> GatewayModel | None`.** Existing `get_model_status` only returns the state string; callers that need to inspect `kind`, `peft_details`, or other resource fields have to re-parse the raw response themselves.\r\n\r\n## Related issues\r\n\r\nTowards LOU-1573\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.", "merged_at": "2026-04-20T21:50:38Z"} {"number": 2381, "title": "Update pydantic requirement from <2.13,>=2.11 to >=2.11,<2.14", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [pydantic](https://github.com/pydantic/pydantic) to permit the latest version.\n
                      \nChangelog\n

                      Sourced from pydantic's changelog.

                      \n
                      \n

                      v2.13.2 (2026-04-17)

                      \n

                      GitHub release

                      \n

                      What's Changed

                      \n

                      Fixes

                      \n
                        \n
                      • Fix ValidationInfo.field_name missing with model_validate_json() by @\u200bViicos in #13084
                        • \n
                      \n

                      v2.13.1 (2026-04-15)

                      \n

                      GitHub release

                      \n

                      What's Changed

                      \n

                      Fixes

                      \n\n

                      v2.13.0 (2026-04-13)

                      \n

                      GitHub release

                      \n

                      The highlights of the v2.13 release are available in the blog post.\nSeveral minor changes (considered non-breaking changes according to our versioning policy)\nare also included in this release. Make sure to look into them before upgrading.

                      \n

                      This release contains the updated pydantic.v1 namespace, matching version 1.10.26 which includes support for Python 3.14.

                      \n

                      What's Changed

                      \n

                      See the beta releases for all changes sinces 2.12.

                      \n

                      New Features

                      \n
                        \n
                      • Allow default factories of private attributes to take validated model data by @\u200bViicos in #13013
                        • \n
                      \n

                      Changes

                      \n\n

                      Fixes

                      \n
                        \n
                      • Change type of Any when synthesizing _build_sources for BaseSettings.__init__() signature in the mypy plugin by @\u200bViicos in #13049
                        • \n
                      • Fix model equality when using runtime extra configuration by @\u200bViicos in #13062
                        • \n
                      \n

                      Packaging

                      \n\n\n
                      \n

                      ... (truncated)

                      \n
                      \n
                      \nCommits\n
                        \n
                      • ca3ddd1 Prepare release v2.13.2
                        • \n
                      • 000e823 Fix ValidationInfo.field_name missing with model_validate_json()
                        • \n
                      • d45d8be Prepare release 2.13.1
                        • \n
                      • 54aca60 Fix ValidationInfo.data=0.17 to >=0.17,<0.20", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [peft](https://github.com/huggingface/peft) to permit the latest version.\n
                        \nRelease notes\n

                        Sourced from peft's releases.

                        \n
                        \n

                        v0.19.1

                        \n

                        A small patch release containing these fixes:

                        \n\n

                        Full Changelog: https://github.com/huggingface/peft/compare/v0.19.0...v0.19.1

                        \n
                        \n
                        \n
                        \nCommits\n\n
                        \n
                        \n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
                        \nDependabot commands and options\n
                        \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dep", "merged_at": "2026-04-29T04:22:27Z"} {"number": 2377, "title": "fix(analyze): preserve sample indices and fix threshold truthiness", "files": ["src/oumi/analyze/testing/batch_engine.py", "src/oumi/analyze/testing/engine.py", "tests/unit/analyze/test_testing_engine.py"], "area": "evaluation", "area_votes": {"evaluation": 2}, "body": "# Description\n\nPart of the **analyze v2** split of #2370 (Phase 3 of 4). Standalone fixes \u2014 not dependent on PRs #2375 / #2376.\n\nTwo related fixes to the analyze test engine.\n\n## 1. Preserve original sample indices\n\n`TestEngine._extract_metric_values` now returns `(original_index, value)` pairs so that `sample_indices` and `all_affected_indices` on the resulting `TestResult` point to real conversation positions in the dataset.\n\n**Why a sample can be \"missing\" a metric** (i.e., why `_get_nested_value` returns `None`):\n\n- The analyzer's result model declares the field as optional and leaves it `None` for that sample (e.g., couldn't compute on an empty conversation, or a conditional code path that doesn't produce the field).\n- The metric lives in a `CustomMetricResult.values` dict whose keys vary per sample (custom metric functions are user-written and may emit different keys per input).\n- A nested dict path has a missing intermediate key.\n\n**The bug.** Previously, the code enumerated `analyzer_results` and dropped Nones, then re-numbered the survivors `0..N-1`. With 100 samples where #5 and #42 lacked the metric, indices `5..98` in the result list pointed to dataset samples `6..99`. Any downstream \"drop these flagged rows\" tooling consuming `all_affected_indices` deleted the wrong rows.\n\n## 2. Fix threshold truthiness bug\n\nThe result-builder used:\n\n```python\nthreshold=test.max_percentage or test.min_percentage,\n```\n\n`or` returns the first truthy operand, and **`0.0` is falsy in Python**. So a \"zero tolerance\" rule like:\n\n```yaml\n- id: no_pii_allowed\n type: threshold\n metric: pii.count\n operator: \">\"\n value: 0\n max_percentage: 0.0 # fail if ANY sample has pii.count > 0\n```\n\n\u2026produced `0.0 or None` \u2192 `None` on the `TestResult.threshold` field. The CLI table, `--output-json`, and any downstream tool reading the result then displayed `threshold: \u2014` instead of `0.0`, hiding the user's actual policy.\n\nThe \"both set\" variant is sneakier: with `max_percentage=0.0` and `min_percentage=50.0`, `0.0 or 50.0` \u2192 `50.0` \u2014 confidently misattributing the threshold to the min rule when the max rule fired.\n\nThe pass/fail decision itself wasn't broken (that path uses explicit `is not None` checks). What was broken was the *reported* threshold. Replaced with explicit `is not None` in both the in-memory `TestEngine` and the incremental `BatchTestEngine`:\n\n```python\nthreshold=(\n test.max_percentage\n if test.max_percentage is not None\n else test.min_percentage\n),\n```\n\n## Tests\n\n`tests/unit/analyze/test_testing_engine.py` adds three regression tests with docstrings that quote the buggy code so the intent survives future rewrites:\n\n- `test_sample_indices_map_to_original_positions_when_values_are_missing`\n- `test_zero_tolerance_threshold_is_reported_as_zero_not_none`\n- `test_zero_tolerance_threshold_picks_max_when_both_are_set`\n\n## Related issues\n\nLinear Issue: [OPE-1868](https://linear.app/oumi-team/issue/OPE-1868)\nTowards OPE-1868\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n\ud83e\udd16 Generated with [Claude Code](https://claude.com/claude-code)", "merged_at": "2026-04-29T20:43:15Z"} {"number": 2374, "title": "docs: add user guide for the Oumi MCP server", "files": ["docs/index.md", "docs/user_guides/mcp.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "## Description\n\nAdds `docs/user_guides/mcp.md` \u2014 a dedicated user guide for the Oumi MCP server. Also wires it into the User Guides toctree in `docs/index.md`.\n\nThis is a targeted replacement for the MCP section currently proposed in #2372. In that PR the MCP page lists tool names (`launch_job`, `poll_status`, `fetch_logs`, `list_running_jobs`, `list_completed_jobs`) that don't exist in `src/oumi/mcp/server.py`, omits 6 tools that do exist, describes a \"Prompts\" primitive the server doesn't register, and documents an `OUMI_LOG_LEVEL=DEBUG` env var that no code reads. A user who tries to follow the page from an MCP-capable assistant hits \"unknown tool\" errors immediately.\n\nThis PR rewrites the page from scratch against the actual server:\n\n- Accurate tool table (15 tools) that matches the `@mcp.tool()` registrations in `src/oumi/mcp/server.py`.\n- Accurate resource table covering both `guidance://*` and `jobs://*` URIs.\n- Per-client setup snippets for Claude Desktop, Claude Code, and Cursor, plus a note on selecting a specific Python environment.\n- A \"When to use it\" framing up front so users know whether they want the MCP server or just the CLI.\n- A representative end-to-end cloud-training flow via the actual tool names.\n- Safety section documenting the dry-run default, automatic pre-flight on cloud launches, and the `down_cluster` confirmation gate.\n- Troubleshooting grounded in real behavior (stdio + stderr logging, `sky check`, in-process job registry, client log paths).\n- Links to `src/oumi/mcp/` so contributors can jump to the source.\n\nOnce this lands, #2372 can drop its MCP section to avoid conflicts.\n\n## Related issues\n\nFixes # (none \u2014 doc-only)\n\n## Before submitting\n\n- [x] This PR only changes documentation.\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed? (N/A \u2014 docs only)", "merged_at": "2026-04-17T21:23:14Z"} {"number": 2373, "title": "feat(deploy): surface Fireworks GatewayStatus on Endpoint", "files": ["src/oumi/deploy/base_client.py", "src/oumi/deploy/fireworks_client.py", "tests/unit/deploy/test_fireworks_client.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Description\r\n\r\nFireworks reports scheduling failures (e.g. \\\"no A100 capacity\\\") via `GatewayDeployment.status` with `code=RESOURCE_EXHAUSTED` while the deployment state stays in `CREATING`. Callers currently have no visibility into that signal and must wait out their full poll budget before timing out.\r\n\r\nThis change adds optional `status_code` / `status_message` fields to the `Endpoint` dataclass and populates them from `deployment.status` in the Fireworks client's `_parse_deployment`. Consumers (e.g. the api worker's poll activity) can now inspect these fields and fail fast with an actionable error instead of polling until timeout.\r\n\r\nParasail endpoints keep the new fields at their `None` defaults \u2014 no parasail-side change is required.\r\n\r\n## Related issues\r\n\r\nFixes (Linear issue, ex. LOU-123)\r\n\r\nTowards LOU-1560\r\n\r\n## Before submitting\r\n\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n- [x] Did you run relevant e2e/integration tests? (manually verified end-to-end from oumi-ai/api \u2014 Fireworks endpoint stuck in STARTING with RESOURCE_EXHAUSTED status code now surfaces via Endpoint.status_code)", "merged_at": "2026-04-17T21:19:11Z"} {"number": 2372, "title": "[docs] refresh docs", "files": ["docs/index.md", "docs/user_guides/deploy.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/judge/judge.md", "docs/user_guides/launch/launch.md", "docs/user_guides/synth.md", "docs/user_guides/train/training_methods.md", "src/oumi/analyze/__init__.py", "src/oumi/core/configs/params/synthesis_params.py", "src/oumi/utils/packaging.py", "src/oumi/utils/str_utils.py"], "area": "docs", "area_votes": {"docs": 7, "configs": 1, "other": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-04-30T02:12:12Z"} {"number": 2371, "title": "New README.md header", "files": ["README.md", "docs/_static/logo/header_readme.svg"], "area": "docs", "area_votes": {"docs": 2}, "body": "New readme header that rebrand as Oumi OSS", "merged_at": "2026-04-16T18:57:19Z"} {"number": 2365, "title": "feat(deploy): support caller-supplied deploymentId in create_endpoint", "files": ["src/oumi/deploy/fireworks_client.py", "tests/unit/deploy/test_fireworks_client.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "## Summary\r\n\r\nAdds an optional `endpoint_id` parameter to `FireworksDeploymentClient.create_endpoint`. When provided, it is passed as the `?deploymentId=...` [query parameter](https://docs.fireworks.ai/api-reference/create-deployment#parameter-deployment-id).\r\n\r\nAlso narrows `FireworksDeploymentClient.__aenter__` return type to `FireworksDeploymentClient` so consumers using `async with` get full access to subclass methods without a cast, needed now that callers reference the new `endpoint_id` kwarg via the context manager.\r\n\r\nTowards LOU-1161.", "merged_at": "2026-04-13T17:15:28Z"} {"number": 2364, "title": "Update kernels requirement from <0.13,>=0.11 to >=0.11,<0.14", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on kernels to permit the latest version.\n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
                        \nDependabot commands and options\n
                        \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n\n\n
                        ", "merged_at": "2026-04-14T21:44:16Z"} {"number": 2363, "title": "Update uvicorn requirement from <0.44.0 to <0.45.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [uvicorn](https://github.com/Kludex/uvicorn) to permit the latest version.\n
                        \nRelease notes\n

                        Sourced from uvicorn's releases.

                        \n
                        \n

                        Version 0.44.0

                        \n

                        What's Changed

                        \n\n

                        Full Changelog: https://github.com/Kludex/uvicorn/compare/0.43.0...0.44.0

                        \n
                        \n
                        \n
                        \nChangelog\n

                        Sourced from uvicorn's changelog.

                        \n
                        \n

                        0.44.0 (April 6, 2026)

                        \n

                        Added

                        \n
                          \n
                        • Implement websocket keepalive pings for websockets-sansio (#2888)
                          • \n
                        \n

                        0.43.0 (April 3, 2026)

                        \n

                        You can quit Uvicorn now. We heard you, @\u200bpamelafox - all 47 of your Ctrl+C's (thanks for flagging it, and thanks to @\u200btiangolo for the fix \ud83d\ude4f). See the tweet.

                        \n

                        Changed

                        \n
                          \n
                        • Emit http.disconnect ASGI receive() event on server shutting down for streaming responses (#2829)
                          • \n
                        • Use native context parameter for create_task on Python 3.11+ (#2859)
                          • \n
                        • Drop cast in ASGI types (#2875)
                          • \n
                        \n

                        0.42.0 (March 16, 2026)

                        \n

                        Changed

                        \n
                          \n
                        • Use bytearray for request body accumulation to avoid O(n^2) allocation on fragmented bodies (#2845)
                          • \n
                        \n

                        Fixed

                        \n
                          \n
                        • Escape brackets and backslash in httptools HEADER_RE regex (#2824)
                          • \n
                        • Fix multiple issues in websockets sans-io implementation (#2825)
                          • \n
                        \n

                        0.41.0 (February 16, 2026)

                        \n

                        Added

                        \n
                          \n
                        • Add --limit-max-requests-jitter to stagger worker restarts (#2707)
                          • \n
                        • Add socket path to scope["server"] (#2561)
                          • \n
                        \n

                        Changed

                        \n
                          \n
                        • Rename LifespanOn.error_occured to error_occurred (#2776)
                          • \n
                        \n

                        Fixed

                        \n
                          \n
                        • Ignore permission denied errors in watchfiles reloader (#2817)
                          • \n
                        • Ensure lifespan shutdown runs when should_exit is set during startup (#2812)
                          • \n
                        • Reduce the log level of 'request limit exceeded' messages (#2788)
                          • \n
                        \n

                        0.40.0 (December 21, 2025)

                        \n

                        Remove

                        \n
                          \n
                        • Drop support for Python 3.9 (#2772)
                          • \n
                        \n\n
                        \n

                        ... (truncated)

                        \n
                        \n
                        \nCommits\n
                          \n
                        • =0.24 to >=0.24,<1.2", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [trl](https://github.com/huggingface/trl) to permit the latest version.\n
                          \nRelease notes\n

                          Sourced from trl's releases.

                          \n
                          \n

                          v1.1.0

                          \n

                          Features

                          \n

                          DistillationTrainer for efficient on-policy distillation

                          \n

                          Read the blog post: https://huggingface.co/spaces/HuggingFaceTB/trl-distillation-trainer

                          \n

                          \"off_vs_on_policy_distillation

                          \n

                          The new DistillationTrainer implements on-policy knowledge distillation as described in On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. It extends the ideas from the GKDTrainer with three key optimizations: a generation buffer that decouples the training microbatch size from the generation batch size (up to 40x speedup), external teacher server support so the teacher doesn't need to fit on training GPUs, and binary-encoded logprob payloads that shrink transfer payloads by ~5x.

                          \n
                          from datasets import load_dataset\r\nfrom trl.experimental.distillation import DistillationConfig, DistillationTrainer\r\n
                          dataset = load_dataset("openai/gsm8k", "main", split="train")\ndataset = dataset.map(\nlambda x: {"messages": [{"role": "user", "content": x["question"]}]},\nremove_columns=dataset.column_names,\n)
                          \n
                          trainer = DistillationTrainer(\nmodel="Qwen/Qwen2.5-1.5B-Instruct",\nteacher_model="Qwen/Qwen2.5-7B-Instruct",\nargs=DistillationConfig(\noutput_dir="results/distill-qwen-gsm8k",\nlmbda=1.0,                   # fully on-policy (student generates)\nbeta=1.0,                    # reverse KL\nteacher_model_init_kwargs={"torch_dtype": "bfloat16"},\n),\ntrain_dataset=dataset,\n)\ntrainer.train()\n

                          \n

                          by @\u200bcmpatino in huggingface/trl#5407, huggingface/trl#5500 and huggingface/trl#5501

                          \n

                          Chunked LM head for memory-efficient log-prob computation in AsyncGRPOTrainer

                          \n

                          AsyncGRPOTrainer now supports a chunked LM-head path that computes per-token log-probs and entropy via online logsumexp without materializing the full [N, V] logits tensor. Combined with completion_mask filtering to skip prompt tokens, this brings massive memory savings on long sequences \u2014 up to 44x lower peak-allocated memory on an 8192-token sequence:

                          \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
                          chunk_lm_head_sizePeak Alloc (GB)ReductionWall Time (ms)
                          None (baseline)18.551.00x808.7
                          40960.4244.32x459.0
                          81920.7624.34x393.0
                          \n

                          Enable it via the new chunk_lm_head_size option in AsyncGRPOConfig:

                          \n
                          </tr></table> \n
                          \n
                          \n

                          ... (truncated)

                          \n=0.10 to >=0.10,<0.20", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [vllm](https://github.com/vllm-project/vllm) to permit the latest version.\n
                          \nRelease notes\n

                          Sourced from vllm's releases.

                          \n
                          \n

                          v0.19.0

                          \n

                          vLLM v0.19.0

                          \n

                          Highlights

                          \n

                          This release features 448 commits from 197 contributors (54 new)!

                          \n
                            \n
                          • Gemma 4 support: Full Google Gemma 4 architecture support including MoE, multimodal, reasoning, and tool-use capabilities (#38826, #38847). Requires transformers>=5.5.0. We recommend using pre-built docker image vllm/vllm-openai:gemma4 for out of box usage.
                            • \n
                          • Zero-bubble async scheduling + speculative decoding: Async scheduling now supports speculative decoding with zero-bubble overlap, significantly improving throughput (#32951).
                            • \n
                          • Model Runner V2 maturation: MRV2 gains piecewise CUDA graphs for pipeline parallelism (#35162), spec decode rejection sampler with greedy/logprobs support (#37238, #37237), multi-modal embeddings for spec decode (#36097), streaming inputs (#37028), and EPLB support (#37488).
                            • \n
                          • ViT Full CUDA Graphs: Vision encoders (ViT) now support full CUDA graph capture for reduced overhead (#35963).
                            • \n
                          • General CPU KV cache offloading: A simple yet general CPU KV cache offloading mechanism for V1, with pluggable cache policy and block-level preemption handling (#37160, #37874, #34805, #36642, #37853).
                            • \n
                          • DBO (Dual-Batch Overlap) generalization: The microbatch optimization (DBO) now works with general models, not just specific architectures (#37926).
                            • \n
                          • NVIDIA B300/GB300 (SM 10.3) support: Allreduce fusion enabled by default with tuned all-reduce communicator (#37755, #37756).
                            • \n
                          • Transformers v5 compatibility: Broad compatibility fixes across many models for HuggingFace Transformers v5 (#37681, #38127, #38090, #38247, #38410).
                            • \n
                          \n

                          Model Support

                          \n
                            \n
                          • New architectures: Gemma 4 (#38826), Cohere ASR (#35809<", "merged_at": "2026-04-06T13:16:33Z"} {"number": 2351, "title": "Update uvicorn requirement from <0.43.0 to <0.44.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [uvicorn](https://github.com/Kludex/uvicorn) to permit the latest version.\n
                            \nRelease notes\n

                            Sourced from uvicorn's releases.

                            \n
                            \n

                            Version 0.43.0

                            \n

                            Changed

                            \n
                              \n
                            • Emit http.disconnect ASGI receive() event on server shutting down for streaming responses (#2829)
                              • \n
                            • Use native context parameter for create_task on Python 3.11+ (#2859)
                              • \n
                            • Drop cast in ASGI types (#2875)
                              • \n
                            \n
                            \n

                            Full Changelog: https://github.com/Kludex/uvicorn/compare/0.42.0...0.43.0

                            \n
                            \n
                            \n
                            \nChangelog\n

                            Sourced from uvicorn's changelog.

                            \n
                            \n

                            0.43.0 (April 3, 2026)

                            \n

                            You can quit Uvicorn now. We heard you, @\u200bpamelafox - all 47 of your Ctrl+C's (thanks for flagging it, and thanks to @\u200btiangolo for the fix \ud83d\ude4f). See the tweet.

                            \n

                            Changed

                            \n
                              \n
                            • Emit http.disconnect ASGI receive() event on server shutting down for streaming responses (#2829)
                              • \n
                            • Use native context parameter for create_task on Python 3.11+ (#2859)
                              • \n
                            • Drop cast in ASGI types (#2875)
                              • \n
                            \n

                            0.42.0 (March 16, 2026)

                            \n

                            Changed

                            \n
                              \n
                            • Use bytearray for request body accumulation to avoid O(n^2) allocation on fragmented bodies (#2845)
                              • \n
                            \n

                            Fixed

                            \n
                              \n
                            • Escape brackets and backslash in httptools HEADER_RE regex (#2824)
                              • \n
                            • Fix multiple issues in websockets sans-io implementation (#2825)
                              • \n
                            \n

                            0.41.0 (February 16, 2026)

                            \n

                            Added

                            \n
                              \n
                            • Add --limit-max-requests-jitter to stagger worker restarts (#2707)
                              • \n
                            • Add socket path to scope["server"] (#2561)
                              • \n
                            \n

                            Changed

                            \n
                              \n
                            • Rename LifespanOn.error_occured to error_occurred (#2776)
                              • \n
                            \n

                            Fixed

                            \n
                              \n
                            • Ignore permission denied errors in watchfiles reloader (#2817)
                              • \n
                            • Ensure lifespan shutdown runs when should_exit is set during startup (#2812)
                              • \n
                            • Reduce the log level of 'request limit exceeded' messages (#2788)
                              • \n
                            \n

                            0.40.0 (December 21, 2025)

                            \n

                            Remove

                            \n
                              \n
                            • Drop support for Python 3.9 (#2772)
                              • \n
                            \n

                            0.39.0 (December 21, 2025)

                            \n

                            Fixed

                            \n
                              \n
                            • Send close frame on ASGI return for WebSockets (=0.11.1 to >=0.11.1,<0.13", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [skypilot](https://github.com/skypilot-org/skypilot) to permit the latest version.\n
                              \nRelease notes\n

                              Sourced from skypilot's releases.

                              \n
                              \n

                              SkyPilot v0.12.0: Slurm Support, Job Groups for RL, Agent Skill, Recipes, Pool Autoscaling for Batch Inference, 7x Data Mounting, and More

                              \n

                              SkyPilot v0.12.0 brings major new capabilities: Slurm integration for running SkyPilot on existing Slurm clusters, Job Groups for heterogeneous parallel workloads like RL training, an Agent Skill that teaches AI coding agents to use SkyPilot, Recipes for sharing reusable YAML templates across teams, and significant Pool enhancements including autoscaling. This release also drops Python 3.7/3.8, with Python 3.9+ now required.

                              \n

                              Get it now with:

                              \n
                              uv pip install "skypilot[all]>=0.12.0"\r\n
                              \n

                              Or, upgrade your team SkyPilot API server:

                              \n
                              NAMESPACE=skypilot\r\nRELEASE_NAME=skypilot\r\nVERSION=0.12.0\r\n
                              helm repo update skypilot\nhelm upgrade -n $NAMESPACE $RELEASE_NAME skypilot/skypilot 
                              \n--set apiService.image=berkeleyskypilot/skypilot:$VERSION 
                              \n--version $VERSION --devel --reuse-values\n

                              \n

                              Deprecations & Breaking Changes:

                              \n
                                \n
                              • Dropped Python 3.7 and 3.8 support\u2014Python 3.9+ is now required (#8489).
                                • \n
                              • sky.jobs.queue(version=1) is deprecated and will be removed in v0.13. Use sky.jobs.queue(version=2) instead. The new version returns richer job metadata as dictionaries (#9118).
                                • \n
                              \n

                              Highlights

                              \n

                              [New] Slurm Support

                              \n

                              SkyPilot now supports connecting your Slurm clusters, bringing its unified interface to one of the most widely used job schedulers in high-performance computing (#5491, #8198, #8219, #8268, #8291, #8470, #8604, #8729, and 25+ additional PRs). Users can launch SkyPilot clusters and managed jobs on Slurm clusters with the same CLI and YAML they use for cloud and Kubernetes, enabling seamless workload portability across all AI infra.

                              \n\n

                              Key capabilities include:

                              \n
                                \n
                              • Multi-node clusters with partition-level resource management
                                • \n
                              • Container support via pyxis/enroot for reproducible environments
                                • \n
                              • SSH ProxyJump for clusters behind bastion hosts
                                • \n
                              • GPU availability viewing with sky show-gpus for Slurm partitions
                                • \n
                              • Custom sbatch directives via sbatch_options in task YAML
                                • \n
                              • Configurable workdir and tmpdir for shared filesystem environments
                                • \n
                              • Admin policy support for Slurm partition routing
                                • \n
                              \n
                              # Launch on a Slurm cluster\r\n</tr></", "merged_at": "2026-04-09T16:24:54Z"}
{"number": 2348, "title": "Update pillow requirement from <12.2,>=11.3 to >=11.3,<12.3", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [pillow](https://github.com/python-pillow/Pillow) to permit the latest version.\n
                              \nRelease notes\n
                              Sourced from pillow's releases.
                              \n
                              \n
                              12.2.0
                              \n
                              https://pillow.readthedocs.io/en/stable/releasenotes/12.2.0.html
                              \n
                              Documentation
                              \n\n
                              Dependencies
                              \n
                                \n
                              • Update xz to 5.8.3 #9523 [@\u200bradarhere]
                                • \n
                              • Update libjpeg-turbo to 3.1.4.1 #9507 [@\u200bradarhere]
                                • \n
                              • Update libpng to 1.6.56 #9499 [@\u200bradarhere]
                                • \n
                              • Update freetype to 2.14.3 #9485 [@\u200bradarhere]
                                • \n
                              • Updated libavif to 1.4.1 #9479 [@\u200bradarhere]
                                • \n
                              • Updated harfbuzz to 13.2.1 #9461 [@\u200bradarhere]
                                • \n
                              • Update Ghostscript to 10.7.0 #9469 [@\u200bradarhere]
                                • \n
                              • Update harfbuzz to 13.0.1 #9453 [@\u200bradarhere]", "merged_at": "2026-04-06T15:03:52Z"}
{"number": 2347, "title": "[Judge] Allow judge functions to accept pre-built conversations", "files": ["src/oumi/judges/base_judge.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nAllow judge functions to accept pre-built conversations\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-04-04T03:08:02Z"}
{"number": 2346, "title": "Raise error for empty anthropic response", "files": ["src/oumi/inference/anthropic_inference_engine.py", "tests/unit/inference/test_anthropic_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nWhat changed:\r\nAnthropic engine now raises a runtime error if it receives a response with empty content.\r\n\r\nWhy:\r\nWe were receiving responses with empty content, which was raising an index error with no context. This change should allow for easier debugging.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-04-03T16:58:10Z"}
{"number": 2345, "title": "[Judge] Remove think tags before parsing the judge response", "files": ["src/oumi/judges/base_judge.py", "tests/unit/judges/test_base_judge.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nRemove think tags before parsing the judge response\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-04-03T02:51:46Z"}
{"number": 2344, "title": "upgrade transformers and trl", "files": ["pyproject.toml", "tests/integration/datasets/test_vision_language_completions_only.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\n\n----\n## Summary by Gitar\n\n- **Dependency upgrades:**\n  - Updated `transformers` upper bound from `<5.5` to `<5.6`\n  - Updated `trl` upper bound from `<0.30` to `<1.1`\n\nThis will update automatically on new commits.", "merged_at": "2026-04-02T20:18:27Z"}
{"number": 2336, "title": "[docs] Update readme with latest updates", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] Update readme with latest updates\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-31T03:28:05Z"}
{"number": 2334, "title": "feat(analyze): add quality checks for missing user message and misplaced system message", "files": ["src/oumi/analyze/analyzers/quality.py", "tests/unit/analyze/test_quality_analyzer.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "## Description\r\n\r\nAdd two new boolean quality checks to `DataQualityAnalyzer`:\r\n\r\n1. **`has_no_user_message`** \u2014 flags conversations that contain no user message\r\n2. **`has_system_message_not_at_start`** \u2014 flags conversations where a system message appears after position 0\r\n\r\n## Related issues\r\n\r\nFixes https://linear.app/oumi/issue/OPE-1856\r\n\r\n## Before submitting\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Test plan\r\n- [x] 11 new unit tests covering both checks (empty conversations, system-only, middle/end system messages, multiple system messages)\r\n- [x] All 43 quality analyzer tests pass\r\n\r\n\ud83e\udd16 Generated with [Claude Code](https://claude.com/claude-code)", "merged_at": "2026-03-31T02:05:02Z"}
{"number": 2333, "title": "[infer] Add list_models api", "files": ["src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/bedrock_inference_engine.py", "src/oumi/inference/fireworks_inference_engine.py", "src/oumi/inference/gcp_inference_engine.py", "src/oumi/inference/gemini_inference_engine.py", "src/oumi/inference/openai_inference_engine.py", "src/oumi/inference/openrouter_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/together_inference_engine.py", "tests/unit/inference/test_list_models.py"], "area": "inference", "area_votes": {"inference": 9}, "body": "# Description\r\n\r\n\r\n\r\n- Add a list_models api to return the available models for each provider\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-30T19:03:46Z"}
{"number": 2330, "title": "Update transformers requirement from <5.4,>=4.57 to >=4.57,<5.5", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [transformers](https://github.com/huggingface/transformers) to permit the latest version.\n
                                \nRelease notes\n
                                Sourced from transformers's releases.
                                \n
                                \n
                                Release v5.4.0: PaddlePaddle models \ud83d\ude4c, Mistral 4, PI0, VidEoMT, UVDoc, SLANeXt, Jina Embeddings v3
                                \n
                                New Model additions
                                \n
                                VidEoMT
                                \n\n
                                Video Encoder-only Mask Transformer (VidEoMT) is a lightweight encoder-only model for online video segmentation built on a plain Vision Transformer (ViT). It eliminates the need for dedicated tracking modules by introducing a lightweight query propagation mechanism that carries information across frames and employs a query fusion strategy that combines propagated queries with temporally-agnostic learned queries. VidEoMT achieves competitive accuracy while being 5x-10x faster than existing approaches, running at up to 160 FPS with a ViT-L backbone.
                                \n
                                Links: Documentation | Paper
                                \n\n
                                UVDoc
                                \n\n
                                UVDoc is a machine learning model designed for document image rectification and correction. The main purpose of this model is to carry out geometric transformation on images to correct document distortion, inclination, perspective deformation and other problems in document images. It provides both single input and batched inference capabilities for processing distorted document images.
                                \n
                                Links: Documentation
                                \n\n
                                Jina Embeddings v3
                                \n\n
                                The Jina-Embeddings-v3 is a multilingual, multi-task text embedding model designed for a variety of NLP applications. Based on the XLM-RoBERTa architecture, this model supports Rotary Position Embeddings (RoPE) replacing absolute position embeddings to support long input sequences up to 8192 tokens. Additionally, it features 5 built-in Task-Specific LoRA Adapters that allow the model to generate task-specific embeddings (e.g., for retrieval vs. classification) without increasing inference latency significantly.
                                \n
                                Links: Documentation | Paper
                                \n\n
                                Mistral4
                                \n\n
                                Mistral 4 is a powerful hybrid model with the capability of acting as both a general instruction model and a reasoning model. It unifies the capabilities of three different model families - Instruct, Reasoning (previously called Magistral), and Devstral - into a single, unified model. The model features a MoE architecture with 128 experts and 4 active, 119B parameters with 6.5B activated per token, 256k context length, and supports multimodal input with both text and image processin", "merged_at": "2026-03-30T16:16:43Z"}
{"number": 2328, "title": "Update datasets requirement from <4.8.4,>=3.2 to >=3.2,<4.8.5", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [datasets](https://github.com/huggingface/datasets) to permit the latest version.\n
                                \nRelease notes\n
                                Sourced from datasets's releases.
                                \n
                                \n
                                4.8.4
                                \n
                                What's Changed
                                \n\n
                                Full Changelog: https://github.com/huggingface/datasets/compare/4.8.3...4.8.4
                                \n
                                \n
                                \n
                                \nCommits\n\n
                                \n
                                \n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
                                \nDependabot commands and options\n
                                \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@d", "merged_at": "2026-03-30T18:47:49Z"}
{"number": 2326, "title": "Update uvicorn requirement from <0.42.0 to <0.43.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [uvicorn](https://github.com/Kludex/uvicorn) to permit the latest version.\n
                                \nRelease notes\n
                                Sourced from uvicorn's releases.
                                \n
                                \n
                                Version 0.42.0
                                \n
                                Changed
                                \n
                                  \n
                                • Use bytearray for request body accumulation to avoid O(n^2) allocation on fragmented bodies (#2845)
                                  • \n
                                \n
                                Fixed
                                \n
                                  \n
                                • Escape brackets and backslash in httptools HEADER_RE regex (#2824)
                                  • \n
                                • Fix multiple issues in websockets sans-io implementation (#2825)
                                  • \n
                                \n
                                \n
                                New Contributors
                                \n\n
                                \n
                                Full Changelog: https://github.com/Kludex/uvicorn/compare/0.41.0...0.42.0
                                \n
                                \n
                                \n
                                \nChangelog\n
                                Sourced from uvicorn's changelog.
                                \n
                                \n
                                0.42.0 (March 16, 2026)
                                \n
                                Changed
                                \n
                                  \n
                                • Use bytearray for request body accumulation to avoid O(n^2) allocation on fragmented bodies (#2845)
                                  • \n
                                \n
                                Fixed
                                \n
                                  \n
                                • Escape brackets and backslash in httptools HEADER_RE regex (#2824)
                                  • \n
                                • Fix multiple issues in websockets sans-io implementation (#2825)
                                  • \n
                                \n
                                0.41.0 (February 16, 2026)
                                \n
                                Added
                                \n
                                  \n
                                • Add --limit-max-requests-jitter to stagger worker restarts (#2707)
                                  • \n
                                • Add socket path to scope["server"] (#2561)
                                  • \n
                                \n
                                Changed
                                \n
                                  \n
                                • Rename LifespanOn.error_occured to error_occurred (#2776)
                                  • \n
                                \n
                                Fixed
                                \n
                                  \n
                                • Ignore permission denied errors in watchfiles reloader (#2817)
                                  • \n
                                • Ensure lifespan shutdown runs when should_exit is set during startup (#2812)
                                  • \n
                                • Reduce the log level of 'request limit exceeded' messages (#2788)
                                  • \n
                                \n
                                0.40.0 (December 21, 2025)
                                \n
                                Remove
                                \n
                                  \n
                                • Drop support for Python 3.9 (#2772)
                                  • \n
                                \n
                                0.39.0 (December 21, 2025)
                                \n
                                Fixed
                                \n
                                  \n
                                • Send close frame on ASGI return for WebSockets (#2769)
                                  • \n
                                • Explicitly start ASGI run with empty context (#2742)
                                  • \n
                                \n
                                0.38.0 (October 18, 2025)
                                \n
                                Added
                                \n
                                  \n
                                • Support Python 3.14 (#2723)
                                  • \n
                                \n
                                0.37.0 (September 23, 2025)
                                \n
                                Added
                                \n\n
                                \n
                                ... (truncated)
                                \n
                                \n
                                \nCommits\n
                                  \n
                                • \r\n\r\n\r\nUpdate analyze, judge, and quantize unit tests to expect OumiConfigError\r\ninstead of ValueError after config validation moved to typed Oumi config\r\nexceptions. This matches runtime behavior and fixes the failing\r\nstatic-checks test cases.\r\n\r\nFixes OPE-1853\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-05-04T17:56:40Z"}
{"number": 2324, "title": "[infer] Enable prompt-caching for anthropic, and add cache tokens reporting for together.ai", "files": ["src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/together_inference_engine.py", "tests/unit/inference/test_anthropic_inference_engine.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Description\r\n\r\n\r\n\r\n- [infer] Enable prompt-caching for anthropic, and add cache tokens reporting for together.ai\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-27T19:28:24Z"}
{"number": 2322, "title": "[bug] fix issue with verl training with latest", "files": ["src/oumi/core/trainers/verl_grpo_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [bug] fix issue with verl training with latest\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\n\n----\n## Summary by Gitar\n\n- **Compatibility fix:**\n  - Added `AsyncActorRolloutRefWorker` import for verl >=0.7 support\n  - Dynamically select worker class based on verl version in `role_worker_mapping`\n  - Moved version check earlier in method for cleaner logic flow\n\nThis will update automatically on new commits.", "merged_at": "2026-03-26T20:09:46Z"}
{"number": 2320, "title": "Add dataset_path existence check in DatasetParams finalize_and_validate", "files": ["src/oumi/core/configs/params/data_params.py", "tests/unit/core/configs/params/test_data_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "\r\n# Description\r\n\r\nDepends on https://github.com/oumi-ai/oumi/pull/2319\r\n\r\n\r\n\r\n\r\nThis ensures invalid local dataset paths fail early and adds unit\r\ntests for missing, existing file, missing path, and existing directory cases.\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1849\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-04-29T04:21:24Z"}
{"number": 2317, "title": "[deps] update to transformers v5", "files": [".github/workflows/install_test.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\n\n----\n## Summary by Gitar\n\n- **Dependency updates:**\n  - Updated `transformers` constraint from `>=4.57,<4.58` to `>=4.57,<5.4` to support v5\n  - Removed outdated comment about Llama Vision attention issue (OPE-875)\n\nThis will update automatically on new commits.", "merged_at": "2026-03-25T21:32:07Z"}
{"number": 2315, "title": "[deps] Handle trl vllm compatibility issues", "files": ["src/oumi/builders/training.py", "src/oumi/utils/packaging.py", "tests/conftest.py", "tests/integration/train/test_train.py", "tests/unit/utils/test_packaging.py"], "area": "other", "area_votes": {"other": 2}, "body": "# Description\r\n\r\n\r\n\r\n- [WIP][deps] Handle trl vllm compatibility issues\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\n\n----\n## Summary by Gitar\n\n- **Dependency version relaxations:**\n  - Relaxed `transformers` upper bound from `<4.58` to `<6`\n  - Extended `trl` upper bound from `<0.27` to `<0.30`\n- **TRL/vLLM compatibility handling:**\n  - Added `verify_trl_vllm_compatibility()` function to check version compatibility before importing TRL trainers\n  - Integrated compatibility check into `TRL_GOLD` trainer initialization in `training.py`\n  - Added compatibility validation to GOLD trainer test with skip on failure\n- **Code refactoring:**\n  - Consolidated packaging imports at top of `training.py` to reduce duplication\n- **Temporary workflow disables:**\n  - Disabled `doctests`, `gpu_install_test`, `install_test`, and `pretest` workflows (marked for re-enablement)\n- **Test coverage:**\n  - Added four unit tests for `verify_trl_vllm_compatibility()` covering missing packages, compatibility pass/fail scenarios\n\nThis will update automatically on new commits.", "merged_at": "2026-03-25T15:26:14Z"}
{"number": 2313, "title": "[deps] Unblock datasets update", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-24T21:34:24Z"}
{"number": 2311, "title": "[deps] handle dataset conversion edge case", "files": ["src/oumi/core/datasets/base_iterable_dataset.py", "tests/unit/core/datasets/test_base_iterable_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [deps] handle dataset conversion edge case\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\n\n----\n## Summary by Gitar\n\n- **Fixed tensor conversion:**\n  - Changed `_convert_tensors_for_arrow()` to use `.item()/.tolist()` instead of `.numpy()` to ensure Arrow compatibility with plain Python types\n  - Added scalar tensor handling with `.item()` for 0-dimensional tensors\n- **Added tests:**\n  - New test file `test_base_iterable_dataset.py` with 11 test cases covering scalar tensors, multi-dimensional tensors, nested structures, and mixed data types\n\nThis will update automatically on new commits.", "merged_at": "2026-03-24T19:16:21Z"}
{"number": 2310, "title": "[pyright] ignore expected lint errors in vllm engine", "files": ["src/oumi/inference/vllm_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [pyright] ignore expected lint errors in vllm engine\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-24T19:10:54Z"}
{"number": 2309, "title": "[params] Update trust_remote_code on DataParams", "files": ["src/oumi/core/configs/params/data_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- That param is now only used for loading tokenizers and processors that require custom code, not datasets\r\n- Update the docstring to match the implementation and remove deprecation notice\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-24T18:46:09Z"}
{"number": 2305, "title": "[configs] Add lora config and example infer configs for qwen 3.5 0.8b", "files": ["configs/recipes/qwen3_5/inference/0.8b_infer.yaml", "configs/recipes/qwen3_5/inference/0.8b_vllm_infer.yaml", "configs/recipes/qwen3_5/sft/0.8b_lora/train.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\n\r\n\r\n- [configs] Add lora config and example infer configs for qwen 3.5 0.8b \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-23T19:08:35Z"}
{"number": 2304, "title": "[ci] Add targets to enable more tests, cleanup-warnings", "files": [".github/workflows/doctests.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 4}, "body": "# Description\r\n\r\n\r\n\r\n- Add targets to enable more tests\r\n- Remove deprecated alpaca_eval\r\n- Cleanup expected pytest warnings\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-23T20:03:33Z"}
{"number": 2303, "title": "[ci] reduce number of combinations to avoid quota errors", "files": [".github/workflows/install_test.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [ci] reduce number of combinations to avoid quota errors\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-23T18:54:08Z"}
{"number": 2302, "title": "[configs] Update configs to explicitly set max_grad_norm to 0.0", "files": ["configs/recipes/deepseek_r1/sft/distill_llama_70b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/gemma3/sft/12b_lora/train.yaml", "configs/recipes/gemma3/sft/27b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml", "configs/recipes/olmo3/sft/32b_lora/train.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_train.yaml", "configs/recipes/qwen3/sft/0.6b_full/train.yaml", "configs/recipes/qwen3/sft/1.7b_full/train.yaml", "configs/recipes/qwen3/sft/14b_lora/train.yaml", "configs/recipes/qwen3/sft/235b_lora/train.yaml", "configs/recipes/qwen3/sft/30b_a3b_lora/train.yaml", "configs/recipes/qwen3/sft/32b_lora/train.yaml", "configs/recipes/qwen3/sft/4b_full/train.yaml", "configs/recipes/qwen3/sft/8b_full/train.yaml", "configs/recipes/qwen3_instruct/sft/30b_a3b_lora/train.yaml", "configs/recipes/qwen3_next/sft/80b_a3b_instruct_lora/train.yaml", "configs/recipes/qwen3_next/sft/80b_a3b_lora/train.yaml", "configs/recipes/qwq/sft/lora_train.yaml"], "area": "configs", "area_votes": {"configs": 22}, "body": "# Description\r\n\r\n\r\n\r\n- Explicitly set max_grad_norm to 0.0 instead of null\r\n- Both values are equivalent\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-23T18:26:25Z"}
{"number": 2301, "title": "[config] set max_grad_norm to 0.0 if None", "files": ["src/oumi/core/configs/params/training_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [config] set max_grad_norm to 0.0 if None\r\n- Both value are equivalent (grad norm disabled), but tfv5 does not accept `None` anymore\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-23T18:26:25Z"}
{"number": 2298, "title": "Update trl requirement from <0.27,>=0.24 to >=0.24,<0.30", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [trl](https://github.com/huggingface/trl) to permit the latest version.\n
                                  \nRelease notes\n
                                  Sourced from trl's releases.
                                  \n
                                  \n
                                  v0.29.1
                                  \n
                                  What's Changed
                                  \n
                                    \n
                                  • Handle mm_token_type_ids in SFT/GRPO/RLOO to fix IndexError by @\u200balbertvillanova in huggingface/trl#5178
                                    • \n
                                  • Fix prepare_multimodal_messages to support tool_calls and tool role by @\u200balvarobartt in huggingface/trl#5212
                                    • \n
                                  • Fix type for model_init_kwargs when passed as CLI JSON string by @\u200balbertvillanova in huggingface/trl#5230
                                    • \n
                                  • Decouple rollout dispatch from vLLM backend in GRPO _generate_single_turn by @\u200balbertvillanova in huggingface/trl#5122
                                    • \n
                                  • Simplify logic for structured outputs across vLLM versions by @\u200balbertvillanova in huggingface/trl#5215
                                    • \n
                                  • Add support for raw ids in prompts in vLLM client and server by @\u200bqgallouedec in huggingface/trl#5225
                                    • \n
                                  • Add VLM support when passing raw token IDs to vLLM client by @\u200bqgallouedec in huggingface/trl#5227
                                    • \n
                                  • Move rollout_func from _generate_single_turn to _generate by @\u200bqgallouedec in huggingface/trl#5232
                                    • \n
                                  • [GRPO/RLOO] Tokenize before vLLM generation call by @\u200bqgallouedec in huggingface/trl#5238
                                    • \n
                                  • Support JSON string parsing of teacher_model_init_kwargs in MiniLLMConfig by @\u200balbertvillanova in huggingface/trl#5259
                                    • \n
                                  • [GRPO/RLOO] Unify tokenization across all generation backends in _generate_single_turn by @\u200bqgallouedec in huggingface/trl#5239
                                    • \n
                                  • [GRPO/RLOO] Extract tokenize prompts from _generate_single_turn by @\u200bqgallouedec in huggingface/trl#5240
                                    • \n
                                  • [CPO/ORPO] Fix handling of different length chosen/rejected prompts. by @\u200bdavmels in huggingface/trl#4639
                                    • \n
                                  • Fix type for teacher_model_init_kwargs when passed as CLI JSON string by @\u200balbertvillanova in huggingface/trl#5258
                                    • \n
                                  • Fix support for model_init_kwargs in GKD/GOLD when passed as CLI JSON string by @\u200balbertvillanova=0.5 to >=0.5,<0.8", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [verl](https://github.com/volcengine/verl) to permit the latest version.\n
                                    \nRelease notes\n
                                    Sourced from verl's releases.
                                    \n
                                    \n
                                    v0.7.1
                                    \n
                                    Highlight
                                    \n
                                    Model Engine
                                    \n
                                    Megatron
                                    \n\n
                                    VeOmni
                                    \n\n
                                    torchtitan
                                    \n\n
                                    Rollout Engine
                                    \n
                                    vLLM
                                    \n
                                      \n
                                    • Separate model runner from training process and refit weights by cuda ipc verl-project/verl#4280
                                      • \n
                                    • FP8 rollout enhancement
                                      • \n
                                    • Upgrade to vllm==0.17.0
                                      • \n
                                    \n
                                    SGLang
                                    \n
                                      \n
                                    • Support router replay
                                      • \n
                                    • FP8 rollout enhancement
                                      • \n
                                    • Upgrade to sglang==0.5.9
                                      • \n
                                    \n
                                    TensorRT-LLM
                                    \n\n
                                    Checkpoint Engine
                                    \n\n
                                    Trainer
                                    \n
                                      \n
                                    • one-step-off/fully async trainer refactor with verl-core\n\n
                                      • \n
                                    \n
                                    What's Changed
                                    \n
                                      \n
                                    • [ci] feat: add npu unit test by @\u200byyyy2000 in verl-project/verl#4626
                                      • \n
                                    • [recipe] fix: workaround for making the one-step off-policy recipe compatible with IPv6 environments on Ascend NPU by @\u200bji-huazhong in verl-project/verl#4782
                                      • \n
                                    • [fsdp] feat: integrate PrefixGrouper for GRPO training acceleration by @\u200bkevssim in verl-project/verl#4368
                                      • \n
                                    • [rollout] fix: use model_dump() for proper Pydantic serialization in to", "merged_at": "2026-03-23T15:25:52Z"}
{"number": 2296, "title": "Update nvidia-ml-py requirement from <13.591,>=13.580 to >=13.580,<13.596", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [nvidia-ml-py](https://forums.developer.nvidia.com) to permit the latest version.\n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
                                      \nDependabot commands and options\n
                                      \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot show  ignore conditions` will show all of the ignore conditions of the specified dependency\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n\n\n
                                      ", "merged_at": "2026-03-23T01:39:56Z"}
{"number": 2295, "title": "Update vllm requirement from <0.11,>=0.10 to >=0.10,<0.19", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [vllm](https://github.com/vllm-project/vllm) to permit the latest version.\n
                                      \nRelease notes\n
                                      Sourced from vllm's releases.
                                      \n
                                      \n
                                      v0.18.0
                                      \n
                                      vLLM v0.18.0
                                      \n
                                      Known issues
                                      \n
                                        \n
                                      • Degraded accuracy when serving Qwen3.5 with FP8 KV cache on B200 (#37618)
                                        • \n
                                      • If you previously ran into CUBLAS_STATUS_INVALID_VALUE and had to use a workaround in v0.17.0, you can reinstall torch 2.10.0. PyTorch published an updated wheel that addresses this bug.
                                        • \n
                                      \n
                                      Highlights
                                      \n
                                      This release features 445 commits from 213 contributors (61 new)!
                                      \n
                                        \n
                                      • gRPC Serving Support: vLLM now supports gRPC serving via the new --grpc flag (#36169), enabling high-performance RPC-based serving alongside the existing HTTP/REST interface.
                                        • \n
                                      • GPU-less Render Serving: New vllm launch render command (#36166, #34551) enables GPU-less preprocessing and rendering, allowing separation of multimodal preprocessing from GPU inference.
                                        • \n
                                      • NGram GPU Speculative Decoding: NGram speculative decoding now runs on GPU and is compatible with the async scheduler (#29184), significantly reducing spec decode overhead.
                                        • \n
                                      • KV Cache Offloading Improvements: Smart CPU offloading that stores only frequently-reused blocks (#35342), plus FlexKV as a new offloading backend (#34328) and support for multiple KV groups in offloading spec (#36610).
                                        • \n
                                      • Elastic Expert Parallelism Milestone 2: NIXL-EP integration (#35627) enables dynamic GPU scaling for MoE experts, with new --enable-ep-weight-filter CLI option (#37351) for faster EP model loading.
                                        • \n
                                      • FlashInfer 0.6.6: Updated FlashInfer dependency (#36768) with numerous performance and correctness improvements.
                                        • \n
                                      • Responses API Streaming Tool Calls: The OpenAI Responses API now supports tool/function calling with streaming (#29947).
                                        • \n
                                      • Online Beam Search for ASR: Beam search support for encoder/decoder models both offline (#36153) and online transcriptions (#36160).
                                        • \n
                                      • Ray No Longer a Default Dependency: Ray has been removed as a default dependency (#36170) \u2014 install it explicitly if needed.
                                        • \n
                                      \n
                                      Model Support
                                      \n
                                        \n
                                      • New architectures: Sarvam MoE (#33942), OLMo Hybrid (#32550), HyperCLOVAX-SEED-Think-32B VLM (#31471), HyperCLOVAX-SEED-Think-14B (\r\n\r\n- [tiny] remove deprecated param from oumi trainer\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-23T15:32:05Z"}
{"number": 2289, "title": "[types] use assert to narrow type in collator", "files": ["src/oumi/core/collators/text_completions_collator_with_padding.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n- tfv5 changed the signature, now we need this to avoid type errors\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-20T17:37:07Z"}
{"number": 2288, "title": "[configs][vl] Add support for qwen3-vl-moe", "files": ["configs/recipes/vision/qwen3_vl_30b_a3b/sft/30b_a3b_instruct_fft_train.yaml", "configs/recipes/vision/qwen3_vl_30b_a3b/sft/30b_a3b_instruct_lora_train.yaml", "src/oumi/core/configs/internal/supported_models.py"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\n\r\n\r\n- [configs][vl] Add support for qwen3-vl-moe\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-20T17:32:39Z"}
{"number": 2286, "title": "[configs] Add qwen3-vl-2b configs", "files": ["configs/recipes/vision/qwen3_vl_2b/inference/vllm_infer.yaml", "configs/recipes/vision/qwen3_vl_2b/sft/2b_instruct_fft_train.yaml", "configs/recipes/vision/qwen3_vl_2b/sft/2b_instruct_lora_train.yaml", "configs/recipes/vision/qwen3_vl_4b/sft/4b_instruct_fft_train.yaml", "configs/recipes/vision/qwen3_vl_4b/sft/4b_instruct_lora_train.yaml"], "area": "configs", "area_votes": {"configs": 5}, "body": "# Description\r\n\r\n\r\n\r\n- Add qwen3-vl-2b configs for testing\r\n- Consolidate 4b configs in a single folder\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-20T16:29:16Z"}
{"number": 2285, "title": "[config] Add example config for qwen3.5 0.8B", "files": ["configs/recipes/qwen3_5/sft/0.8b_full/train.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [config] Add example config for qwen3.5 0.8B\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-20T16:16:30Z"}
{"number": 2284, "title": "[deps] deprecate unsupported datasets and exclude from tests", "files": ["src/oumi/datasets/pretraining/dolma.py", "src/oumi/datasets/pretraining/pile.py", "src/oumi/datasets/pretraining/red_pajama_v1.py", "src/oumi/datasets/pretraining/red_pajama_v2.py", "src/oumi/datasets/pretraining/slim_pajama.py", "src/oumi/datasets/vision_language/coco_captions.py", "src/oumi/datasets/vision_language/flickr30k.py", "tests/integration/datasets/test_pretraining_datasets_full_epoch.py", "tests/integration/datasets/test_sft_datasets_load_datasets.py", "tests/integration/datasets/test_sft_vision_datasets_load_datasets.py", "tests/unit/core/configs/test_training_config.py"], "area": "data", "area_votes": {"data": 7}, "body": "# Description\r\n\r\n\r\n\r\n- [deps] deprecate unsupported datasets and exclude from tests\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\n\n----\n## Summary by Gitar\n\n- **Deprecated datasets:**\n  - Added deprecation warnings to `DolmaDataset`, `PileV1Dataset`, `RedPajamaDataV1Dataset`, and `RedPajamaDataV2Dataset` for HuggingFace datasets>=4.0.0 incompatibility\n  - Added deprecation warnings to `COCOCaptionsDataset` and `Flickr30kDataset` vision datasets requiring datasets<4.0.0\n- **Dataset updates:**\n  - Updated `SlimPajamaDataset` to use community re-upload at `gmongaras/SlimPajama-627B_Reupload` as original repository removed\n  - Added backwards compatibility alias for `cerebras/SlimPajama-627B`\n- **Test exclusions:**\n  - Excluded deprecated datasets from pretraining and SFT vision dataset tests\n  - Replaced `coco_captions` with `HuggingFaceM4/the_cauldron` in training config tests\n\nThis will update automatically on new commits.", "merged_at": "2026-03-20T16:09:07Z"}
{"number": 2279, "title": "[deps] handle deprecated warmup_ratio in tfv5", "files": ["src/oumi/core/configs/params/training_params.py", "tests/unit/core/configs/params/test_training_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [deps] handle deprecated warmup_ratio in tfv5\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-20T05:32:03Z"}
{"number": 2276, "title": "[vl] update default processor to transform messages to dict for transformers v5 compatibility", "files": ["src/oumi/core/processors/default_processor.py", "tests/unit/builders/test_processors.py", "tests/unit/core/types/test_conversation.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [vl] update default processor to transform messages to dict for transformers v5 compatibility\r\n- Add tests\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-20T00:53:16Z"}
{"number": 2274, "title": "[deps] Pin datasets version to avoid breaking change", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [deps] Pin datasets version to avoid breaking change\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-19T23:38:04Z"}
{"number": 2269, "title": "[deps] update training params config to use the right token tracking depending on transformers version", "files": ["src/oumi/core/configs/params/training_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- include_tokens_per_second got deprecated in transformers v5\r\n- update training params config to use the right token tracking depending on transformers version \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-19T18:05:45Z"}
{"number": 2268, "title": "[deps] set tensorboard logging dir manually", "files": ["src/oumi/core/configs/params/training_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- `logging_dir` is deprecated in transformers v5. The only usage is for tensorboard\r\n- This PR sets tensorboard logging dir manually instead of passing logging_dir to transformers\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-19T19:02:25Z"}
{"number": 2266, "title": "[refactor] replace SpecialTokensMixin import with custom dataclass", "files": ["src/oumi/core/tokenizers/special_tokens.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [ops] replace SpecialTokensMixin import with custom dataclass\r\n- SpecialTokensMixin is deprecated in transformers v5+\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-19T16:21:48Z"}
{"number": 2265, "title": "Oumi -> Oumi OSS", "files": ["docs/about/acknowledgements.md", "docs/about/citations.md", "docs/about/telemetry.md", "docs/cli/commands.md", "docs/conf.py", "docs/development/contributing.md", "docs/development/dev_setup.md", "docs/development/docs_guide.md", "docs/development/style_guide.md", "docs/faq/oom.md", "docs/faq/troubleshooting.md", "docs/get_started/core_concepts.md", "docs/get_started/installation.md", "docs/get_started/quickstart.md", "docs/get_started/tutorials.md", "docs/index.md", "docs/resources/datasets/data_formats.md", "docs/resources/datasets/datasets.md", "docs/resources/datasets/other_datasets.md", "docs/resources/datasets/preference_datasets.md", "docs/resources/datasets/pretraining_datasets.md", "docs/resources/datasets/sft_datasets.md", "docs/resources/datasets/vl_sft_datasets.md", "docs/resources/models/custom_models.md", "docs/resources/models/models.md", "docs/resources/recipes.md", "docs/user_guides/analyze/analyze_config.md", "docs/user_guides/customization.md", "docs/user_guides/evaluate/custom_evals.md", "docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/evaluation_config.md", "docs/user_guides/evaluate/generative_benchmarks.md", "docs/user_guides/evaluate/leaderboards.md", "docs/user_guides/evaluate/standardized_benchmarks.md", "docs/user_guides/infer/infer.md", "docs/user_guides/judge/built_in_judges.md", "docs/user_guides/judge/judge.md", "docs/user_guides/launch/custom_cluster.md", "docs/user_guides/launch/kubernetes.md", "docs/user_guides/train/environments/docker.md", "docs/user_guides/train/environments/local.md", "docs/user_guides/train/environments/notebooks.md", "docs/user_guides/train/train.md", "docs/user_guides/train/training_methods.md", "docs/user_guides/tune.md"], "area": "docs", "area_votes": {"docs": 45}, "body": "Replacement of \"Oumi\" with \"Oumi OSS\" throughout", "merged_at": "2026-03-31T03:32:34Z"}
{"number": 2264, "title": "Add start_at, end_at, and cost_per_hour field to oumi launcher", "files": ["src/oumi/core/launcher/base_cluster.py", "src/oumi/launcher/clients/sky_client.py", "src/oumi/launcher/clouds/sky_cloud.py", "src/oumi/launcher/clusters/sky_cluster.py", "tests/unit/launcher/clients/test_sky_client.py", "tests/unit/launcher/clusters/test_sky_cluster.py"], "area": "launcher", "area_votes": {"launcher": 4}, "body": "# Description\r\n\r\nThis adds useful metadata for calculating the total cost of the job.\r\n\r\nTowards LOU-1202\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2026-03-26T20:29:07Z"}
{"number": 2263, "title": "Integrate Reo script for client initialization", "files": ["docs/_static/custom.js"], "area": "docs", "area_votes": {"docs": 1}, "body": "As per title", "merged_at": "2026-03-18T16:34:18Z"}
{"number": 2262, "title": "Fix Anthropic Partial Error Batch Parsing", "files": ["src/oumi/inference/anthropic_inference_engine.py", "tests/unit/inference/test_anthropic_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nThe error format for anthropic looks like:\r\n\r\n`{\"custom_id\":\"request-5\",\"result\":{\"type\":\"errored\",\"error\":{\"type\":\"error\",\"error\":{\"details\":{\"error_visibility\":\"user_facing\"},\"type\":\"invalid_request_error\",\"message\":\"prompt is too long: 224999 tokens > 200000 maximum\"},\"request_id\":\"workerreq_01QTqSsMw5S3FuLBYRtjGwM3\"}}}`\r\n\r\nWe need to tweak our parsing for this format. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-17T20:22:53Z"}
{"number": 2261, "title": "Fix OpenAI batch error parsing", "files": ["src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\nWhen checking the error file openAI produces for batch I found out the format is:\r\n\r\n`{\"id\": \"batch_req_69b9a728c9808190959a5be54bea853b\", \"custom_id\": \"request-5\", \"response\": {\"status_code\": 400, \"request_id\": \"7c91de2b-c24f-4ee7-8291-9bb68093f4ae\", \"body\": {\"error\": {\"message\": \"Input tokens exceed the configured limit of 272000 tokens. Your messages resulted in 500032 tokens. Please reduce the length of the messages.\", \"type\": \"invalid_request_error\", \"param\": \"messages\", \"code\": \"context_length_exceeded\"}}}, \"error\": null}`\r\n\r\nThis breaks our current logic so we won't return the reason on partial failures.\r\n\r\n## Related issues\r\n\r\nN/A\r\n\r\n## Before submitting\r\n\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2026-03-17T20:17:12Z"}
{"number": 2260, "title": "Adding Support For Cancel Batch", "files": ["src/oumi/core/synthesis/attribute_synthesizer.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/together_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"data": 1, "inference": 3}, "body": "# Description\r\n\r\n\r\n\r\nAdding support to cancel batch jobs. This includes support for Anthropic, OpenAI, and Together. Fireworks currently doesn't support this. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-16T14:30:19Z"}
{"number": 2256, "title": "[tiny] improve the text for failure reason", "files": ["src/oumi/analyze/testing/engine.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "## Summary\r\nimprove the text for failure reason", "merged_at": "2026-03-11T20:18:17Z"}
{"number": 2255, "title": "feat(deploy): add Parasail deployment client for dedicated endpoints", "files": ["src/oumi/cli/deploy.py", "src/oumi/deploy/__init__.py", "src/oumi/deploy/base_client.py", "src/oumi/deploy/fireworks_client.py", "src/oumi/deploy/parasail_api.py", "src/oumi/deploy/parasail_client.py", "src/oumi/deploy/utils.py", "tests/unit/deploy/test_fireworks_client.py", "tests/unit/deploy/test_parasail_client.py", "tests/unit/deploy/test_utils.py"], "area": "inference", "area_votes": {"cli": 1, "inference": 5}, "body": "# Description\r\n\r\n\r\n\r\n\r\n- Add parasail_api.py with Pydantic v2 models derived from live API\r\n  responses: DeviceConfig, DeploymentStatusBlock, DeploymentResponse,\r\n  CreateDeploymentRequest, SupportCheckResponse, ModelProperties\r\n- Add parasail_client.py implementing BaseDeploymentClient: create/get/\r\n  update/delete/list endpoints, start/stop (pause/resume), list hardware,\r\n  and upload_model with HuggingFace compatibility check via /dedicated/support\r\n- Add deploy/utils.py with shared HF token resolution, repo-ID/URL\r\n  detection, and private-model accessibility warning helpers\r\n- Wire ParasailDeploymentClient into CLI dispatch (_get_deployment_client)\r\n  and auto-discovery (_get_available_providers via PARASAIL_API_KEY)\r\n- Add PARASAIL to DeploymentProvider enum\r\n- Change --provider CLI parameter type from str to DeploymentProvider\r\n  enum across all subcommands for stricter validation\r\n- Generalize _poll_endpoint_until_ready to _poll_endpoint_until_state\r\n  with a configurable target EndpointState and debug logging\r\n- Add --wait/-w flag to start and stop CLI commands\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-27T15:07:39Z"}
{"number": 2254, "title": "[deps] add is_transformers_v5 version detection helper", "files": ["src/oumi/utils/packaging.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-11T03:18:18Z"}
{"number": 2251, "title": "Bump the all-actions group with 5 updates", "files": [".github/workflows/release_docker.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Bumps the all-actions group with 5 updates:\n\n| Package | From | To |\n| --- | --- | --- |\n| [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) | `3` | `4` |\n| [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) | `3` | `4` |\n| [docker/login-action](https://github.com/docker/login-action) | `3` | `4` |\n| [docker/metadata-action](https://github.com/docker/metadata-action) | `5` | `6` |\n| [docker/build-push-action](https://github.com/docker/build-push-action) | `6` | `7` |\n\nUpdates `docker/setup-qemu-action` from 3 to 4\n
                                        \nRelease notes\n
                                        Sourced from docker/setup-qemu-action's releases.
                                        \n
                                        \n
                                        v4.0.0
                                        \n\n
                                        Full Changelog: https://github.com/docker/setup-qemu-action/compare/v3.7.0...v4.0.0
                                        \n
                                        v3.7.0
                                        \n\n
                                        Full Changelog: https://github.com/docker/setup-qemu-action/compare/v3.6.0...v3.7.0
                                        \n
                                        v3.6.0
                                        \n\n
                                        Full Changelog: https://github.com/docker/setup-qemu-action/compare/v3.5.0...v3.6.0
                                        \n
                                        v3.5.0
                                        \n
                                          \n
                                        • Bump @\u200bdocker/actions-toolkit from 0.54.0 to 0.56", "merged_at": "2026-03-10T22:37:58Z"}
{"number": 2249, "title": "[infer] expose finish_reason from inference engine", "files": ["src/oumi/core/types/__init__.py", "src/oumi/core/types/conversation.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/bedrock_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/sglang_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/integration/infer/test_infer.py", "tests/integration/infer/test_native_text_inference_engine.py", "tests/unit/core/types/test_conversation.py", "tests/unit/inference/test_anthropic_inference_engine.py", "tests/unit/inference/test_generation_params.py", "tests/unit/inference/test_remote_inference_engine.py", "tests/unit/inference/test_sglang_inference_engine.py"], "area": "inference", "area_votes": {"data": 1, "inference": 7}, "body": "# Description\r\n\r\n\r\n\r\n[infer] expose finish_reason from inference engine\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-08T23:48:49Z"}
{"number": 2246, "title": "Treat empty arrays in synthesis config as unset", "files": ["src/oumi/core/configs/params/synthesis_params.py", "tests/unit/core/configs/params/test_synthesis_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "## Summary\n- Empty lists for list attributes in `GeneralSynthesisParams` are now normalized to `None` during validation, instead of raising `ValueError`\n- This makes empty arrays (`[]`) behave identically to omitted attributes, which is the expected behavior when configs are generated programmatically or deserialized with optional fields\n- Also adds missing test coverage for `multiturn_attributes` empty list handling\n\n## Test plan\n- [x] Updated unit tests to verify empty lists are normalized to `None` for all 9 list attributes\n- [x] All 33 synthesis params tests pass\n- [x] All 1145 config tests pass\n- [x] All 191 synthesis pipeline tests pass", "merged_at": "2026-03-04T22:20:25Z"}
{"number": 2244, "title": "Adding Step To Metrics Callback", "files": ["src/oumi/core/callbacks/metrics_logger_callback.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\nBefore we would never get the step field populated in our metrics callback. This allows us to grab the global step from the state variable. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-04T16:30:02Z"}
{"number": 2234, "title": "[MCP Integration] Phase 1: Package scaffold and dependencies", "files": ["pyproject.toml", "src/oumi/mcp/__init__.py", "src/oumi/mcp/__main__.py", "src/oumi/mcp/config_service.py", "src/oumi/mcp/constants.py", "src/oumi/mcp/docs_service.py", "src/oumi/mcp/environment_service.py", "src/oumi/mcp/job_launcher.py", "src/oumi/mcp/job_logs.py", "src/oumi/mcp/job_registry.py", "src/oumi/mcp/job_runtime.py", "src/oumi/mcp/job_service.py", "src/oumi/mcp/models.py", "src/oumi/mcp/preflight_service.py", "src/oumi/mcp/prompts/__init__.py", "src/oumi/mcp/prompts/analyze.py", "src/oumi/mcp/prompts/cloud_launch.py", "src/oumi/mcp/prompts/eval.py", "src/oumi/mcp/prompts/get_started.py", "src/oumi/mcp/prompts/infer.py", "src/oumi/mcp/prompts/mle_workflow.py", "src/oumi/mcp/prompts/post_training.py", "src/oumi/mcp/prompts/synth.py", "src/oumi/mcp/prompts/train.py", "src/oumi/mcp/server.py", "src/oumi/mcp/sync_service.py", "tests/unit/deploy/__init__.py", "tests/unit/mcp/__init__.py", "tests/unit/mcp/test_client_cwd.py", "tests/unit/mcp/test_config_service.py", "tests/unit/mcp/test_docs_service.py", "tests/unit/mcp/test_job_recovery_and_control.py", "tests/unit/mcp/test_job_registry.py", "tests/unit/mcp/test_server_tools.py", "tests/unit/mcp/test_sync_service.py"], "area": "other", "area_votes": {"infra": 1, "other": 23}, "body": "# Description\n\nPart of the **MCP Integration** PR chain (Phase 1 of 10) - Stage: scaffold\n\n**What changed**: Added the `oumi.mcp` package skeleton with optional dependencies (fastmcp, httpx, huggingface-hub, mcp, pyyaml), the `oumi-mcp` console entry point, and ruff lint ignores.\n\n**Why**: This establishes the package structure and dependency declarations needed for the MCP server integration.\n\n## Related issues\n\n\n\n## Before submitting\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?", "merged_at": "2026-03-25T02:11:28Z"}
{"number": 2233, "title": "Refactored Anthropic Engine Error Check", "files": ["src/oumi/inference/anthropic_inference_engine.py", "tests/unit/inference/test_anthropic_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nAnthropic will show a failure even when partial results are returned. Instead of raising we want to check for partial completion instead of failing the entire call. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-03-03T22:40:47Z"}
{"number": 2230, "title": "Don't hardcode booking link", "files": ["notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Build your own Custom Evaluation (Hallucination Classifier).ipynb", "notebooks/Oumi - Bulk Inference of LLM APIs.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - MiniMath-R1-1.5B.ipynb", "notebooks/Oumi - OpenEnv GRPO with trl.ipynb", "notebooks/Oumi - Quantization Tutorial.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Simple Judge.ipynb", "notebooks/Oumi - Train a Letter Counting Model using GRPO.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 20}, "body": "Rather than hardcoding book a meeting with us Calendarly link, use oumi.ai/book", "merged_at": "2026-03-03T22:33:56Z"}
{"number": 2227, "title": "Updated Early Access Program links", "files": ["notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Build your own Custom Evaluation (Hallucination Classifier).ipynb", "notebooks/Oumi - Bulk Inference of LLM APIs.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - MiniMath-R1-1.5B.ipynb", "notebooks/Oumi - OpenEnv GRPO with trl.ipynb", "notebooks/Oumi - Quantization Tutorial.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Simple Judge.ipynb", "notebooks/Oumi - Train a Letter Counting Model using GRPO.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 20}, "body": "As title above", "merged_at": "2026-02-28T01:45:38Z"}
{"number": 2221, "title": "Update typer requirement from <0.23.2 to <0.24.2", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [typer](https://github.com/fastapi/typer) to permit the latest version.\n
                                          \nRelease notes\n
                                          Sourced from typer's releases.
                                          \n
                                          \n
                                          0.24.1
                                          \n
                                          Internal
                                          \n\n
                                          \n
                                          \n
                                          \nChangelog\n
                                          Sourced from typer's changelog.
                                          \n
                                          \n
                                          0.24.1
                                          \n
                                          Internal
                                          \n
                                            \n
                                          • \ud83d\udc77 Fix CI, do not attempt to build typer-slim, nor typer-cli. PR #1569 by @\u200btiangolo.
                                            • \n
                                          • \u2796 Drop support for typer-slim and typer-cli, no more versions will be released, use only typer. PR #1568 by @\u200btiangolo.
                                            • \n
                                          • \u2b06 Bump rich from 14.3.2 to 14.3.3. PR #1565 by @\u200bdependabot[bot].
                                            • \n
                                          • \u2b06 Bump pydantic-settings from 2.13.0 to 2.13.1. PR #1566=0.11 to >=0.11,<0.13", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on kernels to permit the latest version.\n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
                                            \nDependabot commands and options\n
                                            \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot show  ignore conditions` will show all of the ignore conditions of the specified dependency\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n\n\n
                                            ", "merged_at": "2026-02-27T19:40:01Z"}
{"number": 2219, "title": "Update uvicorn requirement from <0.41.0 to <0.42.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [uvicorn](https://github.com/Kludex/uvicorn) to permit the latest version.\n
                                            \nRelease notes\n
                                            Sourced from uvicorn's releases.
                                            \n
                                            \n
                                            Version 0.41.0
                                            \n
                                            Added
                                            \n
                                              \n
                                            • Add --limit-max-requests-jitter to stagger worker restarts (#2707)
                                              • \n
                                            • Add socket path to scope["server"] (#2561)
                                              • \n
                                            \n
                                            Changed
                                            \n
                                              \n
                                            • Rename LifespanOn.error_occured to error_occurred (#2776)
                                              • \n
                                            \n
                                            Fixed
                                            \n
                                              \n
                                            • Ignore permission denied errors in watchfiles reloader (#2817)
                                              • \n
                                            • Ensure lifespan shutdown runs when should_exit is set during startup (#2812)
                                              • \n
                                            • Reduce the log level of 'request limit exceeded' messages (#2788)
                                              • \n
                                            \n
                                            \n
                                            New Contributors
                                            \n\n
                                            \n
                                            Full Changelog: https://github.com/Kludex/uvicorn/compare/0.40.0...0.41.0
                                            \n
                                            \n
                                            \n
                                            \nChangelog\n
                                            Sourced from uvicorn's changelog.
                                            \n
                                            \n
                                            0.41.0 (February 16, 2026)
                                            \n
                                            Added
                                            \n
                                              \n
                                            • Add --limit-max-requests-jitter to stagger worker restarts (#2707)
                                              • \n
                                            • Add socket path to scope["server"] (#2561)
                                              • \n
                                            \n
                                            Changed
                                            \n
                                              \n
                                            • Rename LifespanOn.error_occured to error_occurred (#2776)
                                              • \n
                                            \n
                                            Fixed
                                            \n
                                              \n
                                            • Ignore permission denied errors in watchfiles reloader (#2817)
                                              • \n
                                            • Ensure lifespan shutdown runs when should_exit is set during startup (#2812)
                                              • \n
                                            • Reduce the log level of 'request limit exceeded' messages (#2788)
                                              • \n
                                            \n
                                            0.40.0 (December 21, 2025)
                                            \n
                                            Remove
                                            \n
                                              \n
                                            • Drop support for Python 3.9 (#2772)
                                              • \n
                                            \n
                                            0.39.0 (December 21, 2025)
                                            \n
                                            Fixed
                                            \n
                                              \n
                                            • Send close frame on ASGI return for WebSockets (<", "merged_at": "2026-03-02T18:53:26Z"}
{"number": 2218, "title": "Update responses requirement from <0.26,>=0.25 to >=0.25,<0.27", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [responses](https://github.com/getsentry/responses) to permit the latest version.\n
                                              \nRelease notes\n
                                              Sourced from responses's releases.
                                              \n
                                              \n
                                              0.26.0
                                              \n
                                                \n
                                              • When using assert_all_requests_are_fired=True, assertions about\nunfired requests are now raised even when an exception occurs in the context manager or\ndecorated function. Previously, these assertions were suppressed when exceptions occurred.\nThis new behavior provides valuable debugging context about which mocked requests were\nor weren't called.
                                                • \n
                                              • Consider the Retry-After header when handling retries
                                                • \n
                                              \n
                                              \n
                                              \n
                                              \nChangelog\n
                                              Sourced from responses's changelog.
                                              \n
                                              \n
                                              0.26.0
                                              \n
                                                \n
                                              • When using assert_all_requests_are_fired=True, assertions about\nunfired requests are now raised even when an exception occurs in the context manager or\ndecorated function. Previously, these assertions were suppressed when exceptions occurred.\nThis new behavior provides valuable debugging context about which mocked requests were\nor weren't called.
                                                • \n
                                              • Consider the Retry-After header when handling retries
                                                • \n
                                              \n
                                              0.25.8
                                              \n
                                                \n
                                              • Fix bug where the content type is always recorded as either text/plain or application/json. See #770
                                                • \n
                                              • Allow asserts on add_callback() matches. See #727
                                                • \n
                                              \n
                                              0.25.7
                                              \n
                                                \n
                                              • Added support for python 3.13
                                                • \n
                                              \n
                                              0.25.6
                                              \n
                                                \n
                                              • Added py.typed to package_data
                                                • \n
                                              \n
                                              0.25.5
                                              \n
                                                \n
                                              • Fix readme issue that prevented 0.25.4 from being published to pypi.
                                                • \n
                                              \n
                                              0.25.4
                                              \n
                                                \n
                                              • Responses can now match requests that use data with file-like objects.\nFiles will be read as bytes and stored in the request mock. See #736
                                                • \n
                                              • RequestsMock.matchers was added. This property is an alias to responses.matchers. See #739
                                                • \n
                                              • Removed tests from packaged wheels. See #746
                                                • \n
                                              • Improved recorder API to ease use in REPL environments. See #745
                                                • \n
                                              \n
                                              0.25.3
                                              \n
                                                \n
                                              • Fixed recorder not saving and loading response headers with yaml files. See #715
                                                • \n
                                              \n
                                              0.25.2
                                              \n
                                                \n
                                              • Mulligan on 0.25.1 to run release pipeline correctly.
                                                • \n
                                              \n\n
                                              \n
                                              ... (truncated)
                                              \n
                                              \n
                                              \nCommits\n
                                                \n
                                              • 94913d0 release: 0.26.0
                                                • \n
                                              • 051b79e Make assert_all_requests_are_fired always assert on exception (#782)
                                                • \n
                                              • 0905cb8 Fix query_param_matcher not matching empty query parameter values (#787)
                                                • \n
                                              • e0c6faa ci(release)", "merged_at": "2026-03-02T18:53:42Z"}
{"number": 2216, "title": "Adding Partial Retry For Base Engine", "files": ["src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nCurrently we don't support the case where when a batch request has partial failures we retry those failures. This is adding support for a standard one pass retry for those.\r\n\r\nIn the future we should add more support for after retry logic pass back the requests that fail so the user can be aware of which requests didn't pass. Right now we are keeping it simple so if after the retry we still have failed requests we fail the entire request. If we wanted to support this logic in the base judge would also need to change. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-23T19:44:02Z"}
{"number": 2214, "title": "[tiny] Add kernels package", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdd kernels package for kernels-community/vllm-flash-attn3 in gold trainer\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-20T18:18:37Z"}
{"number": 2213, "title": "fix(analyze): address post-merge review nits from #2204", "files": ["src/oumi/analyze/__init__.py", "src/oumi/analyze/analyzers/turn_stats.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "## Summary\n\nFollow-up to #2204, addressing two nits from @oelachqar that weren't implemented before merge.\n\n- Use `conversation.first_message()` / `conversation.last_message()` instead of direct list indexing `messages[0]` / `messages[-1]` in `TurnStatsAnalyzer`\n- Sort `__all__` alphabetically in `src/oumi/analyze/__init__.py`\n\n## Before submitting\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n\ud83e\udd16 Generated with [Claude Code](https://claude.ai/code)", "merged_at": "2026-02-19T23:06:42Z"}
{"number": 2212, "title": "[infer] handle temperature constraints for all OpenAI reasoning models", "files": ["src/oumi/inference/openai_inference_engine.py", "tests/unit/inference/test_openai_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nExpand the reasoning model check to include o-series (o1, o3, o4) and GPT-5 family models that only support temperature=1.0.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards LOU-772\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-19T17:58:04Z"}
{"number": 2211, "title": "Add MoE model training configurations for Qwen3 variants", "files": ["configs/recipes/qwen3/sft/235b_lora/train.yaml", "configs/recipes/qwen3_instruct/sft/30b_a3b_lora/train.yaml", "configs/recipes/qwen3_next/sft/80b_a3b_instruct_lora/train.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n- Add Qwen3-Next 80B A3B Instruct LoRA SFT recipe config\r\n- Add Qwen3-235B LoRA configuration\r\n- Add Qwen3-30B A3B Instruct training configuration\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-18T23:17:32Z"}
{"number": 2210, "title": "Update liger-kernel requirement from <0.7,>=0.6 to >=0.6,<0.8", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [liger-kernel](https://github.com/linkedin/Liger-Kernel) to permit the latest version.\n
                                                \nRelease notes\n
                                                Sourced from liger-kernel's releases.
                                                \n
                                                \n
                                                v0.7.0
                                                \n
                                                \ud83d\ude80 Liger-Kernel Now Fully Supports Transformers v5
                                                \n
                                                We\u2019ve added full support for Transformers v5!
                                                \n\ud83d\udd17 linkedin/Liger-Kernel#994
                                                \n
                                                Liger now supports all \ud83e\udd17 Transformers versions \u2265 4.52.0, including the latest v5 release.
                                                \n
                                                Broader compatibility. Seamless upgrades. No version headaches.
                                                \n
                                                Thanks to all the contributors!
                                                \n
                                                What's Changed
                                                \n
                                                \n
                                                \nChangelog\n
                                                Sourced from typer's changelog.
                                                \n
                                                \n
                                                0.23.1
                                                \n
                                                Fixes
                                                \n
                                                  \n
                                                • \ud83d\udc1b Fix TYPER_USE_RICH parsing to allow disabling Rich completely. PR #1539 by @\u200bbckohan.
                                                  • \n
                                                \n
                                                Docs
                                                \n
                                                  \n
                                                • \ud83d\udcdd Remove documentation pages that reference using Click directly. PR #1538 by @\u200bsvlandeg.
                                                  • \n
                                                \n
                                                Internal
                                                \n\n
                                                0.23.0
                                                \n
                                                Breaking Changes
                                                \n
                                                  \n
                                                • \u267b\ufe0f When printing error tracebacks with Rich, default to not showing locals, which are sometimes verbose. PR #1072 by @\u200btiangolo.
                                                  • \n
                                                \n
                                                Docs
                                                \n\n
                                                Internal
                                                \n\n
                                                0.22.0
                                                \n
                                                Breaking Changes
                                                \n
                                                  \n
                                                • \ud83d\udca5 Make typer-slim a shallow wrapper around typer, always requiring rich and shellingham. PR #1522 by @\u200bsvlandeg.
                                                  • \n
                                                \n
                                                0.21.2
                                                \n
                                                Fixes
                                                \n
                                                  \n
                                                • \ud83d\udc1b Fix highlighting of optional variadic argument metavars. PR #1508 by @\u200bBenjyWiener.
                                                  • \n
                                                • \ud83d\udc1b Fix --help text alignment when using typer.style() in option descriptions. PR #1356 by =0.21 to >=0.21,<0.26", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [wandb](https://github.com/wandb/wandb) to permit the latest version.\n
                                                  \nRelease notes\n
                                                  Sourced from wandb's releases.
                                                  \n
                                                  \n
                                                  v0.25.0
                                                  \n
                                                  What's Changed
                                                  \n
                                                  Notable Changes
                                                  \n
                                                  This version drops support for Python 3.8.
                                                  \n
                                                  Added
                                                  \n\n
                                                  Changed
                                                  \n\n
                                                  Fixed
                                                  \n\n
                                                  \n
                                                  \n
                                                  \nChangelog\n
                                                  Sourced from wandb's changelog.
                                                  \n
                                                  \n
                                                  [0.25.0] - 2026-02-12
                                                  \n
                                                  Notable Changes
                                                  \n
                                                  This version drops support for Python 3.8.
                                                  \n
                                                  Added
                                                  \n\n
                                                  Changed
                                                  \n\n
                                                  Fixed
                                                  \n
                                                    \n
                                                  • ", "merged_at": "2026-02-17T22:58:00Z"}
{"number": 2205, "title": "Adding Progress Tracking For Together Batch", "files": ["src/oumi/inference/together_inference_engine.py", "tests/unit/inference/test_together_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2026-02-18T12:40:14Z"}
{"number": 2204, "title": "feat(analyze): config schema, HuggingFace tokenizer, and code cleanup", "files": ["configs/examples/analyze/analyze.yaml", "src/oumi/analyze/__init__.py", "src/oumi/analyze/analyzers/__init__.py", "src/oumi/analyze/analyzers/length.py", "src/oumi/analyze/analyzers/quality.py", "src/oumi/analyze/analyzers/turn_stats.py", "src/oumi/analyze/base.py", "src/oumi/analyze/config.py", "src/oumi/analyze/discovery.py", "src/oumi/analyze/pipeline.py", "src/oumi/analyze/testing/engine.py", "src/oumi/analyze/testing/results.py", "src/oumi/core/analyze/length_analyzer.py", "src/oumi/core/configs/analyze_config.py", "tests/unit/analyze/test_config.py", "tests/unit/analyze/test_length_analyzer.py", "tests/unit/analyze/test_pipeline.py", "tests/unit/analyze/test_quality_analyzer.py", "tests/unit/analyze/test_testing_engine.py", "tests/unit/cli/test_cli_analyze.py", "tests/unit/core/analyze/test_dataset_analyzer.py"], "area": "evaluation", "area_votes": {"configs": 2, "evaluation": 10}, "body": "## Summary\n\nEnhances the analyzer framework with config schema support and HuggingFace tokenizer integration, while removing introspection-based metadata, pipeline tokenizer injection, dead code, and verbose AI-generated comments/docstrings.\n\n## Changes\n\n### Config Schema & Tokenizer Support\n- Add abstract `get_config_schema()` classmethod to analyzers for JSON schema generation from configuration parameters\n- Add `huggingface_tokenizer()` factory for loading HF tokenizers with chat template support\n- Enable `rendered_tokens` metric when using HF tokenizers (counts tokens after applying chat template)\n- Add `_check_supports_disallowed_special()` to handle tokenizers that choke on special tokens like `<|endoftext|>`\n\n### Instance-Aware Metrics\n- Add `get_available_metric_names()` instance method to analyzers\n- Enables per-instance metric filtering (e.g., tiktoken excludes `rendered_tokens`)\n- Add `get_instance_metrics()` in `discovery.py` for API/CLI/UI usage\n\n### Introspection Removal\n- Remove `_require_result_type()`, `_get_result_type()`, and all `__orig_bases__` / `get_args` / `get_origin` introspection from `BaseAnalyzer`\n- `get_result_schema()`, `get_metric_names()`, `get_metric_descriptions()` are now abstract \u2014 subclasses implement them explicitly\n\n### Pipeline: Remove Tokenizer Injection\n- Remove `_inject_tokenizer()`, `_TOKENIZER_ATTR`, `tokenizer` and `tiktoken_encoding` constructor params, and `tiktoken` dependency from the pipeline\n- Analyzers are now fully responsible for their own tokenizer (passed via constructor factory methods)\n- This also fixes a bug where `_supports_disallowed_special` was stale after post-init injection\n\n### Pipeline: Analyzer Name Deduplication\n- Add `_assigned_names` set to auto-generate unique suffixed names (`LengthAnalyzer_2`, etc.) when multiple unnamed analyzers of the same type are added\n- Inline `_get_analyzer_scope()` helper\n\n### Config Refactoring\n- Remove dead `AnalyzerType` enum\n- Remove unused `get_test_configs()` method\n- Refactor `from_dict()` into `_parse_analyzers()`, `_parse_custom_metrics()`, `_parse_tests()`\n- Fix `to_dict()` / `from_dict()` asymmetry \u2014 `to_dict()` now serializes `eval_name`, `parent_eval_id`, `output_schema`, and `depends_on`\n- Fix `_parse_custom_metrics()` mutating its input dict via `.pop()`\n- Remove deprecated `tokenizer_config` field and migration logic from core config\n- Move `import yaml` to top-level\n\n### Test Engine Improvements\n- Threshold test semantics: matching samples are \"flagged\", test passes when none match\n- Better failure reasons: `\"{value} satisfies {op} {threshold}\"`\n- Replace magic numbers with `MAX_SAMPLE_INDICES` and `MAX_FAILURE_REASONS` constants\n- Remove unreachable `bool` type handling in `_get_actual_value()` (bool is a subclass of int)\n- Inline `_calculate_percentage()` at call sites\n\n### Discovery Cleanup\n- Remove dead `get_metric_path()` and `generate_test_template()` functions\n- Remove redundant deduplication pass in `print_analyzer_metrics()`\n- Move registry imports to top-level\n- Type `console` parameter as `Console` instead of `Any`\n\n### Code Quality\n- Trim verbose docstrings on private methods and trivial properties throughout\n- Remove restating inline comments (e.g., `# Dataset source`, `# Type aliases for consistency`)\n- Remove section divider comments\n- Move lazy stdlib/required-dep imports to top-level (`json`, `graphlib`, `yaml`, `copy`, `transformers`)\n- Add type-safe `cast()` calls in pipeline analyzer dispatch\n- Remove duplicate `@register_sample_analyzer` from `core.analyze.length_analyzer`\n\n## Test Plan\n\n- [x] All analyze unit tests pass\n- [x] Test fixtures updated with explicit abstract method implementations\n- [x] Pipeline tokenizer injection tests removed (feature removed)\n- [x] Test engine key convention updated from class names to instance IDs\n- [x] New test for multi-instance metric resolution\n\n## Related Issues\n\nBuilds on #2192 (testing framework)", "merged_at": "2026-02-19T22:55:48Z"}
{"number": 2203, "title": "Batch Support For Judge", "files": ["src/oumi/judges/base_judge.py", "tests/unit/judges/test_base_judge.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nThis PR provides new methods in the Judge to support batch inference.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-10T19:44:18Z"}
{"number": 2202, "title": "fix: format lambda expressions in test_params validation config", "files": ["src/oumi/core/configs/params/test_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nRun `pre-commit run --all` to address CI failures in the pre-commit hooks\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-10T15:18:06Z"}
{"number": 2201, "title": "Add token usage accumulation to AttributeSynthesizer", "files": ["src/oumi/core/synthesis/attribute_synthesizer.py", "tests/unit/core/synthesis/test_attribute_synthesizer.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n- Add `_total_input_tokens` and `_total_output_tokens` tracking to `AttributeSynthesizer`\r\n- Accumulate token counts from inference response metadata after `synthesize()` and `get_batch_results()` calls\r\n- Expose via `total_input_tokens` and `total_output_tokens` properties\r\n\r\nFollows the same pattern as #2199 for token tracking.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards LOU-528\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-10T17:07:28Z"}
{"number": 2199, "title": "Add token usage accumulators to BaseJudge", "files": ["src/oumi/judges/base_judge.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n- Add `_total_input_tokens` and `_total_output_tokens` accumulators to `BaseJudge`\r\n- Accumulate token counts from `conversation.metadata[\"usage\"]` in `_infer()` after each inference batch\r\n- Expose via `total_input_tokens` and `total_output_tokens` read-only properties\r\n\r\nBuilds on #2198 which stores usage metadata in conversations. This enables downstream consumers (e.g., the judge workflow in the API repo) to read token counts.\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards LOU-571\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-09T23:54:04Z"}
{"number": 2198, "title": "Add inference usage metadata for RemoteInferenceEngine", "files": ["src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_anthropic_inference_engine.py", "tests/unit/inference/test_openrouter_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\nExtract the token usage metadata from the API response. Tested with a handful of inference providers. \r\n```\r\n  OpenAI:\r\n    usage    : {'prompt_tokens': 14, 'completion_tokens': 6, 'total_tokens': 20}\r\n\r\n  Anthropic:\r\n    usage    : {'prompt_tokens': 14, 'completion_tokens': 8, 'total_tokens': 22}\r\n\r\n  Together:\r\n    usage    : {'prompt_tokens': 42, 'completion_tokens': 5, 'total_tokens': 47}\r\n\r\n  Fireworks:\r\n    usage    : {'prompt_tokens': 42, 'completion_tokens': 5, 'total_tokens': 47}\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes LOU-517\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-09T20:55:07Z"}
{"number": 2195, "title": "Add news items for February 2026 updates", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Oumi + Lambda announcement blog and demo video\r\n\r\n", "merged_at": "2026-02-06T21:57:27Z"}
{"number": 2194, "title": "[infer] Add batch api support to anthropic,  fireworks, together engines", "files": ["src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/fireworks_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/together_inference_engine.py", "tests/e2e/test_batch_inference.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 4}, "body": "# Description\r\n\r\n\r\n\r\n- [infer] Add batch api support to anthropic,  fireworks, together engines\r\n- Add e2e test\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-06T16:48:25Z"}
{"number": 2192, "title": "add testing framework to analyze", "files": ["src/oumi/analyze/__init__.py", "src/oumi/analyze/config.py", "src/oumi/analyze/testing/__init__.py", "src/oumi/analyze/testing/engine.py", "src/oumi/analyze/testing/results.py", "src/oumi/core/configs/params/test_params.py", "tests/unit/analyze/test_testing_engine.py"], "area": "evaluation", "area_votes": {"evaluation": 3, "configs": 1}, "body": "# Description\r\nAdding the testing framework and engine\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-10T04:29:34Z"}
{"number": 2191, "title": "add upper bound for skypilot", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-03T23:10:55Z"}
{"number": 2190, "title": "upgrade skypilot version to latest", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes LOU-534\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-03T22:58:11Z"}
{"number": 2187, "title": "fix pyright failure by removing warning", "files": [".pre-commit-config.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\nPyRight gets 0 erros but 124 warnings but still fails. By removing the warning, the test passes\r\n0 errors, 124 warnings, 0 informations\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-03T18:58:03Z"}
{"number": 2184, "title": "Address Llama4Scout LoRA training config", "files": ["configs/recipes/llama4/sft/scout_instruct_lora/train.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "\r\n# Description\r\n\r\n\r\n\r\n\r\n1. Disable torch.compile to avoid Dynamo pin_memory issue in transformers library\r\n\r\n2. Fix OOM host memory issues in Llama4 Scout LoRA training config\r\n\r\n  -- Reduce data loading parallelism (num_workers=0, prefetch_factor=2)\r\n     to lower memory footprint.\r\n  -- Disable FSDP CPU offload to prevent host OOM errors by keeping all data on GPU.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-02-11T13:49:27Z"}
{"number": 2183, "title": "feat: add typed analyzer core framework", "files": ["src/oumi/analyze/__init__.py", "src/oumi/analyze/analyzers/__init__.py", "src/oumi/analyze/base.py", "src/oumi/analyze/config.py", "src/oumi/analyze/discovery.py", "src/oumi/analyze/pipeline.py", "src/oumi/analyze/utils/__init__.py", "src/oumi/analyze/utils/dataframe.py", "src/oumi/core/configs/params/test_params.py", "tests/unit/analyze/__init__.py", "tests/unit/analyze/test_config.py", "tests/unit/analyze/test_dataframe_utils.py", "tests/unit/analyze/test_pipeline.py"], "area": "evaluation", "area_votes": {"evaluation": 5, "configs": 1}, "body": "# Description\r\n\r\nAdds a typed analyzer core framework with the following components:\r\n\r\n- **Base Analyzer Classes**: Defines base classes for different analyzer scopes:\r\n  - `MessageAnalyzer`: Analyzes individual messages\r\n  - `ConversationAnalyzer`: Analyzes complete conversations\r\n  - `DatasetAnalyzer`: Analyzes entire datasets (cross-sample operations)\r\n  - `PreferenceAnalyzer`: Analyzes preference pairs (for DPO data)\r\n\r\n- **Configuration System**: Type-safe configuration classes for analyzers\r\n- **Plugin Discovery**: Auto-discovery mechanism for analyzer plugins\r\n- **Pipeline System**: Orchestrates multiple analyzers\r\n- **Registry**: Central registry for managing analyzer instances\r\n- **Results Framework**: Strongly-typed Pydantic models for analyzer results\r\n- **DataFrame Utilities**: Helper functions for dataframe operations\r\n\r\n## Files Added\r\n\r\n- `src/oumi/analyze/base.py` (381 lines) - Base analyzer classes\r\n- `src/oumi/analyze/config.py` (414 lines) - Configuration system\r\n- `src/oumi/analyze/discovery.py` (426 lines) - Plugin discovery\r\n- `src/oumi/analyze/pipeline.py` (361 lines) - Pipeline orchestration\r\n- `src/oumi/analyze/registry.py` (83 lines) - Analyzer registry\r\n- `src/oumi/analyze/results/__init__.py` (32 lines) - Results framework\r\n- `src/oumi/analyze/utils/dataframe.py` (209 lines) - DataFrame utilities\r\n- Supporting `__init__.py` files\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n", "merged_at": "2026-02-03T21:01:27Z"}
{"number": 2180, "title": "[Evaluations] Fixing Typecheker errors", "files": ["src/oumi/core/evaluation/backends/lm_harness.py", "tests/unit/core/evaluation/test_backend_lm_harness.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\n*********** Main fix: Typechecker errors (not actual bugs) *********** \r\n\r\n1) /home/runner/work/oumi/oumi/src/oumi/core/evaluation/backends/lm_harness.py:325:9 - error: Argument of type \"dict[str | ConfigurableGroup, Task | dict[Unknown, Unknown]]\" cannot be assigned to parameter \"model_args\" of type \"str | dict[str, str | int | float] | None\" in function \"simple_evaluate\"\r\n  - What was the problem: Type checker confused evaluate() with simple_evaluate() which have different signatures  => Added # type: ignore[arg-type] \r\n\r\n2) /home/runner/work/oumi/oumi/src/oumi/core/evaluation/backends/lm_harness.py:377:49 - error: Cannot access attribute \"get_model_info\" for class \"LM\" Attribute \"get_model_info\" is unknown (reportAttributeAccessIssue)\r\n  - What was the problem: Type checker complained about get_model_info() method that exists on some LM classes but not in base type stubs => Added # type: ignore[attr-defined] (note that: code already safely handles this corner case with hasattr()) \r\n\r\n*********** Testing *********** \r\n\r\nI wasn't able to kick off E2E tests, but a toy use case works, meaning we call the API the right way:\r\n\r\n```\r\nfrom oumi import evaluate\r\nfrom oumi.core.configs import EvaluationConfig\r\n\r\nconfig_str = \"\"\"\r\nmodel:\r\n  model_name: \"Qwen/Qwen2.5-0.5B-Instruct\"\r\n  trust_remote_code: True\r\n\r\ntasks:\r\n  - evaluation_backend: lm_harness\r\n    task_name: mmlu_abstract_algebra \r\n    num_samples: 30\r\n\r\ngeneration:\r\n  batch_size: 16\r\n  max_new_tokens: 512\r\n  temperature: 0.0\r\n\r\ninference_engine: NATIVE\r\n\"\"\"\r\n\r\neval_config = EvaluationConfig.from_str(config_str)\r\nresult = evaluate(config=eval_config)\r\n```\r\n\r\nResult:\r\n```\r\n[{'results': {'mmlu_abstract_algebra': {'alias': 'abstract_algebra', 'acc,none': 0.3, 'acc_stderr,none': 0.0850962943396763}}}]\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-28T16:05:39Z"}
{"number": 2178, "title": "Return INIT skypilot clusters in get clusters by class", "files": ["src/oumi/launcher/clouds/sky_cloud.py", "tests/unit/launcher/clouds/test_sky_cloud.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description\r\n\r\n\r\n\r\nCheck for and return both INIT and UP skypilot clusters in the skycloud get clusters by class method.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-27T01:49:43Z"}
{"number": 2177, "title": "Update torch requirement from <2.10.0,>=2.6 to >=2.6,<2.11.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [torch](https://github.com/pytorch/pytorch) to permit the latest version.\n
                                                    \nRelease notes\n
                                                    Sourced from torch's releases.
                                                    \n
                                                    \n
                                                    PyTorch 2.10.0 Release Notes
                                                    \n\n
                                                    Highlights
                                                    \n\n
                                                    For more details about these highlighted features, you can look at the release blogpost. Below are the full release notes for this release.
                                                    \n
                                                    Backwards Incompatible Changes
                                                    \n
                                                    Dataloader Frontend
                                                    \n
                                                      \n
                                                    • Removed unused data_source argument from Sampler (#163134). This is a no-op, unless you have a custom sampler that uses this argument. Please update your custom sampler accordingly.
                                                      • \n
                                                    • Removed deprecated imports for torch.utils.data.datapipes.iter.grouping (#163438). from torch.utils.data.datapipes.iter.grouping import SHARDING_PRIORITIES, ShardingFilterIterDataPipe is no longer supported. Please import from torch.utils.data.datapipes.iter.sharding instead.
                                                      • \n
                                                    \n
                                                    torch.nn
                                                    \n
                                                      \n
                                                    • Remove Nested Jagged Tensor support from nn.attention.flex_attention (#161734)
                                                      • \n
                                                    \n\n
                                                    \n
                                                    ... (truncated)
                                                    \n
                                                    \n
                                                    \nChangelog\n
                                                    Sourced from torch's changelog.
                                                    \n
                                                    \n
                                                    Releasing PyTorch
                                                    \n\n
                                                      \n
                                                    • Release Compatibility Matrix\n\n
                                                      • \n
                                                    • Release Cadence
                                                      • \n
                                                    • General Overview\n\n
                                                      • \n
                                                    • Cutting a release branch preparations
                                                      • \n
                                                    • Cutting release branches\n
                                                        \n
                                                      • pytorch/pytorch
                                                        • \n
                                                      • PyTorch ecosystem libraries
                                                        • \n
                                                      • Making rele", "merged_at": "2026-02-03T19:34:08Z"}
{"number": 2175, "title": "Update skypilot requirement from <0.11,>=0.10.2 to >=0.10.2,<0.12", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "[//]: # (dependabot-start)\n\u26a0\ufe0f  **Dependabot is rebasing this PR** \u26a0\ufe0f \n\nRebasing might not happen immediately, so don't worry if this takes some time.\n\nNote: if you make any changes to this PR yourself, they will take precedence over the rebase.\n\n---\n\n[//]: # (dependabot-end)\n\nUpdates the requirements on [skypilot](https://github.com/skypilot-org/skypilot) to permit the latest version.\n
                                                        \nRelease notes\n
                                                        Sourced from skypilot's releases.
                                                        \n
                                                        \n
                                                        SkyPilot v0.11.1
                                                        \n
                                                        This patch release is a minor bump from v0.11.0 to to get you the latest fixes:
                                                        \n
                                                          \n
                                                        • #8285: Fix an issue where the API server request database is incorrectly locked causing an irresponsive API server with an error:\n
                                                          sqlite3.OperationalError: database is locked\n
                                                          \n
                                                          • \n
                                                        \n
                                                        See the full v0.11 release notes for everything new in SkyPilot v0.11!
                                                        \n
                                                        \n
                                                        \n
                                                        \nCommits\n
                                                          \n
                                                        • a7380a5 Release 0.11.1 (#8310)
                                                          • \n
                                                        • 4584c34 Cherry-pick: more logs about request operations #8270 #8271 (#8309)
                                                          • \n
                                                        • 6c979ce Fixed stall txn block write operation (#8285)
                                                          • \n
                                                        • b2f3519 Release 0.11.0 (#8255)
                                                          • \n
                                                        • 59ff48f Release 0.11.0rc2 (#8236)
                                                          • \n
                                                        • 078dad7 [core] restore cluster_name_on_cloud from cluster yaml (#8233)
                                                          • \n
                                                        • 0207108 Fixed incorrect user info in handlers (#8199 #8209) (#8234)
                                                          • \n
                                                        • 4546cbc [Core] Put Daemonize Call Back to Make Sky Cancel Reliable (#8203) (#8208)
                                                          • \n
                                                        • 2f0b272 Fix ephemeral volume creation (#8179) (#8194)
                                                          • \n
                                                        • =0.21 to >=0.21,<0.26", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "[//]: # (dependabot-start)\n\u26a0\ufe0f  **Dependabot is rebasing this PR** \u26a0\ufe0f \n\nRebasing might not happen immediately, so don't worry if this takes some time.\n\nNote: if you make any changes to this PR yourself, they will take precedence over the rebase.\n\n---\n\n[//]: # (dependabot-end)\n\nUpdates the requirements on [torchvision](https://github.com/pytorch/vision) to permit the latest version.\n
                                                          \nRelease notes\n
                                                          Sourced from torchvision's releases.
                                                          \n
                                                          \n
                                                          TorchVision 0.25 Release
                                                          \n
                                                          TorchVision 0.25 is out! It is compatible with torch 2.10. It's a small release that comes with the following improvements:
                                                          \n
                                                          Enhancement
                                                          \n
                                                          [transforms] KeyPoints aren't clamped by default anymore after a transform. This is a bug-fix that comes with a change of behavior. We also added the SanitizeKeyPoints transform to remove keypoints outside of the image area (#9236, #9235)\n[utils] draw_bounding_boxes now supports a label_background_colors parameter (#9204)\n[io] Fixed an issue in the GIF decoder (decode_gif, decode_image) which affected some (not all) animated GIFs. (#9241)\n[misc] Various code-quality and docs improvements (#9218, #9270, #9250, #9247)
                                                          \n
                                                          Contributors
                                                          \n
                                                          \ud83c\udf89 We're grateful for our community, which helps us improve Torchvision by submitting issues and PRs, and providing feedback and suggestions. The following persons have contributed patches for this release:
                                                          \n
                                                          Andrei Moraru, Andrey Talman, Antoine Simoulin , Arun Prakash A, Bj\u00f6rn Barz, Huy Do, Nicolas Hug, Sean Gilligan, Wes Castro, Zhitao Yu
                                                          \n
                                                          \n
                                                          \n
                                                          \nCommits\n
                                                            \n
                                                          • 8ac84ee [Cherry-pick for release/0.25] Fix the dataset test failures (#9327) (#9329)
                                                            • \n
                                                          • 82df5f5 [Release 0.25] Update test-infra refs (#9312)
                                                            • \n
                                                          • 8886375 [Release 0.25] Remove CVCUDA stuff - it'll be released in 0.26 (#9313)
                                                            • \n
                                                          • 5e40fe8 [Release 0.25] Remove prototype stuff (#9310)
                                                            • \n
                                                          • 85b9027 [Release 0.25] update version number (#9311)
                                                            • \n
                                                          • 96e7797 Remove r2 upload for wheels  (#9303)
                                                            • \n
                                                          • 6b56de1\r\n\r\n- [infer] Add docs for engines support infer_batch, + raise explicit error if not supported\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-22T22:01:30Z"}
{"number": 2169, "title": "[docs] update installation instructions", "files": ["README.md", "docs/get_started/installation.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-22T21:53:28Z"}
{"number": 2167, "title": "[docs] finish docs clean-up & refresh", "files": ["docs/_static/custom.css", "docs/_static/custom.js", "docs/_templates/oumi-footer-links.html", "docs/conf.py", "docs/index.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] finish docs clean-up & refresh\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-22T05:12:06Z"}
{"number": 2166, "title": "[bug] revert trl version update & make sure tests run on pyproject.toml changes", "files": [".github/workflows/gpu_tests.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Revert trl 0.27 upgrade as it contains several breaking changes\r\n- Update gpu tests to run on any version change to prevent this in the future\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-22T03:48:24Z"}
{"number": 2163, "title": "Update trl requirement from <0.27,>=0.24 to >=0.24,<0.28", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [trl](https://github.com/huggingface/trl) to permit the latest version.\n
                                                            \nRelease notes\n
                                                            Sourced from trl's releases.
                                                            \n
                                                            \n
                                                            v0.27.0
                                                            \n
                                                            Features
                                                            \n\n
                                                            Experimental
                                                            \n
                                                              \n
                                                            • Move AutoModelForCausalLMWithValueHead and AutoModelForSeq2SeqLMWithValueHead to experimental by @\u200bqgallouedec in huggingface/trl#4654
                                                              • \n
                                                            • Move DPODataCollatorWithPadding to experimental.utils  by @\u200bqgallouedec in huggingface/trl#4667
                                                              • \n
                                                            • Move DataCollatorForChatML to experimental.utils by @\u200bqgallouedec in huggingface/trl#4668
                                                              • \n
                                                            • Move add_bos_token_if_needed and add_eos_token_if_needed to experimental.utils by @\u200bqgallouedec in huggingface/trl#4674
                                                              • \n
                                                            • Move truncate_right and SIMPLE_CHAT_TEMPLATE to experimental.utils by @\u200bqgallouedec in =0.21 to >=0.21,<0.25", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [wandb](https://github.com/wandb/wandb) to permit the latest version.\n
                                                              \nRelease notes\n
                                                              Sourced from wandb's releases.
                                                              \n
                                                              \n
                                                              v0.24.0
                                                              \n
                                                              What's Changed
                                                              \n
                                                              Notable Changes
                                                              \n
                                                              This version removes the legacy, deprecated wandb.beta.workflows module, including its log_model()/use_model()/link_model() functions. This is formally a breaking change.
                                                              \n
                                                              Added
                                                              \n
                                                                \n
                                                              • wandb agent and wandb.agent() now accept a forward_signals flag (CLI: --forward-signals/-f) to relay SIGINT/SIGTERM and other catchable signals from the agent to its sweep child runs, enabling cleaner shutdowns when you interrupt an agent process (@\u200bkylegoyette, @\u200bdomphan-wandb in wandb/wandb#9651)
                                                                • \n
                                                              • wandb beta sync now supports a --live option for syncing a run while it's being logged (@\u200btimoffex in wandb/wandb#11079)
                                                                • \n
                                                              \n
                                                              Removed
                                                              \n
                                                                \n
                                                              • Removed the deprecated wandb.beta.workflows module, including its log_model(), use_model(), and link_model() functions, and whose modern successors are the Run.log_artifact, Run.use_artifact, and Run.link_artifact methods, respectively (@\u200btonyyli-wandb in [TODO: PR link])
                                                                • \n
                                                              \n
                                                              Fixed
                                                              \n\n
                                                              \n
                                                              \n
                                                              \nChangelog\n
                                                              Sourced from wandb's changelog.
                                                              \n
                                                              \n
                                                              [0.24.0] - 2026-01-13
                                                              \n
                                                              Notable Changes
                                                              \n
                                                              This version removes the legacy, depr", "merged_at": "2026-01-19T17:44:27Z"}
{"number": 2161, "title": "[tiny] explicitly set wandb as 3rd party to avoid isort issues", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- explicitly set wandb as 3rd party to avoid isort issues\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-16T07:36:29Z"}
{"number": 2160, "title": "[tiny] tweaks to main cli", "files": ["src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [tiny] tweaks to main cli\r\n- enable completions install, disable telemetry for simple -h queries\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-16T07:36:11Z"}
{"number": 2157, "title": "Fix generic print output in the Oumi Tour notebook", "files": ["notebooks/Oumi - A Tour.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nThis change prints the contents of the log directory instead of the iterator object.\r\n\r\nBefore:\r\n\"image\"\r\n\r\nAfter:\r\n\"image\"\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-13T00:47:24Z"}
{"number": 2156, "title": "Update gemma3-4b-it SFT training config after offline tuning", "files": ["configs/recipes/gemma3/sft/4b_full/train.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\nUpdate the gemma3-4b-it training config with improved defaults for full-rank SFT training.\r\n\r\nThrough offline experiments, we found that this training config is effective at improving target task performance without catastrophic regression on general capabilities for both an open-ended data analysis task (tat-qa, train n=13k) and a ternary classification task (pubmedqa, train n=800). Training with a lower LR (e.g. 1e-5) and additional steps produces similar performance on these tasks. Experiment workflows documented in branch [`lefft/ent-train-expts`](https://github.com/oumi-ai/oumi/tree/lefft/ent-train-expts).\r\n\r\n\r\n\r\nFurther iterations of config optimization are likely to produce benefits, depending on the relevant downstream task(s). \r\n\r\n## Related issues\r\n\r\nRelated to OPE-542\r\n\r\n## Before submitting\r\n\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed? [N/A]\r\n", "merged_at": "2026-01-13T04:12:12Z"}
{"number": 2155, "title": "[ops] Add install.sh script", "files": ["install.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [ops] Add install.sh script \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-12T18:45:58Z"}
{"number": 2154, "title": "[docs] cleanup css style for python reference page, remove reading progress indicator", "files": ["docs/_static/custom.css", "docs/_static/custom.js"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] cleanup css style for python reference page, remove reading progress indicator\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-12T18:31:40Z"}
{"number": 2153, "title": "Update typer requirement from <0.21.1 to <0.21.2", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [typer](https://github.com/fastapi/typer) to permit the latest version.\n
                                                              \nRelease notes\n
                                                              Sourced from typer's releases.
                                                              \n
                                                              \n
                                                              0.21.1
                                                              \n
                                                              Fixes
                                                              \n
                                                                \n
                                                              • \ud83d\udc1b Fix escaping in help text when rich is installed but not used. PR #1089 by @\u200bsvlandeg.
                                                                • \n
                                                              \n
                                                              Internal
                                                              \n\n
                                                              \n
                                                              \n
                                                              \nChangelog\n
                                                              Sourced from typer's changelog.
                                                              \n
                                                              \n
                                                              0.21.1
                                                              \n
                                                              Fixes
                                                              \n
                                                                \n
                                                              • \ud83d\udc1b Fix escaping in help text when rich is installed but not used. PR #1089 by @\u200bsvlandeg.
                                                                • \n
                                                              \n
                                                              Internal
                                                              \n
                                                                \n
                                                              • \u2b06\ufe0f  Migrate to uv. PR #1472 by @\u200bDoctorJohn.
                                                                • \n
                                                              • \u2b06 Bump mypy from 1.18.2 to 1.19.1. PR #1469 by @\u200bdependabot[bot].
                                                                • \n
                                                              • \u2b06 Bump actions/checkout from 5 to 6. PR ", "merged_at": "2026-01-12T03:31:01Z"}
{"number": 2149, "title": "[docker] Add tmux to docker image", "files": ["Dockerfile"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n[docker] Add tmux to docker image\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-06T00:50:25Z"}
{"number": 2147, "title": "Update pillow requirement from <12.1,>=11.3 to >=11.3,<12.2", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [pillow](https://github.com/python-pillow/Pillow) to permit the latest version.\n
                                                                \nRelease notes\n
                                                                Sourced from pillow's releases.
                                                                \n
                                                                \n
                                                                12.1.0
                                                                \n
                                                                https://pillow.readthedocs.io/en/stable/releasenotes/12.1.0.html
                                                                \n
                                                                Deprecations
                                                                \n\n
                                                                Documentation
                                                                \n\n
                                                                Dependencies
                                                                \n
                                                                  \n
                                                                • Update harfbuzz to 12.3.0 #9355 [@\u200bradarhere]
                                                                  • \n
                                                                • Update xz to 5.8.2 #9343 [@\u200bradarhere]
                                                                  • \n
                                                                • Updated libjpeg-turbo to 3.1.3 #9333 [@\u200bradarhere]
                                                                  • \n
                                                                • Updated zlib-ng to 2.3.2 #9324 [@\u200bradarhere]
                                                                  • \n
                                                                • Updated libpng to 1.6.53 #9325 [@\u200bradarhere]
                                                                  • \n
                                                                • Update actions/checkout action to v6 #9323 [@renovate[bot]]
                                                                  • \n
                                                                • Update dependency mypy to v1.19.0 #9322 [@renovate[bot]]
                                                                  • \n
                                                                • Updated libpng to 1.6.51 #9305 [@\u200bradarhere]
                                                                  • \n
                                                                • Updated brotli to 1.2.0 #9284 [@\u200bradarhere]
                                                                  • \n
                                                                • Update libimagequant to 4.4.1 \r\n\r\n- [ops] disable telemetry during CI, update variables\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-02T19:52:20Z"}
{"number": 2144, "title": "[datasets] add RaR datasets and base rubric dataset classes", "files": ["src/oumi/core/datasets/__init__.py", "src/oumi/core/datasets/base_rubric_dataset.py", "src/oumi/datasets/__init__.py", "src/oumi/datasets/grpo/__init__.py", "src/oumi/datasets/grpo/rar_dataset.py", "src/oumi/datasets/grpo/rlvr_rubric.py"], "area": "data", "area_votes": {"data": 3}, "body": "# Description\r\n\r\n\r\n\r\n- [datasets] add RaR datasets and base rubric dataset classes\r\n- these datasets can be used with the GRPO `rubric_reward` reward function\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2026-01-02T18:40:21Z"}
{"number": 2141, "title": "[docker] add useful utils, enable arm", "files": [".github/workflows/release_docker.yaml", "Dockerfile"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- [docker] add useful utils, enable arm\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-31T10:08:53Z"}
{"number": 2139, "title": "Update pycares requirement from <5.0.0 to <6.0.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "[//]: # (dependabot-start)\n\u26a0\ufe0f  **Dependabot is rebasing this PR** \u26a0\ufe0f \n\nRebasing might not happen immediately, so don't worry if this takes some time.\n\nNote: if you make any changes to this PR yourself, they will take precedence over the rebase.\n\n---\n\n[//]: # (dependabot-end)\n\nUpdates the requirements on [pycares](https://github.com/saghul/pycares) to permit the latest version.\n
                                                                  \nChangelog\n
                                                                  Sourced from pycares's changelog.
                                                                  \n
                                                                  \n
                                                                  No longer updated. Please check the commits.
                                                                  \n
                                                                  Version 4.0.0
                                                                  \n
                                                                    \n
                                                                  • doc: uppdate README
                                                                    • \n
                                                                  • core: add support for CAA queries
                                                                    • \n
                                                                  • core: add support for getaddrinfo()
                                                                    • \n
                                                                  • doc: update README
                                                                    • \n
                                                                  • core: add ability to use the system installed c-ares
                                                                    • \n
                                                                  • misc: set version to 4.0.0
                                                                    • \n
                                                                  • test: remove TTL tests
                                                                    • \n
                                                                  • core: update c-ares and use a submodule
                                                                    • \n
                                                                  • core: drop bundled c-ares version
                                                                    • \n
                                                                  • misc: drop tasks.py
                                                                    • \n
                                                                  \n
                                                                  Version 3.2.0
                                                                  \n
                                                                    \n
                                                                  • misc: add Python 3.9 classifier
                                                                    • \n
                                                                  • core: drop py3.5 from CI and documentation
                                                                    • \n
                                                                  • ci: run tests in Python 3.9
                                                                    • \n
                                                                  • ci: fix SDK path in appveyor
                                                                    • \n
                                                                  • ci: fix VS linker in appveyor
                                                                    • \n
                                                                  • ci: update python installer script in appveyor
                                                                    • \n
                                                                  • misc: add compiled windows dll to gitignore
                                                                    • \n
                                                                  • test: skip SOA non-ascii test due to changes in remote host
                                                                    • \n
                                                                  • test: fix broken chunked TXT test due to changes in remote host
                                                                    • \n
                                                                  • test: skip ANY test due to problems with mac
                                                                    • \n
                                                                  • ci: add action to release wheels to PyPi
                                                                    • \n
                                                                  • ci: drop AppVeyor
                                                                    • \n
                                                                  • ci: don't fail fast
                                                                    • \n
                                                                  • ci: stop testing on Travis
                                                                    • \n
                                                                  • test: add generic way to check for a CI
                                                                    • \n
                                                                  • test: relax check
                                                                    • \n
                                                                  • test: try to avoid spurious CI failures
                                                                    • \n
                                                                  • ci: test all platforms on GH actions
                                                                    • \n
                                                                  • ci: build wheels on GH Actions
                                                                    • \n
                                                                  • build: fix build error on macOS
                                                                    • \n
                                                                  • ci: explicitly set Python versions to build wheels for
                                                                    • \n
                                                                  • ci: update cibuildwheel
                                                                    • \n
                                                                  • ci: re-add IDNA test
                                                                    • \n
                                                                  \n
                                                                  Version 3.1.1
                                                                  \n
                                                                    \n
                                                                  • ffi: new style callbacks
                                                                    • \n
                                                                  \n
                                                                  Version 3.1.0
                                                                  \n
                                                                    \n
                                                                  • misc: add Python 3.8 classifier
                                                                    • \n
                                                                  • (origin/master, origin/HEAD) build: use Travis to build Python Wheels
                                                                    • \n
                                                                  \n\n
                                                                  \n
                                                                  ... (truncated)
                                                                  \n
                                                                  \n
                                                                  \nCommits\n
                                                                    \n
                                                                  • 8ef6485 Update README
                                                                    • \n
                                                                  • db895c8 Update documentation
                                                                    • \n
                                                                  • 36950d8 Extend timeout for test
                                                                    • \n
                                                                  • 188ee73 Fix crash if socket state callback is called during destruction
                                                                    • \n
                                                                  • 9b38166 Explicitly close channel in test tearDown
                                                                    • \n
                                                                  • 0cb3c42 Workaround test failure on Windows 11 ARM
                                                                    • \n
                                                                  • 27aeb52 Make test more resilient
                                                                    • \n
                                                                  • 1a79e76 Fixup tests
                                                                    • \n
                                                                  • d7db89d Make arguments to Channel keyword only
                                                                    • \n
                                                                  • 64aaa54 Remove event_thread opt", "merged_at": "2026-01-15T18:59:25Z"}
{"number": 2137, "title": "Update torchao requirement from <0.15,>=0.12 to >=0.12,<0.16", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [torchao](https://github.com/pytorch/ao) to permit the latest version.\n
                                                                    \nRelease notes\n
                                                                    Sourced from torchao's releases.
                                                                    \n
                                                                    \n
                                                                    v0.15.0
                                                                    \n
                                                                    Highlights
                                                                    \n
                                                                    We are excited to announce the 0.15.0 release of torchao! This release adds:
                                                                    \n
                                                                      \n
                                                                    • MXFP8 MoE training demonstrates 1.2x e2e training speedup with identical convergence versus bf16, training Llama4 Scout on a 64 node GB200 Crusoe cluster!
                                                                      • \n
                                                                    • MXFP8 MoE kernels shipped with torchao builds for CUDA 12.8+ (just pip install instead of building from source to use!)
                                                                      • \n
                                                                    • Safetensors enablement
                                                                      • \n
                                                                    • Quantization with parameter level targeting
                                                                      • \n
                                                                    \n
                                                                    MXFP8 MoE training demonstrates 1.2x e2e training speedup with identical convergence versus bf16, training Llama4 Scout on a 64 node GB200 Crusoe cluster
                                                                    \n
                                                                    Training runs on 64 node GB200 cluster with TorchTitan Llama4 Scout demonstrated a 1.2x e2e training speedup with equivalent convergence to bfloat16 training baseline. In fact, after 3,000 steps it finishes with slightly lower loss than bfloat16! This is consistent with our scaling experiments with MXFP8 training for dense models.
                                                                    \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
                                                                    Number of GPUsBF16 tokens/secMXFP8 tokens/secMXFP8 speedup vs BF16
                                                                    512616974011.20x
                                                                    \n
                                                                    See the TorchAO MXFP8 MoE training documentation for more details. You can also check out the TorchTitan MXFP8 documentation to run pretraining jobs with TorchAO MXFP8 by adding a single config.
                                                                    \n
                                                                    Safetensors Enablement
                                                                    \n
                                                                    You can now save and load TorchAO model checkpoints using safetensors! This feature is integrated with Hugging Face transformers starting from v5.0.0 and vLLM 0.13.0 for model inference/serving.
                                                                    \n
                                                                    We currently support the following stable configs:\nFloat8DynamicActivationFloat8WeightConfig\nInt4WeightOnlyConfig\nIntxWeightOnlyConfig\nInt8DynamicActivationIntxWeightConfig\nInt8WeightOnlyConfig\nInt8DynamicActivationInt8WeightConfig
                                                                    \n
                                                                    and will continue to add support for configs as they become stable in the future.
                                                                    \n
                                                                    Example:
                                                                    \n
                                                                    import torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TorchAoConfig\nfrom torchao.quantization import Float8WeightOnlyConfig\n
                                                                    model_id = "facebook/opt-125m"\nquant_config = Float8WeightOnlyConfig()\nquantization_config = TorchAoConfig(quant_type=quant_config)\nquantized_model = AutoModelForCausalLM.from_pretrained(\nmodel_id,\n</tr></table>\n
                                                                    \n
                                                                    \n
                                                                    ... (truncated)
                                                                    \n
                                                                    \n
                                                                    \nCommits\n
                                                                      \n
                                                                    • 9338966 use python version agnostic binding for mxfp8 cuda kernels (#3471)
                                                                      • \n
                                                                    • acc9103 Fix NVFP4 QAT backward typo (#3478)
                                                                      • \n
                                                                    • 286c2d8 Fix NVFP4 QAT convert path (#3450", "merged_at": "2025-12-30T19:13:47Z"}
{"number": 2136, "title": "Update pillow requirement from <11.4,>=11.3 to >=11.3,<12.1", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [pillow](https://github.com/python-pillow/Pillow) to permit the latest version.\n
                                                                      \nRelease notes\n
                                                                      Sourced from pillow's releases.
                                                                      \n
                                                                      \n
                                                                      12.0.0
                                                                      \n
                                                                      https://pillow.readthedocs.io/en/stable/releasenotes/12.0.0.html
                                                                      \n
                                                                      Removals
                                                                      \n\n
                                                                      Deprecations
                                                                      \n\n
                                                                      Documentation
                                                                      \n
                                                                        \n
                                                                      • ImagingHistogramInstance can use two bands #9251 [@\u200bradarhere]
                                                                        • \n
                                                                      • Update 12.0.0 release notes #9247 [@\u200bhugovk]
                                                                        • \n
                                                                      • Added ImageDraw alpha channel examples #9201 [@\u200bradarhere]
                                                                        • \n
                                                                      • Update Python version #9230 [@\u200bradarhere]
                                                                        • \n
                                                                      • Updated macOS tested Pillow versions #9209 [@\u200bradarhere]
                                                                        • \n
                                                                      • Add GitHub profile link to release notes #9197 [@\u200bradarhere]
                                                                        • \n
                                                                      • Split versionadded info #9190 [@\u200bradarhere]
                                                                        • \n
                                                                      • Document ImageFile.MAXBLOCK #9163 [@\u200bradarhere]
                                                                        • \n
                                                                      • Updated macOS version in CI targets #9157 [@\u200bradarhere]
                                                                        • \n
                                                                      • Fix typos #9135 [@\u200bradarhere]
                                                                        • \n
                                                                      • Added "Colors" to concepts #9067 [@\u200bradarhere]
                                                                        • \n
                                                                      • Update macOS tested Pillow versions #9068 [@\u200bradarhere]
                                                                        • \n
                                                                      • Thanks, folks! #9056 [@\u200baclark4life]
                                                                        • \n
                                                                      • Setup nit: "f", "merged_at": "2025-12-30T17:32:01Z"}
{"number": 2135, "title": "Update typer requirement from <0.20.1 to <0.21.1", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [typer](https://github.com/fastapi/typer) to permit the latest version.\n
                                                                        \nRelease notes\n
                                                                        Sourced from typer's releases.
                                                                        \n
                                                                        \n
                                                                        0.21.0
                                                                        \n
                                                                        Breaking Changes
                                                                        \n\n
                                                                        Docs
                                                                        \n\n
                                                                        Internal
                                                                        \n\n
                                                                        \n
                                                                        \n
                                                                        \nChangelog\n
                                                                        Sourced from typer's changelog.
                                                                        \n
                                                                        \n
                                                                        0.21.0
                                                                        \n
                                                                        Breaking Changes
                                                                        \n\n
                                                                        Docs
                                                                        \n\n
                                                                        Internal
                                                                        \n
                                                                          \n
                                                                        • \ud83d\udc9a Move ruff dependency to shared requirements-docs-tests.txt to fix "Build docs" workflow in CI. PR #1458 by @\u200bYuriiMotov.
                                                                          • \n
                                                                        • \u2b06 Bump markdown-include-variants from 0.0.5 to 0.0.8. PR #1442 by @\u200bdependabot[bot].
                                                                          • \n
                                                                        • \ud83d\udc77 Add pre-commit workflow. PR #1453 by @\u200btiangolo.<", "merged_at": "2025-12-30T17:32:17Z"}
{"number": 2134, "title": "[docs] add docs build step", "files": [".github/workflows/doctests.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add docs build step to catch sphinx errors\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-27T01:50:11Z"}
{"number": 2133, "title": "[docs] Add custom theme", "files": ["docs/_static/custom.css", "docs/_static/custom.js", "docs/_templates/sidebar-logo-version.html", "docs/conf.py", "oumi_style.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\n- Add custom theme matching oumi theme\r\n- Move version selector dropdown under the logo\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-27T01:07:50Z"}
{"number": 2132, "title": "[cli] minor cli updates", "files": ["src/oumi/cli/cli_utils.py", "src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Improve footer / help section with examples\r\n- Add error message for missing config path \r\n- Cleanup styling\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-26T18:49:39Z"}
{"number": 2131, "title": "[docs] misc updates & cleanup", "files": ["docs/about/changelog.md", "docs/get_started/core_concepts.md", "docs/index.md", "docs/resources/models/models.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\n- Remove changelog page as it's not maintained. All changes are in the github release page\r\n- Refresh core concepts page with recent API changes and additions\r\n- Update index page with tiles \r\n- Fix import errors in samples\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-26T17:51:01Z"}
{"number": 2130, "title": "[docs] update CLI command tables with complete command list", "files": ["docs/get_started/core_concepts.md", "docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] update CLI command tables with complete command list\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T19:19:39Z"}
{"number": 2129, "title": "[tiny] minor judge command UX improvements", "files": ["src/oumi/cli/judge.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add `-c` alias for `--config` for consistency with other commands\r\n- Add examples for docs\r\n- Use console logs for more consistency\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T18:51:34Z"}
{"number": 2128, "title": "[tiny] fix typo", "files": ["src/oumi/cli/cli_utils.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n- fix typo in user facing message\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T18:31:19Z"}
{"number": 2127, "title": "deprecate protobuf conversation definitions", "files": [".pre-commit-config.yaml", "src/oumi/core/types/conversation.py", "src/oumi/core/types/proto/conversation.proto", "src/oumi/core/types/proto/generated/__init__.py", "src/oumi/core/types/proto/generated/conversation_pb2.py", "src/oumi/core/types/proto/generated/conversation_pb2.pyi", "tests/unit/core/types/test_conversation.py", "tests/unit/test_apache_license_header.py"], "area": "data", "area_votes": {"infra": 1, "data": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T18:11:31Z"}
{"number": 2126, "title": "[docs] add analyze, tune, and quantize CLI reference sections", "files": ["docs/cli/commands.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T18:22:35Z"}
{"number": 2125, "title": "[docs] fix inference recipe paths to match directory structure", "files": ["docs/resources/recipes.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-24T01:43:03Z"}
{"number": 2124, "title": "add skypilot k8s dependency", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards LOU-208\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T04:45:22Z"}
{"number": 2123, "title": "[tiny] update deprecated torch_dtype", "files": ["src/oumi/builders/models.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- torch_dtype is marked as deprecated by `transformers`. This PR updates to `dtype`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T04:45:21Z"}
{"number": 2122, "title": "Update cli: list aliases, auto-complete, help, common args", "files": ["src/oumi/cli/alias.py", "src/oumi/cli/analyze.py", "src/oumi/cli/cli_utils.py", "src/oumi/cli/completions.py", "src/oumi/cli/evaluate.py", "src/oumi/cli/infer.py", "src/oumi/cli/judge.py", "src/oumi/cli/main.py", "src/oumi/cli/quantize.py", "src/oumi/cli/synth.py", "src/oumi/cli/train.py", "src/oumi/cli/tune.py", "tests/unit/cli/test_cli_completions.py", "tests/unit/cli/test_cli_synth.py"], "area": "cli", "area_votes": {"cli": 12}, "body": "# Description\r\n\r\n\r\n\r\n- Add a `--list` option to all commands, showing available aliases\r\n- Add missing verb aliases (e.g. synth)\r\n- Add auto-complete with fuzzy matching for recipes\r\n- Add common changed parameters to CLI (merged into the config)\r\n- Add misc help and tooltips\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T18:16:27Z"}
{"number": 2120, "title": "Add ty config", "files": ["Makefile", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Add config to use ty for type checking, matching pyright as much as possible\r\n- Not enabled by default yet, users can run with `make ty`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-23T04:45:38Z"}
{"number": 2118, "title": "[bug] Pin typer version causing build docs failures", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Pin typer version causing build docs failures\r\n- More details here: https://github.com/sphinx-contrib/typer/issues/54\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1806\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-22T18:37:33Z"}
{"number": 2117, "title": "Update click requirement from <8.3.0 to <8.4.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [click](https://github.com/pallets/click) to permit the latest version.\n
                                                                          \nRelease notes\n
                                                                          Sourced from click's releases.
                                                                          \n
                                                                          \n
                                                                          8.3.1
                                                                          \n
                                                                          This is the Click 8.3.1 fix release, which fixes bugs but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.
                                                                          \n
                                                                          PyPI: https://pypi.org/project/click/8.3.1/\nChanges: https://click.palletsprojects.com/page/changes/#version-8-3-1\nMilestone: https://github.com/pallets/click/milestone/28
                                                                          \n
                                                                            \n
                                                                          • Don't discard pager arguments by correctly using subprocess.Popen. #3039 #3055
                                                                            • \n
                                                                          • Replace Sentinel.UNSET default values by None as they're passed through the Context.invoke() method. #3066 #3065 #3068
                                                                            • \n
                                                                          • Fix conversion of Sentinel.UNSET happening too early, which caused incorrect behavior for multiple parameters using the same name. #3071 #3079
                                                                            • \n
                                                                          • Fix rendering when prompt and confirm parameter prompt_suffix is empty. #3019 #3021
                                                                            • \n
                                                                          • When Sentinel.UNSET is found during parsing, it will skip calls to type_cast_value. #3069 #3090
                                                                            • \n
                                                                          • Hide Sentinel.UNSET values as None when looking up for other parameters through the context inside parameter callbacks. #3136 #3137
                                                                            • \n
                                                                          \n
                                                                          \n
                                                                          \n
                                                                          \nChangelog\n
                                                                          Sourced from click's changelog.
                                                                          \n
                                                                          \n
                                                                          Version 8.3.1
                                                                          \n
                                                                          Released 2025-11-15
                                                                          \n
                                                                            \n
                                                                          • Don't discard pager arguments by correctly using subprocess.Popen. :issue:3039\n:pr:3055
                                                                            • \n
                                                                          • Replace Sentinel.UNSET default values by None as they're passed through\nthe Context.invoke() method. :issue:3066 :issue:3065 :pr:3068
                                                                            • \n
                                                                          • Fix conversion of Sentinel.UNSET happening too early, which caused incorrect\nbehavior for multiple parameters using the same name. :issue:3071 :pr:3079
                                                                            • \n
                                                                          • Hide Sentinel.UNSET values as None when looking up for other parameters\nthrough the context inside parameter callbacks. :issue:3136 :pr:3137
                                                                            • \n
                                                                          • Fix rendering when prompt and confirm parameter prompt_suffix is\nempty. :issue:3019 :pr:3021
                                                                            • \n
                                                                          • When Sentinel.UNSET is found during parsing, it will skip calls to\ntype_cast_value. :issue:3069 :pr:3090
                                                                            • \n
                                                                          \n
                                                                          Version 8.3.0
                                                                          \n
                                                                          Released 2025-09-17
                                                                          \n
                                                                            \n
                                                                          • \n
                                                                            I", "merged_at": "2025-12-22T16:32:49Z"}
{"number": 2116, "title": "Update bitsandbytes requirement from <0.49,>=0.47 to >=0.47,<0.50", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [bitsandbytes](https://github.com/bitsandbytes-foundation/bitsandbytes) to permit the latest version.\n
                                                                            \nRelease notes\n
                                                                            Sourced from bitsandbytes's releases.
                                                                            \n
                                                                            \n
                                                                            0.49.0
                                                                            \n
                                                                            Highlights
                                                                            \n
                                                                            x86-64 CPU Improvements
                                                                            \n
                                                                            CPU performance for 4bit is significantly improved on x86-64, with optimized kernel paths for CPUs that have AVX512 or AVX512BF16 support.
                                                                            \n
                                                                            AMD ROCm Experimental Wheels
                                                                            \n
                                                                              \n
                                                                            • Experimental support for AMD devices is now included in our PyPI wheels on Linux x86-64.
                                                                              • \n
                                                                            • We've added additional GPU target devices as outlined in our docs.
                                                                              • \n
                                                                            • Support for using the default blocksize of 64 for 4bit was added for RDNA GPUs in #1748.
                                                                              • \n
                                                                            \n
                                                                            macOS 14+ Wheels
                                                                            \n
                                                                              \n
                                                                            • We're now publishing wheels for macOS 14+!
                                                                              • \n
                                                                            • The 4bit and 8bit quantization features are supported on MPS by slow implementations. We plan to enable Metal kernels with improved performance in the future.
                                                                              • \n
                                                                            \n
                                                                            \ud83d\udea8 Breaking Changes
                                                                            \n
                                                                              \n
                                                                            • Dropped support for Python 3.9.
                                                                              • \n
                                                                            • Dropped compilation support for Maxwell GPUs in the CUDA backend.
                                                                              • \n
                                                                            \n
                                                                            What's Changed
                                                                            \n
                                                                            \n
                                                                            \nChangelog\n
                                                                            Sourced from uvicorn's changelog.
                                                                            \n
                                                                            \n
                                                                            0.40.0 (December 21, 2025)
                                                                            \n
                                                                            Remove
                                                                            \n
                                                                              \n
                                                                            • Drop support for Python 3.9 (#2772)
                                                                              • \n
                                                                            \n
                                                                            0.39.0 (December 21, 2025)
                                                                            \n
                                                                            Fixed
                                                                            \n
                                                                              \n
                                                                            • Send close frame on ASGI return for WebSockets (#2769)
                                                                              • \n
                                                                            • Explicitly start ASGI run with empty context (#2742)
                                                                              • \n
                                                                            \n
                                                                            0.38.0 (October 18, 2025)
                                                                            \n
                                                                            Added
                                                                            \n
                                                                              \n
                                                                            • Support Python 3.14 (#2723)
                                                                              • \n
                                                                            \n
                                                                            0.37.0 (September 23, 2025)
                                                                            \n
                                                                            Added
                                                                            \n
                                                                              \n
                                                                            • Add --timeout-worker-healthcheck option (#2711)
                                                                              • \n
                                                                            • Add os.PathLike[str] type to ssl_ca_certs (#2676)
                                                                              • \n
                                                                            \n
                                                                            0.36.1 (September 23, 2025)
                                                                            \n
                                                                            Fixed
                                                                            \n
                                                                              \n
                                                                            • Raise an exception when calling removed Config.setup_event_loop() (#2709)
                                                                              • \n
                                                                            \n
                                                                            0.36.0 (September 20, 2025)
                                                                            \n
                                                                            Added
                                                                            \n
                                                                              \n
                                                                            • Support custom IOLOOPs (#2435)
                                                                              • \n
                                                                            • Allow to provide importable string in --http, --ws and --loop (#2658)
                                                                              • \n
                                                                            \n
                                                                            0.35.0 (June 28, 2025)
                                                                            \n
                                                                            Added
                                                                            \n
                                                                              \n
                                                                            • Add WebSocketsSansIOProtocol (#2540)
                                                                              • \n
                                                                            \n
                                                                            Changed
                                                                            \n
                                                                              \n
                                                                            • Refine help message for option --proxy-headers (#2653)
                                                                              • \n
                                                                            \n
                                                                            0.34.3 (June 1, 2025)
                                                                            \n\n
                                                                            \n
                                                                            ... (truncated)
                                                                            \n
                                                                            \n
                                                                            \nCommits\n
                                                                              \n
                                                                            • 9ff6004 Version 0.40.0 (#2773)
                                                                              • \n
                                                                            • 19df042 Drop Python 3.9 (#2772)
                                                                              • \n
                                                                            • 865ce7c Run strict mypy on test suite (#2771)
                                                                              • \n
                                                                            • 4f40b84 Version 0.39.0 (#2770)
                                                                              • \n
                                                                            • 5692dfc=0.17 to >=0.17,<0.19", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [peft](https://github.com/huggingface/peft) to permit the latest version.\n
                                                                              \nRelease notes\n
                                                                              Sourced from peft's releases.
                                                                              \n
                                                                              \n
                                                                              0.18.0: RoAd, ALoRA, Arrow, WaveFT, DeLoRA, OSF, and more
                                                                              \n
                                                                              Highlights
                                                                              \n\n
                                                                              FIXME update list of all changes, so some more commits were added
                                                                              \n
                                                                              New Methods
                                                                              \n
                                                                              RoAd
                                                                              \n
                                                                              @\u200bppetrushkov added RoAd: 2D Rotary Adaptation to PEFT in #2678. RoAd learns 2D rotation matrices that are applied using only element-wise multiplication, thus promising very fast inference with adapters in unmerged state.
                                                                              \n
                                                                              Remarkably, besides LoRA, RoAd is the only PEFT method that supports mixed adapter batches. This means that when you have loaded a model with multiple RoAd adapters, you can use all of them for different samples in the same batch, which is much more efficient than switching adapters between batches:
                                                                              \n
                                                                              model = PeftModel.from_pretrained(base_model, <path-to-road-adapter-A>, adapter_name="adapter-A")\nmodel.add_adapter("adapter-B", <path-to-road-adapter-B>)\n
                                                                              inputs = ...  # input with 3 samples
                                                                              \n
                                                                              apply adapter A to sample 0, adapter B to sample 1, and use the base model for sample 2:
                                                                              \n
                                                                              adapter_names = ["adapter-A", "adapter-B", "base"]\noutput_mixed = model(**inputs, adapter_names=adapter_names)\ngen_mixed = model.generate(**inputs, adapter_names=adapter_names)\n
                                                                              \n
                                                                              ALoRA
                                                                              \n
                                                                              Activated LoRA is a technique added by @\u200bkgreenewald in #2609 for causal language models, allowing to selectively enable LoRA adapters depending on a specific token invocation sequence in the input. This has the major benefit of being able to re-use most of the KV cache during inference when the adapter is only used to generate part of the response, after which the base model takes over again.
                                                                              \n
                                                                              Arrow & GenKnowSub
                                                                              \n
                                                                              @\u200bTheTahaaa contributed not only support for Arrow, a dynamic routing algorithm between multiple loaded LoRAs in #2644, but also GenKnowSub, a technique built upon Arrow where the 'library' of LoRAs available to Arrow is first modified by subtracting general knowledge adapters (e.g., trained on subsets of Wikipedia) to enhance task-specific performance.
                                                                              \n
                                                                              WaveFT
                                                                              \n
                                                                              Thanks to @\u200bBilican, Wavelet Fine-Tuning (WaveFT) was added to PEFT in #2560. This method trains sparse updates in the wavelet domain of residual matrices, which is especially parameter efficient. It is very interesting for image generation, as it promises to generate diverse outputs while preserving subject fidelity.
                                                                              \n
                                                                              DeLoRA
                                                                              \n
                                                                              Decoupled Low-rank Adaptation (DeLoRA) was added by @\u200bmwbini in #2780. This new PEFT method is similar to DoRA in so far as it decouples the angle and magnitude of the learned adapter weights. However, DeLoRA implements this in a way that promises to better prevent divergence. Moreover, it constrains the dev", "merged_at": "2025-12-22T02:30:40Z"}
{"number": 2112, "title": "Bump actions/cache from 4 to 5 in the all-actions group", "files": [".github/workflows/gpu_install_test.yaml", ".github/workflows/install_test.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "Bumps the all-actions group with 1 update: [actions/cache](https://github.com/actions/cache).\n\nUpdates `actions/cache` from 4 to 5\n
                                                                              \nRelease notes\n
                                                                              Sourced from actions/cache's releases.
                                                                              \n
                                                                              \n
                                                                              v5.0.0
                                                                              \n
                                                                              \n
                                                                              [!IMPORTANT]\nactions/cache@v5 runs on the Node.js 24 runtime and requires a minimum Actions Runner version of 2.327.1.
                                                                              \n
                                                                              If you are using self-hosted runners, ensure they are updated before upgrading.
                                                                              \n
                                                                              \n
                                                                              \n
                                                                              What's Changed
                                                                              \n\n
                                                                              Full Changelog: https://github.com/actions/cache/compare/v4.3.0...v5.0.0
                                                                              \n
                                                                              v4.3.0
                                                                              \n
                                                                              What's Changed
                                                                              \n\n
                                                                              New Contributors
                                                                              \n\n
                                                                              Full Changelog: https://github.com/actions/cache/compare/v4...v4.3.0
                                                                              \n
                                                                              v4.2.4
                                                                              \n
                                                                              What's Changed
                                                                              \n\n
                                                                              New Contributors
                                                                              \n\n
                                                                              Full Changelog: https://github.com/actions/cache/compare/v4...v4.2.4
                                                                              \n
                                                                              v4.2.3
                                                                              \n
                                                                              What's Changed
                                                                              \n
                                                                                \n
                                                                              • Update to use @\u200bactions/cache 4.0.3 package & prepare for new release by @\u200bsalmanmkc in actions/cache#1577 (SAS tokens for cache entries are now masked in debug logs)
                                                                                • \n
                                                                              \n
                                                                              New Contributors
                                                                              \n\n
                                                                              Full Changelog: https://github.com/actions/cache/compare/v4.2.2...v4.2.3
                                                                              \n\n
                                                                              \n
                                                                              ... (truncated)
                                                                              \n
                                                                              \n
                                                                              \nChangelog\n
                                                                              Sourc", "merged_at": "2025-12-22T01:30:27Z"}
{"number": 2111, "title": "Dec news updates", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "December updates:\r\n\r\n* Oumi v0.6.0 release\r\n* Hackathon webinar\r\n* Hackathon per se", "merged_at": "2025-12-20T04:11:13Z"}
{"number": 2110, "title": "[perf] Add lazy imports in the cli", "files": ["src/oumi/cli/analyze.py", "src/oumi/cli/cache.py", "src/oumi/cli/cli_utils.py", "src/oumi/utils/hf_cache_utils.py", "tests/unit/cli/test_cli_cache.py", "tests/unit/cli/test_cli_utils.py"], "area": "cli", "area_votes": {"cli": 3, "other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add lazy imports in the cli code paths\r\n- Benchmarks:\r\n\r\n| Module | Before | After | Improvement |\r\n|--------|--------|-------|-------------|\r\n| CLI startup | 320-340ms | 56-58ms | **83%** |\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-19T23:39:30Z"}
{"number": 2109, "title": "[deps] upgrade pytorch 2.9 & python 3.14", "files": [".github/workflows/install_test.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Enable python 3.14 and pytorch 2.9\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-19T19:05:38Z"}
{"number": 2105, "title": "edits to docs guide", "files": ["docs/development/docs_guide.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "\r\nReorganized the to-do list for submitting changes to the docs. Moved, \"to update the documentation\" to the beginning of the sentence to emphasize the action that will be completed. Now \"make a PR\" is a checklist item.\r\n\r\n+ small type editing.\r\n", "merged_at": "2025-12-18T15:40:47Z"}
{"number": 2104, "title": "Fix issue with synthesis rounding errors", "files": ["src/oumi/core/configs/params/synthesis_params.py", "tests/unit/core/configs/params/test_synthesis_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\n\n\n\nDue to rounding errors, many configs fail to parse because they don't sum up to exactly 1, so this adds some tolerance.\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-12-18T19:07:13Z"}
{"number": 2102, "title": "[configs] create folders for infer", "files": ["configs/recipes/deepseek_r1/inference/671b_together/infer.yaml", "configs/recipes/deepseek_r1/inference/distill_llama_70b/infer.yaml", "configs/recipes/deepseek_r1/inference/distill_llama_8b/infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_1_5b/infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b/gguf_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b/gguf_macos_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b/infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b/vllm_infer.yaml", "src/oumi/cli/alias.py"], "area": "configs", "area_votes": {"configs": 8, "cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-17T21:22:12Z"}
{"number": 2100, "title": "[configs] Add train/infer/eval configs", "files": ["configs/recipes/gemma3/evaluation/12b/eval.yaml", "configs/recipes/gemma3/evaluation/27b/eval.yaml", "configs/recipes/gemma3/evaluation/4b/eval.yaml", "configs/recipes/gemma3/inference/12b_instruct_infer.yaml", "configs/recipes/gemma3/inference/27b_instruct_infer.yaml", "configs/recipes/gemma3/inference/4b_instruct_infer.yaml", "configs/recipes/gemma3/sft/12b_lora/train.yaml", "configs/recipes/gemma3/sft/27b_lora/train.yaml", "configs/recipes/gemma3/sft/4b_full/train.yaml", "configs/recipes/olmo3/evaluation/32b/eval.yaml", "configs/recipes/olmo3/evaluation/7b/eval.yaml", "configs/recipes/olmo3/inference/32b_infer.yaml", "configs/recipes/olmo3/inference/7b_infer.yaml", "configs/recipes/olmo3/sft/32b_lora/train.yaml", "configs/recipes/olmo3/sft/7b_full/train.yaml", "configs/recipes/qwen3_next/evaluation/80b_a3b_eval.yaml", "configs/recipes/qwen3_next/inference/80b_a3b_infer.yaml", "configs/recipes/qwen3_next/inference/80b_a3b_instruct_infer.yaml", "configs/recipes/qwen3_next/sft/80b_a3b_lora/train.yaml", "configs/recipes/vision/qwen3_vl_2b/inference/infer.yaml", "configs/recipes/vision/qwen3_vl_4b/inference/infer.yaml", "configs/recipes/vision/qwen3_vl_8b/inference/infer.yaml"], "area": "configs", "area_votes": {"configs": 22}, "body": "# Description\r\n\r\n\r\n\r\n- Add example train/eval/infer configs for recent models \r\n- Tested on 8xH100\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-17T20:37:30Z"}
{"number": 2099, "title": "Make top_p optional for API models", "files": ["src/oumi/core/configs/params/generation_params.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/bedrock_inference_engine.py", "src/oumi/inference/gcp_inference_engine.py", "src/oumi/inference/gemini_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/remote_vllm_inference_engine.py", "src/oumi/inference/sambanova_inference_engine.py", "src/oumi/inference/sglang_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/unit/inference/test_generation_params.py"], "area": "inference", "area_votes": {"configs": 1, "inference": 10}, "body": "# Description\n\n\n\nSonnet 4.5 requires that either temperature or top_p be set, but NOT BOTH.\n\nUpdating our generation params to allow us to support this type of definition more broadly across API models (which often have their own default top_p which we wouldn't want to overwrite anyway).\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-12-17T17:50:07Z"}
{"number": 2097, "title": "[deps] upgrade trl to 0.26", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Upgrade trl version\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-16T21:02:34Z"}
{"number": 2092, "title": "Enable python 3.13 support & add github actions cache", "files": [".github/workflows/gpu_install_test.yaml", ".github/workflows/install_test.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 3}, "body": "# Description\r\n\r\n\r\n\r\n- Enable support for python 3.13\r\n- Add github actions cache for hf resources\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-16T17:44:14Z"}
{"number": 2090, "title": "Update nvidia-ml-py requirement from <13.581,>=13.580 to >=13.580,<13.591", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [nvidia-ml-py](https://forums.developer.nvidia.com) to permit the latest version.\n\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
                                                                              \nDependabot commands and options\n
                                                                              \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot merge` will merge this PR after your CI passes on it\n- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it\n- `@dependabot cancel merge` will cancel a previously requested merge and block automerging\n- `@dependabot reopen` will reopen this PR if it is closed\n- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually\n- `@dependabot show  ignore conditions` will show all of the ignore conditions of the specified dependency\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n\n\n
                                                                              ", "merged_at": "2025-12-16T06:04:33Z"}
{"number": 2089, "title": "Update skypilot requirement from <0.11,>=0.10.2 to >=0.10.2,<0.12", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "[//]: # (dependabot-start)\n\u26a0\ufe0f  **Dependabot is rebasing this PR** \u26a0\ufe0f \n\nRebasing might not happen immediately, so don't worry if this takes some time.\n\nNote: if you make any changes to this PR yourself, they will take precedence over the rebase.\n\n---\n\n[//]: # (dependabot-end)\n\nUpdates the requirements on [skypilot](https://github.com/skypilot-org/skypilot) to permit the latest version.\n
                                                                              \nRelease notes\n
                                                                              Sourced from skypilot's releases.
                                                                              \n
                                                                              \n
                                                                              SkyPilot v0.11.0: Multi-Cloud Pools, Fast Managed Jobs, Enterprise-Readiness at Large Scale, Programmability
                                                                              \n
                                                                              SkyPilot v0.11.0 delivers major new features: Pools and Managed Jobs Consolidation Mode; significant improvement enterprise-readiness at large scale: supports hundreds of AI engineers with a single API server instance, avoid OOM, >10x performance improvement on many requests, additional observability, and more; and UX improvements: Templates, Python SDK, CI/CD, Git Support, and more.
                                                                              \n
                                                                              Get it now with:
                                                                              \n
                                                                              uv pip install "skypilot>=0.11.0"\n
                                                                              \n
                                                                              Or, upgrade your team SkyPilot API server:
                                                                              \n
                                                                              NAMESPACE=skypilot\nRELEASE_NAME=skypilot\nVERSION=0.11.0\n
                                                                              helm repo update skypilot\nhelm upgrade -n $NAMESPACE $RELEASE_NAME skypilot/skypilot 
                                                                              \n--set apiService.image=berkeleyskypilot/skypilot:$VERSION 
                                                                              \n--version $VERSION --devel --reuse-values\n
                                                                              \n
                                                                              Highlights
                                                                              \n
                                                                              [Beta] SkyPilot Pools: Batch inference across clouds & k8s
                                                                              \n
                                                                              SkyPilot supports spawning a pool that launches a set of workers across many clouds and Kubernetes clusters (docs, #6260, #6426, #6459, #6552, #6591, #6665, #6675, #7332, #7963, #8008, #7876, #8047, #7930,#8039, #7846, #7855, #7919, #7920). Jobs can be scheduled on this pool and distributed to workers as they become available.
                                                                              \n
                                                                              Key benefits include:
                                                                              \n
                                                                                \n
                                                                              • Fully utilize your GPU capacity across clouds & k8s
                                                                                • \n
                                                                              • Unified queue for jobs on all infra
                                                                                • \n
                                                                              • Keep workers warm, scale elastically
                                                                                • \n
                                                                              • Step aside for other higher priority jobs; reschedule when GPUs become available
                                                                                • \n
                                                                              \n
                                                                              Learn more in our blog post.
                                                                              \n\nRelease notes\n
                                                                              Sourced from actions/cache's releases.
                                                                              \n
                                                                              \n
                                                                              v5.0.0
                                                                              \n
                                                                              \n
                                                                              [!IMPORTANT]\nactions/cache@v5 runs on the Node.js 24 runtime and requires a minimum Actions Runner version of 2.327.1.
                                                                              \n
                                                                              If you are using self-hosted runners, ensure they are updated before upgrading.
                                                                              \n
                                                                              \n
                                                                              \n
                                                                              What's Changed
                                                                              \n\n
                                                                              Full Changelog: https://github.com/actions/cache/compare/v4.3.0...v5.0.0
                                                                              \n
                                                                              v4.3.0
                                                                              \n
                                                                              What's Changed
                                                                              \n\n
                                                                              New Contributors
                                                                              \n\n
                                                                              Full Changelog: https://github.com/actions/cache/compare/v4...v4.3.0
                                                                              \n
                                                                              v4.2.4
                                                                              \n
                                                                              What's Changed
                                                                              \n\n
                                                                              New Contributors
                                                                              \n\n
                                                                              Full Changelog: https://github.com/actions/cache/compare/v4...v4.2.4
                                                                              \n
                                                                              v4.2.3
                                                                              \n
                                                                              What's Changed
                                                                              \n
                                                                                \n
                                                                              • Update to use @\u200bactions/cache 4.0.3 package & prepare for new release by @\u200bsalmanmkc in actions/cache#1577 (SAS tokens for cache entries are now masked in debug logs)
                                                                                • \n
                                                                              \n
                                                                              New Contributors
                                                                              \n\n
                                                                              Full Changelog: https://github.com/actions/cache/compare/v4.2.2...v4.2.3
                                                                              \n\n
                                                                              \n
                                                                              ... (truncated)
                                                                              \n
                                                                              \n
                                                                              \nChangelog\n
                                                                              Sourc", "merged_at": "2025-12-15T19:19:35Z"}
{"number": 2087, "title": "feat(judges): add code evaluation judges for software quality assessment", "files": ["configs/projects/judges/code/code_quality.yaml", "configs/projects/judges/code/correctness.yaml", "configs/projects/judges/code/maintainability.yaml", "configs/projects/judges/code/performance.yaml", "configs/projects/judges/code/security.yaml"], "area": "configs", "area_votes": {"configs": 5}, "body": "Add 5 new judge configurations for evaluating code quality:\r\n- code_quality.yaml: Readability, structure, documentation, DRY principle\r\n- security.yaml: Injection prevention, auth, data protection, crypto\r\n- correctness.yaml: Functional correctness, edge cases, logic accuracy\r\n- performance.yaml: Time/space complexity, I/O efficiency, caching\r\n- maintainability.yaml: Modularity, testability, extensibility\r\n\r\nThese judges enable automated evaluation of LLM-generated code outputs, useful for code generation, refactoring, and code review tasks.\r\n\r\nContributed as part of AssembleHack25 hackathon submission.", "merged_at": "2025-12-14T17:27:54Z"}
{"number": 2084, "title": "pin pycares", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nhttps://pypi.org/project/pycares/#history recent bump has caused errors on our dataset \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards LOU-331\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-11T20:16:02Z"}
{"number": 2083, "title": "Remove references to sphinx_rtd_theme", "files": ["docs/conf.py"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nRemove a module as its import was removed in #2075\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-08T21:15:21Z"}
{"number": 2081, "title": "Update pydantic requirement from <2.12,>=2.11 to >=2.11,<2.13", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [pydantic](https://github.com/pydantic/pydantic) to permit the latest version.\n
                                                                              \nRelease notes\n
                                                                              Sourced from pydantic's releases.
                                                                              \n
                                                                              \n
                                                                              v2.12.5 2025-11-26
                                                                              \n
                                                                              v2.12.5 (2025-11-26)
                                                                              \n
                                                                              This is the fifth 2.12 patch release, addressing an issue with the MISSING sentinel and providing several documentation improvements.
                                                                              \n
                                                                              The next 2.13 minor release will be published in a couple weeks, and will include a new polymorphic serialization feature addressing\nthe remaining unexpected changes to the serialize as any behavior.
                                                                              \n
                                                                                \n
                                                                              • Fix pickle error when using model_construct() on a model with MISSING as a default value by @\u200bornariece in #12522.
                                                                                • \n
                                                                              • Several updates to the documentation by @\u200bViicos.
                                                                                • \n
                                                                              \n
                                                                              Full Changelog: https://github.com/pydantic/pydantic/compare/v2.12.4...v2.12.5
                                                                              \n
                                                                              \n
                                                                              \n
                                                                              \nChangelog\n
                                                                              Sourced from pydantic's changelog.
                                                                              \n
                                                                              \n
                                                                              v2.12.5 (2025-11-26)
                                                                              \n
                                                                              GitHub release
                                                                              \n
                                                                              This is the fifth 2.12 patch release, addressing an issue with the MISSING sentinel and providing several documentation improvements.
                                                                              \n
                                                                              The next 2.13 minor release will be published in a couple weeks, and will include a new polymorphic serialization feature addressing\nthe remaining unexpected changes to the serialize as any behavior.
                                                                              \n
                                                                                \n
                                                                              • Fix pickle error when using model_construct() on a model with MISSING as a default value by @\u200bornariece in #12522.
                                                                                • \n
                                                                              • Several updates to the documentation by @\u200bViicos.
                                                                                • \n
                                                                              \n
                                                                              v2.12.4 (2025-11-05)
                                                                              \n
                                                                              GitHub release
                                                                              \n
                                                                              This is the fourth 2.12 patch release, fixing more regressions, and reverting a change in the build() method\nof the AnyUrl and Dsn types.
                                                                              \n
                                                                              This patch release also fixes an issue with the serialization of IP address types, when serialize_as_any is used. The next patch release\nwill try to address the remaining issues with serialize as any behavior by introducing a new polymorphic serialization feature, that\nshould be used in most cases in place of serialize as any.
                                                                              \n
                                                                                \n
                                                                              • \n
                                                                                Fix issue with forward references in parent TypedDict classes by @\u200bViicos in #12427.
                                                                                \n
                                                                                This issue is only relevant on Python 3.14 and greater.
                                                                                \n
                                                                                • \n
                                                                              • \n
                                                                                Exclude fields with exclude_if from JSON Schema required fields by @\u200bViicos in #12430
                                                                                \n
                                                                                • \n
                                                                              • \n
                                                                                Revert URL percent-encoding of credentials in the build() method\nof the AnyUrl and Dsn types by @\u200bdavidhewitt in\n=0.17.0 to >=0.17.0,<0.19.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [deepspeed](https://github.com/deepspeedai/DeepSpeed) to permit the latest version.\n
                                                                                \nRelease notes\n
                                                                                Sourced from deepspeed's releases.
                                                                                \n
                                                                                \n
                                                                                v0.18.2 Patch Release
                                                                                \n
                                                                                What's Changed
                                                                                \n\n
                                                                                New Contributors
                                                                                \n\n
                                                                                Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.18.1...v0.18.2
                                                                                \n
                                                                                \n
                                                                                \n
                                                                                \nCommits\n
                                                                                  \n
                                                                                • e993fea README refresh (#7668)
                                                                                  • \n
                                                                                • 4691beb Fix typo in pytorch-profiler.md documentation (#7652)
                                                                                  • \n
                                                                                • 6fe3c7a resolved a 0-dim tensor slicing bug from _get_state_without_padding (#7659)
                                                                                  • \n
                                                                                • =0.12 to >=0.12,<0.15", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [torchao](https://github.com/pytorch/ao) to permit the latest version.\n
                                                                                  \nRelease notes\n
                                                                                  Sourced from torchao's releases.
                                                                                  \n
                                                                                  \n
                                                                                  v0.14.1
                                                                                  \n
                                                                                  Highlights
                                                                                  \n
                                                                                  We are excited to announce the 0.14.1 release of torchao! This release adds support for MoE training on Backwell GPUs and NVFP4 QAT!
                                                                                  \n
                                                                                  (Prototype) MoE training on Blackwell GPUs
                                                                                  \n
                                                                                  We\u2019ve added a quantized building block for speeding up MoE training on Blackwell GPUs: torchao\u2019s `_scaled_grouped_mm`! It is a differentiable drop-in replacement for `torch._grouped_mm` that dynamically quantizes inputs using the given recipe, performs a scaled grouped GEMM, then returns the results in original precision. This results in significant speedups (see benchmarks below)!
                                                                                  \n
                                                                                  import torch\nfrom torch.nn import functional as F\nfrom torchao.prototype.moe_training import (\n    _scaled_grouped_mm as torchao_scaled_grouped_mm\n)\nfrom torchao.prototype.moe_training.conversion_utils import MoEScalingType\nfrom torchao.prototype.moe_training.utils import generate_jagged_offs\n
                                                                                  num_groups, total_M, N, K = 8, 131072, 8192, 5120
                                                                                  \n
                                                                                  A = input actvations, B = expert weights
                                                                                  \n
                                                                                  A = torch.randn(total_M, K, dtype=torch.bfloat16, device="cuda", requires_grad=True)\nB = torch.randn(num_groups, N, K, dtype=torch.bfloat16, device="cuda", requires_grad=True)
                                                                                  \n
                                                                                  Token group offsets computed by router in actual MoE layer
                                                                                  \n
                                                                                  offs = generate_jagged_offs(num_groups, total_M, device="cuda")
                                                                                  \n
                                                                                  Forward and backward example
                                                                                  \n
                                                                                  out = torchao_scaled_grouped_mm(\nA,\nB.transpose(-2, -1),\noffs=offs,\nscaling_type=MoEScalingType.MXFP8,\n)\nlabels = torch.ones_like(out)\nloss = F.mse_loss(out, labels)\nloss.backward()\n
                                                                                  \n
                                                                                  Microbenchmarks (see README for commands to reproduce benchmarks):
                                                                                  \n
                                                                                    \n
                                                                                  • Forward + backward pass vs torch._grouped_mm:\n
                                                                                      \n
                                                                                    • ~1.4-1.8x faster for Llama4 17bx16e shapes
                                                                                      • \n
                                                                                    • ~1.2-1.4x faster for DeepSeekV3 671b shapes
                                                                                      • \n
                                                                                    \n
                                                                                    • \n
                                                                                  • Full MoE layer forward + backward pass:\n
                                                                                      \n
                                                                                    • ~1.4x faster (Llama4 17bx16e shapes, batch_size=8, seq_len=16384)
                                                                                      • \n
                                                                                    • ~1.2x faster (DeepSeekV3 671b shapes, batch_size=8, seq_len=16384).
                                                                                      • \n
                                                                                    \n
                                                                                    • \n
                                                                                  \n
                                                                                  It\u2019s also already integrated into TorchTitan for E2E training with DeepSeekV3 and Llama4! Just use the command line flag: `--model.converters=\u201dquantize.grouped_mm.mx\u201d, which will convert all `torch._grouped_mm` ops to torchao _scaled_grouped_mm ops under the hood:
                                                                                  \n\n
                                                                                  \n
                                                                                  ... (truncated)
                                                                                  \n
                                                                                  \n
                                                                                  \nCommits\n
                                                                                    \n
                                                                                  • d94bb19 Update version to 0.14.1 (#3213)
                                                                                    • \n
                                                                                  • 1a578d1 Fix TORCHAO_SKIP_LOADING_SO_FILES behavior (#3189)
                                                                                    • \n
                                                                                  • 937e3a4 Fix SyntaxWarning during installation / import (#3184)
                                                                                    • \n
                                                                                  • 2dedd73 Update compatibility matrix (#3178)
                                                                                    • \n
                                                                                  • 0294124 Fix setuptools version for docs build (\r\n\r\n- Skip torchdata since it has known compatibility issues\r\n- Increase the number of concurrent PRs to 5\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-04T20:02:16Z"}
{"number": 2072, "title": "[cli] Add cli groups & update descriptions", "files": ["src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n**Changes**\r\n- Add cli typer groups to organize all the commands since we have quite a few now\r\n- Update / cleanup descriptions to make them more consistent\r\n\r\n**After**\r\n\"image\"\r\n\r\n**Before**\r\n\"image\"\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-04T05:56:46Z"}
{"number": 2071, "title": "[docs] add analyze documentation and example configs", "files": ["configs/examples/analyze/analyze.yaml", "docs/index.md", "docs/user_guides/analyze/analyze.md", "docs/user_guides/analyze/analyze_config.md", "tests/unit/cli/test_cli_analyze.py", "tests/unit/core/configs/test_parse_configs.py"], "area": "docs", "area_votes": {"configs": 1, "docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-04T05:56:46Z"}
{"number": 2069, "title": "Add analyze CLI command", "files": ["src/oumi/cli/alias.py", "src/oumi/cli/analyze.py", "src/oumi/cli/main.py", "src/oumi/core/analyze/dataset_analyzer.py", "tests/unit/cli/test_cli_analyze.py", "tests/unit/core/analyze/test_dataset_analyzer.py"], "area": "cli", "area_votes": {"cli": 3, "evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add analyze command to CLI for dataset analysis\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-02T22:40:16Z"}
{"number": 2068, "title": "adding k8s as cloud provider for skypilot", "files": ["src/oumi/launcher/clients/sky_client.py", "src/oumi/launcher/clouds/sky_cloud.py", "tests/unit/launcher/clients/test_sky_client.py", "tests/unit/launcher/clouds/test_sky_cloud.py"], "area": "launcher", "area_votes": {"launcher": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\nAdding support for k8s for oumi launcher\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #2067\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-02T04:29:48Z"}
{"number": 2063, "title": "update github actions triggers", "files": [".github/workflows/gpu_install_test.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/install_test.yaml", ".github/workflows/release_gcp.yaml"], "area": "infra", "area_votes": {"infra": 4}, "body": "# Description\r\n\r\n\r\n\r\n- Run GPU tests when integration tests are updated\r\n- Deprecate gcp release action\r\n- Avoid running install tests on all workflows\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-12-01T16:44:51Z"}
{"number": 2060, "title": "Fix too many files open error", "files": ["src/oumi/inference/remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\nCap TCPConnector connection limit at 200 to prevent file descriptor\r\nexhaustion when requests_per_minute is set to 500 or 1000. HTTP\r\nconnections are reused, so we don't need one connection per concurrent\r\nrequest. The semaphore still controls concurrency properly.\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards LOU-283\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-26T18:20:36Z"}
{"number": 2058, "title": "[docs] Add local docker guide", "files": ["docs/get_started/installation.md", "docs/user_guides/train/environments/docker.md", "docs/user_guides/train/environments/environments.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] Add local docker guide\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-26T01:16:00Z"}
{"number": 2057, "title": "Fix length analyzer column naming and add comprehensive message summary tests", "files": ["src/oumi/core/analyze/length_analyzer.py", "tests/unit/core/analyze/test_dataset_analyzer.py", "tests/unit/core/test_length_analyzer.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\n\nAfter the latest upgrade to analyze to use a schema, the summery generation is broken. The summary generation expected column names like text_content_length_analyzer_char_count (5+ parts), but analyzers create columns like text_content_char_count (4 parts), so the parsing never matched and returned empty summaries. We need to change the column names to include the analyzer's name. The change should happen in length_analzyer to generate the expected column names\n\n- Updated length analyzer to use analyzer_id in column names (text_content_length_char_count format)\n- Updated all length analyzer tests to use new column naming format\n- Added comprehensive test for message level summary structure and statistics\n- Verified summary generation works correctly with new column naming format\n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes LOU-277\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-11-25T20:03:10Z"}
{"number": 2056, "title": "Remove deprecated `target_col` field mentions", "files": ["configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/chatqa/chatqa_stage1_train.yaml", "configs/projects/chatqa/chatqa_stage2_train.yaml", "configs/projects/coalm/405b_train.yaml", "configs/projects/coalm/70b_train.yaml", "configs/projects/coalm/8b_train.yaml", "configs/projects/halloumi/8b_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_train.yaml", "configs/recipes/gpt2/pretraining/macos_train.yaml", "configs/recipes/gpt2/pretraining/train.yaml", "configs/recipes/gpt_oss/sft/20b_lora_single_gpu_train.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml", "configs/recipes/phi3/sft/lora_macos_train.yaml", "configs/recipes/phi3/sft/lora_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/full_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_train.yaml", "configs/recipes/qwen2_5/sft/3b_full/train.yaml", "configs/recipes/qwen2_5/sft/7b_full/train.yaml", "configs/recipes/qwen3/sft/0.6b_full/train.yaml", "configs/recipes/qwen3/sft/1.7b_full/train.yaml", "configs/recipes/qwen3/sft/14b_lora/train.yaml", "configs/recipes/qwen3/sft/30b_a3b_lora/train.yaml", "configs/recipes/qwen3/sft/32b_lora/train.yaml", "configs/recipes/qwen3/sft/4b_full/train.yaml", "configs/recipes/qwen3/sft/8b_full/train.yaml", "configs/recipes/qwq/sft/full_train.yaml", "configs/recipes/qwq/sft/lora_train.yaml", "configs/recipes/qwq/sft/qlora_train.yaml", "configs/recipes/smollm/sft/135m/quickstart_train.yaml", "configs/recipes/smollm/sft/135m/train.yaml", "configs/recipes/smollm/tuning/135m/tune.yaml", "docs/user_guides/train/configuration.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "scripts/datasets/pretokenize/process_dataset.py", "scripts/demo.py", "src/experimental/configs/projects/zephyr/sft/full_train.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_train.yaml", "src/oumi/core/configs/params/data_params.py", "tests/unit/builders/test_data_mixtures.py"], "area": "configs", "area_votes": {"configs": 64, "docs": 3, "infra": 2, "other": 2}, "body": "# Description\r\n\r\n- This field doesn't do anything, so removed all references to it. We can delete it in a future Oumi version.\r\n- We can also remove lines specifying `dataset_text_field: \"text\"` as this is the default value in HF TrainingArguments.\r\n- Added a deprecation warning for the field\r\n\r\n## Related issues\r\n\r\nTowards OPE-1781\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-11-25T17:44:26Z"}
{"number": 2054, "title": "add instructions to deploy Oumi on a k8s cluster", "files": ["docs/user_guides/launch/kubernetes.md", "docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nadd documentation for deploying Oumi on a k8s cluster\r\n\r\nincluding a quickstart of launching an Oumi pod and examples for AWS and GKE.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1337\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-25T23:15:38Z"}
{"number": 2051, "title": "Update aiofiles requirement from <25,>=24.1.0 to >=24.1.0,<26", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [aiofiles](https://github.com/Tinche/aiofiles) to permit the latest version.\n
                                                                                    \nRelease notes\n
                                                                                    Sourced from aiofiles's releases.
                                                                                    \n
                                                                                    \n
                                                                                    v25.1.0
                                                                                    \n
                                                                                      \n
                                                                                    • Switch to uv + add Python v3.14 support.  (#219)
                                                                                      • \n
                                                                                    • Add ruff formatter and linter. #216
                                                                                      • \n
                                                                                    • Drop Python 3.8 support. If you require it, use version 24.1.0. #204
                                                                                      • \n
                                                                                    \n
                                                                                    New Contributors
                                                                                    \n\n
                                                                                    Full Changelog: https://github.com/Tinche/aiofiles/compare/v24.1.0...v25.1.0
                                                                                    \n
                                                                                    \n
                                                                                    \n
                                                                                    \nChangelog\n
                                                                                    Sourced from aiofiles's changelog.
                                                                                    \n
                                                                                    \n
                                                                                    25.1.0 (2025-10-09)
                                                                                    \n
                                                                                      \n
                                                                                    • Switch to uv + add Python v3.14 support.\n(#219)
                                                                                      • \n
                                                                                    • Add ruff formatter and linter.\n#216
                                                                                      • \n
                                                                                    • Drop Python 3.8 support. If you require it, use version 24.1.0.\n#204
                                                                                      • \n
                                                                                    \n
                                                                                    24.1.0 (2024-06-24)
                                                                                    \n
                                                                                      \n
                                                                                    • Import os.link conditionally to fix importing on android.\n#175
                                                                                      • \n
                                                                                    • Remove spurious items from aiofiles.os.__all__ when running on Windows.
                                                                                      • \n
                                                                                    • Switch to more modern async idioms: Remove types.coroutine and make AiofilesContextManager an awaitable instead a coroutine.
                                                                                      • \n
                                                                                    • Add aiofiles.os.path.abspath and aiofiles.os.getcwd.\n#174
                                                                                      • \n
                                                                                    • aiofiles is now tested on Python 3.13 too.\n#184
                                                                                      • \n
                                                                                    • Drop Python 3.7 support. If you require it, use version 23.2.1.
                                                                                      • \n
                                                                                    \n
                                                                                    23.2.1 (2023-08-09)
                                                                                    \n
                                                                                      \n
                                                                                    • Import os.statvfs conditionally to fix importing on non-UNIX systems.\n#171 #172
                                                                                      • \n
                                                                                    • aiofiles is now also tested on Windows.
                                                                                      • \n
                                                                                    \n
                                                                                    23.2.0 (2023-08-09)
                                                                                    \n
                                                                                    \n
                                                                                    \nChangelog\n
                                                                                    Sourced from actions/checkout's changelog.
                                                                                    \n
                                                                                    \n
                                                                                    Changelog
                                                                                    \n
                                                                                    V6.0.0
                                                                                    \n\n
                                                                                    V5.0.1
                                                                                    \n\n
                                                                                    V5.0.0
                                                                                    \n\n
                                                                                    V4.3.1
                                                                                    \n\n
                                                                                    V4.3.0
                                                                                    \n
                                                                                      \n
                                                                                    • docs: update README.md by @\u200bmotss in actions/checkout#1971
                                                                                      • \n
                                                                                    • Add internal repos for checking out multiple repositories by \r\n\r\n- [docker] build arm docker image for mac\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-26T20:33:16Z"}
{"number": 2048, "title": "[nit] avoid using mutable dict as default argument in conversation", "files": ["src/oumi/core/types/conversation.py", "tests/unit/core/types/test_conversation.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [nit] avoid using mutable dict as default argument in conversation \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-21T18:31:54Z"}
{"number": 2047, "title": "[docs] Add docs for missing inference params and how to serve lora adapters", "files": ["docs/user_guides/infer/common_workflows.md", "docs/user_guides/infer/configuration.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/train/configuration.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] Add docs for missing inference params and how to serve lora adapters\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # OPE-1330 & OPE-1583\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-21T18:21:17Z"}
{"number": 2046, "title": "[tiny] cleanup logs & warnings", "files": ["src/oumi/utils/logging.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Ignore some of the expected logs and warnings that pollute stdout (e.g. triton not available on mac)\r\n- In debug mode, everything is still logged\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-21T17:57:17Z"}
{"number": 2045, "title": "October updates in README", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Added October Oumi updates to README\r\n", "merged_at": "2025-11-24T16:03:50Z"}
{"number": 2039, "title": "Bump omegaconf from 2.4.0.dev3 to 2.4.0.dev4", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Bumps [omegaconf](https://github.com/omry/omegaconf) from 2.4.0.dev3 to 2.4.0.dev4.\n
                                                                                      \nRelease notes\n
                                                                                      Sourced from omegaconf's releases.
                                                                                      \n
                                                                                      \n
                                                                                      OmegaConf 2.4.0dev4
                                                                                      \n
                                                                                      Final pre-release, barring bug reports/fixes, before stable release of OmegaConf 2.4.0
                                                                                      \n
                                                                                      \n
                                                                                      \n
                                                                                      \nCommits\n\n
                                                                                      \n
                                                                                      \n\n\n[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=omegaconf&package-manager=pip&previous-version=2.4.0.dev3&new-version=2.4.0.dev4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)\n\nDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.\n\n[//]: # (dependabot-automerge-start)\n[//]: # (dependabot-automerge-end)\n\n---\n\n
                                                                                      \nDependabot commands and options\n
                                                                                      \n\nYou can trigger Dependabot actions by commenting on this PR:\n- `@dependabot rebase` will rebase this PR\n- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it\n- `@dependabot merge` will merge this PR after your CI passes on it\n- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it\n- `@dependabot cancel merge` will cancel a previously requested merge and block automerging\n- `@dependabot reopen` will reopen this PR if it is closed\n- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually\n- `@dependabot show  ignore conditions` will show all of the ignore conditions of the specified dependency\n- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)\n- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)\n\n\n
                                                                                      ", "merged_at": "2025-11-20T05:49:37Z"}
{"number": 2038, "title": "Update bitsandbytes requirement from <0.48,>=0.47 to >=0.47,<0.49", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [bitsandbytes](https://github.com/bitsandbytes-foundation/bitsandbytes) to permit the latest version.\n
                                                                                      \nRelease notes\n
                                                                                      Sourced from bitsandbytes's releases.
                                                                                      \n
                                                                                      \n
                                                                                      0.48.2
                                                                                      \n
                                                                                      What's Changed
                                                                                      \n\n
                                                                                      Full Changelog: https://github.com/bitsandbytes-foundation/bitsandbytes/compare/0.48.1...0.48.2
                                                                                      \n
                                                                                      \n
                                                                                      \n
                                                                                      \nChangelog\n
                                                                                      Sourced from bitsandbytes's changelog.
                                                                                      \n
                                                                                      \n
                                                                                      v0.45.1
                                                                                      \n
                                                                                      Improvements:
                                                                                      \n
                                                                                        \n
                                                                                      • Compatibility for triton>=3.2.0
                                                                                        • \n
                                                                                      • Moved package configuration to pyproject.toml
                                                                                        • \n
                                                                                      • Build system: initial support for NVIDIA Blackwell B100 GPUs, RTX 50 Blackwell series GPUs and Jetson Thor Blackwell.\n
                                                                                          \n
                                                                                        • Note: Binaries built for these platforms are not included in this release. They will be included in future releases upon the availability of the upcoming CUDA Toolkit 12.7 and 12.8.
                                                                                          • \n
                                                                                        \n
                                                                                        • \n
                                                                                      \n
                                                                                      Bug Fixes:
                                                                                      \n
                                                                                        \n
                                                                                      • Packaging: wheels will no longer include unit tests. (#1478)
                                                                                        • \n
                                                                                      \n
                                                                                      Dependencies:
                                                                                      \n
                                                                                        \n
                                                                                      • Sets the minimum PyTorch version to 2.0.0.
                                                                                        • \n
                                                                                      \n
                                                                                      0.45.0
                                                                                      \n
                                                                                      This is a significant release, bringing support for LLM.int8() to NVIDIA Hopper GPUs such as the H100.
                                                                                      \n
                                                                                      As part of the compatibility enhancements, we've rebuilt much of the LLM.int8() code in order to simplify for future compatibility and maintenance. We no longer use the col32 or architecture-specific tensor layout formats while maintaining backwards compatibility. We additionally bring performance improvements targeted for inference scenarios.
                                                                                      \n
                                                                                      Performance Improvements
                                                                                      \n
                                                                                      This release includes broad performance improvements for a wide variety of inference scenarios. See this X thread for a detailed explanation.
                                                                                      \n
                                                                                      Breaking Changes
                                                                                      \n
                                                                                      \ud83e\udd17PEFT users wishing to merge adapters with 8-bit weights will need to upgrade to peft>=0.14.0.
                                                                                      \n
                                                                                      Packaging Improvements
                                                                                      \n
                                                                                        \n
                                                                                      • The size of our wheel has been reduced by ~43.5% from 122.4 MB to 69.1 MB! This results in an on-disk size decrease from ~396MB to ~224MB.
                                                                                        • \n
                                                                                      • Binaries built with CUDA Toolkit 12.6.2 are now included in the PyPI distribution.
                                                                                        • \n
                                                                                      • The CUDA 12.5.0 build has been updated to CUDA Toolkit 12.5.1.
                                                                                        • \n
                                                                                      \n
                                                                                      Deprecations
                                                                                      \n
                                                                                        \n
                                                                                      • A number of public API functions have been marked for deprecation and will emit FutureWarning when used. These functions will become unavailable in future releases. This should have minimal impact on most end-users.
                                                                                        • \n
                                                                                      • The k-bit quantization feature", "merged_at": "2025-11-20T05:50:35Z"}
{"number": 2037, "title": "[bug] remove unsupported assignee in dependabot config", "files": [".github/dependabot.yml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- dependabot assignee cannot be a team. removing for now\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T22:31:51Z"}
{"number": 2034, "title": "Deprecate experimental cambrian model", "files": [".pre-commit-config.yaml", "pyproject.toml", "src/oumi/builders/models.py", "src/oumi/models/experimental/cambrian/__init__.py", "src/oumi/models/experimental/cambrian/constants.py", "src/oumi/models/experimental/cambrian/mm_utils.py", "src/oumi/models/experimental/cambrian/model/__init__.py", "src/oumi/models/experimental/cambrian/model/builder.py", "src/oumi/models/experimental/cambrian/model/cambrian_arch.py", "src/oumi/models/experimental/cambrian/model/language_model/cambrian_llama.py", "src/oumi/models/experimental/cambrian/model/language_model/cambrian_phi3.py", "src/oumi/models/experimental/cambrian/model/language_model/phi3/__init__.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/__init__.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/base_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/builder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/clip_convnext_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/clip_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/dino_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/load.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/siglip_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_projector/builder.py", "src/oumi/models/experimental/cambrian/model/multimodal_projector/projectors.py", "src/oumi/models/experimental/cambrian/model/vision_sampler.py", "src/oumi/models/experimental/cambrian/utils.py", "tests/unit/test_apache_license_header.py"], "area": "other", "area_votes": {"infra": 2, "other": 18}, "body": "# Description\r\n\r\n\r\n\r\n- Deprecate experimental cambrian model\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T19:30:43Z"}
{"number": 2033, "title": "[tiny] Allow trl 0.24 to unblock python 3.9 support", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- trl 0.25 dropped support for python 3.9\r\n- to unblock python 3.9 users, this PR updates the trl dependency to include 0.24\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T18:19:05Z"}
{"number": 2032, "title": "Update wandb requirement from <0.22,>=0.21 to >=0.21,<0.24", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [wandb](https://github.com/wandb/wandb) to permit the latest version.\n
                                                                                        \nRelease notes\n
                                                                                        Sourced from wandb's releases.
                                                                                        \n
                                                                                        \n
                                                                                        v0.23.0
                                                                                        \n
                                                                                        What's Changed
                                                                                        \n
                                                                                        Added
                                                                                        \n\n
                                                                                        Fixed
                                                                                        \n\n
                                                                                        \n
                                                                                        \n
                                                                                        \nChangelog\n
                                                                                        Sourced from wandb's changelog.
                                                                                        \n
                                                                                        \n
                                                                                        [0.23.0] - 2025-11-11
                                                                                        \n
                                                                                        Added
                                                                                        \n\n
                                                                                        Fixed
                                                                                        \n
                                                                                          \n
                                                                                        • Artifact.files() now has a correc", "merged_at": "2025-11-19T19:08:09Z"}
{"number": 2031, "title": "Update safetensors requirement from <0.7,>=0.6 to >=0.6,<0.8", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Updates the requirements on [safetensors](https://github.com/huggingface/safetensors) to permit the latest version.\n
                                                                                          \nRelease notes\n
                                                                                          Sourced from safetensors's releases.
                                                                                          \n
                                                                                          \n
                                                                                          v0.7.0
                                                                                          \n
                                                                                          What's new
                                                                                          \n
                                                                                          complex64 support
                                                                                          \n
                                                                                          This release adds support for storing complex64 tensors.
                                                                                          \n
                                                                                          What's Changed
                                                                                          \n\n
                                                                                          New Contributors
                                                                                          \n
                                                                                            \n
                                                                                          • @\u200bzeroRains made their first contribution in huggingface/safetensors#630
                                                                                            • \n
                                                                                          • @\u200bc", "merged_at": "2025-11-19T19:15:22Z"}
{"number": 2030, "title": "Bump the minor-and-patch group with 2 updates", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Bumps the minor-and-patch group with 2 updates: [antlr4-python3-runtime](http://www.antlr.org) and [einops](https://github.com/arogozhnikov/einops).\n\nUpdates `antlr4-python3-runtime` from 4.11 to 4.13.2\n\nUpdates `einops` from 0.6.1 to 0.8.1\n
                                                                                            \nRelease notes\n
                                                                                            Sourced from einops's releases.
                                                                                            \n
                                                                                            \n
                                                                                            v0.8.1 Multiple improvements
                                                                                            \n
                                                                                            What's Changed
                                                                                            \n
                                                                                            TLDR:
                                                                                            \n
                                                                                              \n
                                                                                            • ellipsis is added to EinMix
                                                                                              • \n
                                                                                            • tests moved into the package
                                                                                              • \n
                                                                                            • devcontainer provided
                                                                                              • \n
                                                                                            • added backend for pyTensor
                                                                                              • \n
                                                                                            • niceties: citation, docs, fixed broken links
                                                                                              • \n
                                                                                            • this did not require any changes in einops, but array API is supported by more libs these days, and einops can operate on them
                                                                                              • \n
                                                                                            \n
                                                                                            PRs:
                                                                                            \n
                                                                                              \n
                                                                                            • Update README.md by @\u200bproject-delphi in arogozhnikov/einops#316
                                                                                              • \n
                                                                                            • change tests for numpy2.0 + fix for numpy regression  by @\u200barogozhnikov in arogozhnikov/einops#325
                                                                                              • \n
                                                                                            • PR from @\u200bpzread: fix tinygrad concat + changed formatting to fix CI errors by @\u200barogozhnikov in arogozhnikov/einops#329
                                                                                              • \n
                                                                                            • Add devcontainer and dockerfile by @\u200barogozhnikov in arogozhnikov/einops#330
                                                                                              • \n
                                                                                            • delete experimental/indexing by @\u200barogozhnikov in arogozhnikov/einops#333
                                                                                              • \n
                                                                                            • Use same ruff version in CI and in pre-commit, reformat notebooks by @\u200barogozhnikov in arogozhnikov/einops#336
                                                                                              • \n
                                                                                            • Delete chainer backend by @\u200barogozhnikov in arogozhnikov/einops#335
                                                                                              • \n
                                                                                            • Mention support of cupy, pydata/sparse, ndonnx and others by @\u200barogozhnikov in arogozhnikov/einops#337
                                                                                              • \n
                                                                                            • Moving tests into source tree by @\u200barogozhnikov in arogozhnikov/einops#338
                                                                                              • \n
                                                                                            • Update documentation, mention changes in testing + devcontainers by @\u200barogozhnikov in arogozhnikov/einops#339
                                                                                              • \n
                                                                                            • Auto-deploy mkdocs + update mkdocs requirements by @\u200barogozhnikov in arogozhnikov/einops#341
                                                                                              • \n
                                                                                            • Add dict syntax to the einops layers example by @\u200beadadi in arogozhnikov/einops#342
                                                                                              • \n
                                                                                            • include CITATION.cff for einops by @\u200bluke-carlson in arogozhnikov/einops#345
                                                                                              • \n
                                                                                            • mention that cupy and jax now support array api by @\u200barogozhnikov in \nRelease notes\n
                                                                                              Sourced from actions/checkout's releases.
                                                                                              \n
                                                                                              \n
                                                                                              v5.0.0
                                                                                              \n
                                                                                              What's Changed
                                                                                              \n\n
                                                                                              \u26a0\ufe0f Minimum Compatible Runner Version
                                                                                              \n
                                                                                              v2.327.1
                                                                                              \nRelease Notes
                                                                                              \n
                                                                                              Make sure your runner is updated to this version or newer to use this release.
                                                                                              \n
                                                                                              Full Changelog: https://github.com/actions/checkout/compare/v4...v5.0.0
                                                                                              \n
                                                                                              v4.3.1
                                                                                              \n
                                                                                              What's Changed
                                                                                              \n\n
                                                                                              Full Changelog: https://github.com/actions/checkout/compare/v4...v4.3.1
                                                                                              \n
                                                                                              v4.3.0
                                                                                              \n
                                                                                              What's Changed
                                                                                              \n\n
                                                                                              New Contributors
                                                                                              \n
                                                                                                \n
                                                                                              • @\u200bmotss made their first contribution in actions/checkout#1971
                                                                                                • \n
                                                                                              • \r\n\r\n- Add github action to manage stale issues / PRs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T18:12:13Z"}
{"number": 2027, "title": "Update github actions runs on config", "files": [".github/workflows/doctests.yaml", ".github/workflows/gpu_install_test.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/install_test.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 5}, "body": "# Description\r\n\r\n\r\n\r\n- `merge_group` does not support `paths`, unlike `pull_request`\r\n- Re-enable some tests to run on PRs: `docs`, `install`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T18:32:43Z"}
{"number": 2026, "title": "[nit] run yaml linter on github config files", "files": [".github/workflows/doctests.yaml", ".github/workflows/gpu_install_test.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/install_test.yaml", ".github/workflows/pretest.yaml", ".github/workflows/release_gcp.yaml", ".github/workflows/release_pypi.yaml"], "area": "infra", "area_votes": {"infra": 7}, "body": "# Description\r\n\r\n\r\n\r\n- Run yaml linter on github config files\r\n- No logic change, just linting\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T17:24:18Z"}
{"number": 2025, "title": "Add dependabot config", "files": [".github/dependabot.yml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add dependabot config\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T17:59:03Z"}
{"number": 2024, "title": "[tiny] fix lint error on readme.md", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [tiny] fix lint error on readme.md\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T00:46:49Z"}
{"number": 2023, "title": "Fix docker release action", "files": [".github/workflows/release_docker.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- The `docker_release` github action is running into disk space limits of the github VMs (~14GB available)\r\n  - Upgrading to larger VM did not help as they all have the same disk quota \r\n- This PRs adds a step to clear some unused files to allow for the image to build.\r\n  - Found this trick on this [blog](https://carlosbecker.com/posts/github-actions-disk-space/) \r\n- Tested here: https://github.com/oumi-ai/oumi/pkgs/container/oumi\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T01:08:03Z"}
{"number": 2022, "title": "Added news item on OpenEnv notebook", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "See title", "merged_at": "2025-11-18T23:15:00Z"}
{"number": 2021, "title": "enable custom master port for distributed training", "files": ["src/oumi/cli/distributed_run.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\nThe Pytorch distributed port is hard coded as 8007 ( _DEFAULT_MASTER_PORT ) and can not be changed when using Slurm or Polaris scheduler. However, due to some internal/entreprise restriction, this port might be unavailable.\r\nIt should be possible to use the 'classical' environement variable MASTER_PORT defined in Pytorch for more flexibility\r\n\r\n\r\n\r\n# Solution\r\nThe changes in the code is to use the existing code from the local machine process\r\n\r\n`master_port=int(env.get(_MASTER_PORT_ENV, _DEFAULT_MASTER_PORT))`\r\n\r\nIf the environement is not set, we fall back to the default value  = 8007\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-19T18:11:56Z"}
{"number": 2020, "title": "[bug] Fix oumi distributed on Slurm to use correct node rank env var", "files": ["src/oumi/cli/distributed_run.py", "tests/unit/cli/test_cli_distributed_run.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\nIn `oumi distributed`, we set the node rank to the value of `SLURM_PROCID` or `PMI_RANK`. However, this only works if the user specifies 1 task per node for the Slurm job, and will crash otherwise. This PR instead uses `SLURM_NODEID`, which correctly reflects the node id. Tested on Exun and Perlmutter.\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-11-18T19:54:21Z"}
{"number": 2019, "title": "[docs] add documentation page for oumi tune", "files": ["docs/index.md", "docs/user_guides/tune.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] add documentation page for oumi tune\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-18T17:47:15Z"}
{"number": 2018, "title": "[bug] Fix oumi distributed slurm node list bug", "files": ["src/oumi/cli/distributed_run.py", "tests/unit/cli/test_cli_distributed_run.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\nOur previous logic didn't expand Slurm's nodelist format out properly in the case of multiple nodes. This PR fixes that by using the recommended utility `scontrol show hostnames`. Tested this change on Exun, Frontier and Perlmutter.\r\n\r\n## Related issues\r\n\r\nFixes #2015\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-11-17T23:06:01Z"}
{"number": 2017, "title": "[bug] add tune target to ci_gpu", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- While `tune` is optional, it is still needed to run the tune integration tests\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-17T21:30:25Z"}
{"number": 2016, "title": "[docs] add/update docs for v0.5 release", "files": ["docs/get_started/tutorials.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] add/update docs for v0.5 release\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-17T20:18:10Z"}
{"number": 2014, "title": "[tune] make optuna import optional", "files": ["src/oumi/core/tuners/optuna_tuner.py", "tests/unit/core/tuners/test_optuna_tuner.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Make optuna import optional since it's not part of the default dependencies\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-17T18:22:16Z"}
{"number": 2012, "title": "Update OpenEnv notebook with reward graph", "files": ["notebooks/Oumi - OpenEnv GRPO with trl.ipynb", "notebooks/assets/openenv_echo_reward.png"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n- Add instructions for enabling wandb\r\n- Increase max completion length\r\n\r\n## Related issues\r\n\r\nFixes OPE-1762\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-11-14T21:14:09Z"}
{"number": 2011, "title": "deps: Upgrade trl to 0.25", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nRan e2e tests.\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-11-14T17:41:09Z"}
{"number": 2010, "title": "Updated SLURM process environment variable: replaced PMI_RANK with SLURM_PROCID", "files": ["src/oumi/cli/distributed_run.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "\r\n# Description\r\nWhile using the Oumi distributed wrapper, I encountered an issue where processes failed to establish a rendezvous. After investigating the code, I found that during the detection of the slurm process run info (`_detect_slurm_process_run_info)`, the wrapper was assigning the wrong node_rank. It used the environment variable PMI_RANK instead of SLURM_PROCID.\r\n\r\n# Solution\r\n- Use either `SLURM_PROCID` or `PMI_RANK` to set the node rank in multi-node setups.  \r\n- For single-node setups, the node rank is automatically set to 0.  \r\n- Raise an error if `node_rank` is not set in multi-node setups, since it is mandatory for `torchrun`.\r\n## Related issues\r\n#2007 \r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-14T22:20:38Z"}
{"number": 2009, "title": "[bug] fix issue initializing gkd trainer", "files": ["src/oumi/core/configs/params/gkd_params.py", "tests/unit/core/configs/params/test_gkd_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Fix bug initializing the gkd trainer, where dtype was missing in some cases\r\n- Add unit tests\r\n- Switch from `torch_dtype` to `dtype`, as the former is getting deprecated\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-13T18:22:15Z"}
{"number": 2004, "title": "Made chat_template_kwargs and image_id_map fields optional for backwards compatibility", "files": ["src/oumi/core/configs/job_config.py", "src/oumi/core/configs/params/model_params.py", "tests/unit/core/configs/params/test_model_params.py", "tests/unit/inference/test_vllm_inference_engine.py"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\nI ran into a backwards compatibility issue with these fields. This should fix it by making them optional params.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n- Made chat_template_kwargs and image_id_map fields optional for backwards compatibility\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-06T22:24:39Z"}
{"number": 2003, "title": "Add call to action to all Oumi notebooks", "files": ["notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Build your own Custom Evaluation (Hallucination Classifier).ipynb", "notebooks/Oumi - Bulk Inference of LLM APIs.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - MiniMath-R1-1.5B.ipynb", "notebooks/Oumi - OpenEnv GRPO with trl.ipynb", "notebooks/Oumi - Quantization Tutorial.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Simple Judge.ipynb", "notebooks/Oumi - Train a Letter Counting Model using GRPO.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 20}, "body": "# Description\r\n\r\nMentions GitHub, Substack, Youtube, and Oumi Platform.\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-11-06T20:22:00Z"}
{"number": 2002, "title": "[bug] add default torch_dtype value required gkd trainer", "files": ["src/oumi/core/configs/params/gkd_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [bug] add default torch_dtype value required gkd trainer\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-04T17:27:22Z"}
{"number": 1995, "title": "Upgrade trl to 0.24.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nTested with E2E and GPU tests\r\n\r\n## Related issues\r\n\r\nTowards OPE-1762\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-10-28T22:01:24Z"}
{"number": 1993, "title": "Update Qwen3 VL 4b_instruct_fft_train.yaml", "files": ["configs/recipes/vision/qwen3_vl/sft/4b_instruct_fft_train.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "", "merged_at": "2025-10-23T18:23:29Z"}
{"number": 1991, "title": "Add tuning module for hyperparameter search/tuning", "files": ["src/oumi/core/configs/__init__.py", "src/oumi/core/configs/params/tuning_params.py", "src/oumi/core/configs/tuning_config.py", "tests/unit/core/configs/params/test_tuning_params.py", "tests/unit/core/configs/test_tuning_config.py"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\nThis PR is the start of a module to run hyperparameters tuning inside oumi. \r\nWith the changes here, I hope to implement the core foundations to use oumi to run a hyperparameter study (with SFT at first). \r\nI hope to use Optuna together with oumi to orchestrate the study.\r\n\r\n## Related issues\r\n_This PR fixes an issue I've opened a few days ago regarding my interest in developing a new hyperparameter tuning module for oumi._\r\n\r\nFixes #1984 \r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ x ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ x ] Did you link the issue(s) related to this PR in the section above?\r\n- [ x ] Did you add / update tests where needed?\r\n", "merged_at": "2025-10-20T15:44:42Z"}
{"number": 1989, "title": "Improve error raising for document processing issues", "files": ["src/oumi/core/synthesis/dataset_planner.py", "tests/unit/core/synthesis/test_dataset_planner.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-10-14T20:04:16Z"}
{"number": 1985, "title": "Lazy init clouds in oumi launcher", "files": ["src/oumi/launcher/launcher.py", "tests/unit/launcher/test_launcher.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Some cloouds have a non hermetic init, which can cause issues in some situations (e.g. air-gapped environments or temporal worfklows)+ slows down the overall launch\r\n- This PR skips initializing all the clouds at startup. Instead they are lazily initialized as needed\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-10-14T06:48:21Z"}
{"number": 1982, "title": "Refactor analyze", "files": ["src/oumi/core/analyze/column_types.py", "src/oumi/core/analyze/dataframe_analyzer.py", "src/oumi/core/analyze/dataset_analyzer.py", "src/oumi/core/analyze/length_analyzer.py", "src/oumi/core/analyze/sample_analyzer.py", "src/oumi/utils/analysis_utils.py", "tests/unit/core/analyze/test_dataset_analyzer.py", "tests/unit/core/test_length_analyzer.py"], "area": "evaluation", "area_votes": {"evaluation": 5, "other": 1}, "body": "# Description\r\nRefactor Analyze to make it more general and able to analyze any dataset: \r\n* Data is modeled as pandas dataframes, it allows repeated keys (unlike a dict)\r\n* Analyzers with Dataframes allow having any number of columns to be analyzed. Schema will define what these columns are\r\n* It can analyze different types of content: text, float, timestamp, object, ... which will be defined for each column in the schema\r\n* Analyzers are now agnostic to the dataset object's logic. they only take a dataframe and a schema\r\n* Analyzers can join multiple dataframes based on the join_on columns passed to them. Nested messages can be handled this way. For example message level dataframe and conversation level dataframe\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards LOU-12\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-10-17T17:44:30Z"}
{"number": 1979, "title": "Update uv pip install commands to include --system", "files": [".github/workflows/doctests.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml", "configs/examples/berry_bench/evaluation/gcp_job.yaml", "configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/examples/grpo_tldr/gcp_job.yaml", "configs/examples/grpo_verl_countdown/gcp_job.yaml", "configs/examples/grpo_verl_geometry3k/gcp_job.yaml", "configs/examples/letter_counting/evaluation/gcp_job.yaml", "configs/examples/letter_counting/grpo/gcp_job.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/projects/chatqa/gcp_job.yaml", "configs/projects/halloumi/gcp_job.yaml", "configs/projects/wc50m/configs/gcp_base_ultrachat.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_gcp_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_0_5b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b_deep/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_34b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_3b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_7b/lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_0_5b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b_deep/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_34b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_3b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_7b/full_lambda_job.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/gpt2/pretraining/gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_3/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama4/sft/scout_base_full/gcp_job.yaml", "configs/recipes/llama4/sft/scout_instruct_full/gcp_job.yaml", "configs/recipes/phi3/dpo/gcp_job.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/phi4/evaluation/reasoning_plus_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/full_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_gcp_job.yaml", "configs/recipes/qwen3/evaluation/0.6b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/1.7b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/14b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/30b_a3b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/32b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/4b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/8b_gcp_job.yaml", "configs/recipes/qwen3/sft/0.6b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/1.7b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/14b_lora/gcp_job.yaml", "configs/recipes/qwen3/sft/30b_a3b_lora/gcp_job.yaml", "configs/recipes/qwen3/sft/32b_lora/gcp_job.yaml", "configs/recipes/qwen3/sft/4b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/8b_full/gcp_job.yaml", "configs/recipes/qwq/evaluation/gcp_job.yaml", "configs/recipes/qwq/sft/full_gcp_job.yaml", "configs/recipes/qwq/sft/lora_gcp_job.yaml", "configs/recipes/qwq/sft/qlora_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/sft/135m/gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/sft/135m/slurm_job.yaml", "configs/recipes/vision/internvl3/sft/full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/full/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/full/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/lora/gcp_job.yaml", "configs/recipes/vision/phi4/sft/full/gcp_job.yaml", "configs/recipes/vision/phi4/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/full/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/full/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/lora/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/full/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/lora/gcp_job.yaml", "docs/user_guides/launch/launch.md", "notebooks/Oumi - Running Jobs Remotely.ipynb", "scripts/datasets/pretokenize/sky.yaml", "scripts/demo.py", "src/experimental/configs/projects/zephyr/sft/full_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_gcp_job.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "tests/scripts/e2e_tests_job.yaml", "tests/scripts/runpod_e2e_tests_job.yaml"], "area": "configs", "area_votes": {"infra": 5, "configs": 116, "docs": 2, "other": 3}, "body": "# Description\n\n\n\nE2E tests are failing now due to a message suggesting to use `uv venv` or `--system`. Updating pip installs to use `--system` which aligns with how our workflow tests run.\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-10-20T15:33:41Z"}
{"number": 1978, "title": "Pin uvicorn version to fix skypilot", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\n\n\n\nUvicorn 0.36.0 made a breaking change for skypilot that has yet to be addressed, pinning <0.36.0 to fix.\n\n\n## Related issues\n\n\n\n\nRelated to https://github.com/skypilot-org/skypilot/issues/7303\n\n\n\n## Before submitting\n\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-09-22T18:01:52Z"}
{"number": 1974, "title": "Make stdout_file optional in slurm client", "files": ["src/oumi/launcher/clients/slurm_client.py", "tests/unit/launcher/clients/test_slurm_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description\n\nUpdates the type annotation for the `stdout_file` parameter in `SlurmClient.submit_job` from `str` to `Optional[str]`. This change correctly reflects that the parameter can be `None`, aligning with other optional parameters in the method signature.\n\n## Related issues\n\nFixes #OPE-1555\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n---\nLinear Issue: [OPE-1555](https://linear.app/oumi/issue/OPE-1555)\n\n\n  \n    \n    \n    \"Open\n  \n\n\n  \n    \n    \n    \"Open\n  \n\n\n", "merged_at": "2025-09-12T15:20:32Z"}
{"number": 1969, "title": "extract conversation_turns from conversation_level_summary to top level", "files": ["src/oumi/core/analyze/dataset_analyzer.py", "tests/unit/core/analyze/test_dataset_analyzer.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\nextract conversation_turns from conversation_level_summary to top level\n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-09-04T22:04:54Z"}
{"number": 1968, "title": "Update README.md", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n- Mention 0.4.0 release\r\n- Add mentions of new trainers\r\n- Update list of supported models", "merged_at": "2025-09-05T16:02:28Z"}
{"number": 1967, "title": "Fixed Sky Pilot Unit Tests Failing", "files": ["src/oumi/launcher/clients/sky_client.py", "tests/unit/cli/test_cli_launch.py", "tests/unit/launcher/clients/test_sky_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description\r\n\r\n\r\n\r\nThere was a regression from this PR (https://github.com/oumi-ai/oumi/pull/1951) where it broke unit tests even though it still got merged into main. This PR fixes the sky pilot failing unit tests based on the new mocks we needed for the upgraded sky pilot version.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1531\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-09-04T02:32:52Z"}
{"number": 1966, "title": "[tiny] Upgrade transformers to 4.56", "files": ["pyproject.toml", "tests/e2e/test_eval_e2e.py", "tests/scripts/e2e_tests_job.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nUpgade to latest version. Confirmed E2E tests pass, and made minor changes\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-09-05T16:03:08Z"}
{"number": 1961, "title": "Fix NaN values in dataset analyzer statistics for single conversations", "files": ["src/oumi/core/analyze/dataset_analyzer.py", "tests/unit/core/analyze/test_dataset_analyzer.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\nStd is set to 0 instead of NaN when aggregating a single item \n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-09-02T22:10:35Z"}
{"number": 1960, "title": "Penfever/llamacpp role fix", "files": ["src/oumi/inference/llama_cpp_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nAdding missing support for SYSTEM role in LlamaCPP engine inference.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-09-01T22:22:06Z"}
{"number": 1957, "title": "[tiny] Remove `add_special_tokens` kwarg from vision DPO dataset processor", "files": ["src/oumi/core/datasets/vision_language_dpo_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\nThis causes an error for Phi3 DPO because it's not a kwarg for the Phi3 processor: https://huggingface.co/microsoft/Phi-3-vision-128k-instruct/blob/main/processing_phi3_v.py#L58. It's not in the Qwen2 VL processor either: https://github.com/huggingface/transformers/blob/e3d8fd730ed063a88edc49ed5f3c8acfabb53368/src/transformers/models/qwen2_vl/processing_qwen2_vl.py#L93\r\n\r\n## Related issues\r\n\r\nTowards OPE-1524\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-28T16:36:04Z"}
{"number": 1954, "title": "Follow-up updates to DeepSpeed PR", "files": ["configs/examples/deepspeed/llama3_1_8b_deepspeed_z2_train.yaml", "configs/examples/deepspeed/llama3_1_8b_deepspeed_z3_offload_train.yaml", "configs/examples/deepspeed/llama3_1_8b_deepspeed_z3_train.yaml", "configs/projects/limo/qwen2.5_7b_fft.yaml", "configs/projects/limo/qwen2.5_7b_fft_yarn.yaml", "configs/projects/limo/qwen2.5_7b_fft_yarn_deepspeed.yaml", "configs/projects/limo/qwen2.5_7b_fft_yarn_deepspeed_memory_optimized_train.yaml", "src/oumi/core/configs/params/deepspeed_params.py"], "area": "configs", "area_votes": {"configs": 8}, "body": "# Description\r\n\r\nMisc. changes building on top of #1886, which added DeepSpeed support.\r\n\r\n- Moved Limo configs to configs/projects/limo.\r\n- Updated typing and documentation in DeepspeedParams\r\n\r\n## Related issues\r\n\r\nFixes OPE-1513\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-26T23:35:33Z"}
{"number": 1952, "title": "Run GitHub workflows in merge queue instead of on PR events", "files": [".github/workflows/doctests.yaml", ".github/workflows/gpu_install_test.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/install_test.yaml", ".github/workflows/pretest.yaml", ".github/workflows/release_docker.yaml"], "area": "infra", "area_votes": {"infra": 6}, "body": "# Description\r\n\r\nCurrently, our test workflows are run whenever a PR is opened, synchronize, or reopened. This is eating into our quota. This PR instead has those tests run in a merge queue when the PR is submitted (with the exception of pretest.yaml, which we keep running for PR updates). The user is still able to run tests manually before the PR is submitted with the `workflow_dispatch` hook.\r\n\r\nDocumentation: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue\r\nhttps://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#pull_request\r\n\r\n## Related issues\r\n\r\nFixes OPE-1512\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-25T22:32:55Z"}
{"number": 1951, "title": "Add The Ability To Trail Logs For Launcher Jobs", "files": ["pyproject.toml", "src/oumi/cli/launch.py", "src/oumi/core/launcher/base_cluster.py", "src/oumi/launcher/clients/sky_client.py", "src/oumi/launcher/clients/slurm_client.py", "src/oumi/launcher/clusters/frontier_cluster.py", "src/oumi/launcher/clusters/local_cluster.py", "src/oumi/launcher/clusters/polaris_cluster.py", "src/oumi/launcher/clusters/sky_cluster.py", "src/oumi/launcher/clusters/slurm_cluster.py", "tests/unit/cli/test_cli_launch.py"], "area": "launcher", "area_votes": {"infra": 1, "cli": 1, "launcher": 8}, "body": "# Description\r\n\r\n\r\n\r\nWe currently have built-in support from SkyPilot for trailing logs through their SDK after launching a job. We want to extend this functionality to our other clusters as well.\r\n\r\nThis PR adds support for Slurm clusters by tailing the log file and streaming its contents directly to the user\u2019s terminal. For clusters not covered in this PR, the behavior will remain unchanged and continue to show the loading spinner.\r\n\r\nFeatures Added:\r\n- Tailing logs is now supported in Slurm as an IOStream to our rich console\r\n- Tailing logs is Sky Pilot is now returning an IOStream to our rich console\r\n- Support for passing in an IO file for the logs to be streamed to works with the parameter \"--output-filepath\"\r\n- Sky Pilot upgraded and patched to support streaming to a file\r\n\r\n**BEFORE With Slurm**\r\n\"Screenshot\r\n\r\n**AFTER With Slurm**\r\n\"image\"\r\n\r\n**AFTER With Sky Pilot**\r\n\"Screenshot\r\n\r\n**After With Local Cluster**\r\n\"Screenshot\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1344\r\nFixes #1735 \r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-09-03T17:07:02Z"}
{"number": 1950, "title": "Add a parsable enum state field to the launcher to convey job status", "files": ["docs/user_guides/launch/custom_cluster.md", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "src/oumi/core/launcher/__init__.py", "src/oumi/core/launcher/base_cluster.py", "src/oumi/launcher/clients/local_client.py", "src/oumi/launcher/clients/polaris_client.py", "src/oumi/launcher/clients/sky_client.py", "src/oumi/launcher/clients/slurm_client.py", "src/oumi/launcher/clusters/sky_cluster.py", "tests/unit/cli/test_cli_launch.py", "tests/unit/launcher/clients/data/qstat.txt", "tests/unit/launcher/clients/test_local_client.py", "tests/unit/launcher/clients/test_polaris_client.py", "tests/unit/launcher/clients/test_sky_client.py", "tests/unit/launcher/clients/test_slurm_client.py", "tests/unit/launcher/clouds/test_frontier_cloud.py", "tests/unit/launcher/clouds/test_local_cloud.py", "tests/unit/launcher/clouds/test_polaris_cloud.py", "tests/unit/launcher/clouds/test_sky_cloud.py", "tests/unit/launcher/clouds/test_slurm_cloud.py", "tests/unit/launcher/clusters/test_frontier_cluster.py", "tests/unit/launcher/clusters/test_local_cluster.py", "tests/unit/launcher/clusters/test_polaris_cluster.py", "tests/unit/launcher/clusters/test_sky_cluster.py", "tests/unit/launcher/clusters/test_slurm_cluster.py", "tests/unit/launcher/test_launcher.py"], "area": "launcher", "area_votes": {"docs": 2, "launcher": 6}, "body": "# Description\r\n\r\n\r\n\r\n\r\nPreviously the oumi launcher only exposed a `done` enum, as well as a string for status parsing. This PR adds an enum `state` to easily check if a job is `pending`, `failed`, `succeeded`, or `cancelled`.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-25T20:05:15Z"}
{"number": 1947, "title": "Penfever/api and macos configs", "files": [".gitignore", "configs/apis/anthropic/infer_claude_opus_4_1.yaml", "configs/apis/gemini/infer_gemini_2_5_pro.yaml", "configs/apis/openai/infer_chatgpt_4o_latest.yaml", "configs/apis/openai/infer_gpt_4_1.yaml", "configs/apis/openai/infer_gpt_4_1_mini.yaml", "configs/apis/openai/infer_gpt_4o_mini.yaml", "configs/apis/openai/infer_gpt_5.yaml", "configs/apis/openai/infer_gpt_5_chat_latest.yaml", "configs/apis/openai/infer_gpt_5_mini.yaml", "configs/apis/openai/infer_gpt_5_nano.yaml", "configs/apis/openai/infer_o1.yaml", "configs/apis/openai/infer_o1_mini.yaml", "configs/apis/openai/infer_o3_mini.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b_gguf_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b_gguf_macos_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b_vllm_infer.yaml", "configs/recipes/gemma3/inference/3n_e4b_it_gguf_infer.yaml", "configs/recipes/gemma3/inference/3n_e4b_it_gguf_macos_infer.yaml", "configs/recipes/gemma3/inference/3n_e4b_it_infer.yaml", "configs/recipes/gemma3/inference/3n_e4b_it_vllm_infer.yaml", "configs/recipes/glm4/inference/air_gguf_macos_infer.yaml", "configs/recipes/llama3_3/inference/nemotron_super_49b_gguf_macos_infer.yaml", "configs/recipes/llama4/inference/scout_instruct_gguf_macos_infer.yaml", "configs/recipes/qwen3/inference/30b_a3b_instruct_gguf_macos_infer.yaml", "configs/recipes/qwen3/inference/4b_instruct_gguf_infer.yaml", "configs/recipes/qwen3/inference/4b_instruct_gguf_macos_infer.yaml", "configs/recipes/qwen3/inference/4b_instruct_infer.yaml", "configs/recipes/qwen3/inference/4b_instruct_vllm_infer.yaml", "configs/recipes/qwen3_coder/inference/30b_a3b_instruct_gguf_macos_infer.yaml", "pyproject.toml"], "area": "configs", "area_votes": {"infra": 2, "configs": 29}, "body": "# Description\r\n\r\n\r\n\r\n\r\nAdds config files for newer API and local models (GPT 5, Qwen 3, Claude Opus 4.1, Gemma 3)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-25T20:56:36Z"}
{"number": 1946, "title": "Update llama_cpp_inference_engine.py", "files": ["src/oumi/inference/llama_cpp_inference_engine.py", "tests/unit/inference/test_llama_cpp_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nChange default LlamaCPP engine parameters for better memory management when running locally\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-22T18:41:18Z"}
{"number": 1945, "title": "Add shell script for launching jobs on NERSC Perlmutter HPC cluster", "files": ["scripts/frontier/launcher.sh", "scripts/perlmutter/README.md", "scripts/perlmutter/jobs/example_job.sh", "scripts/perlmutter/launcher.sh", "scripts/perlmutter/perlmutter_init.sh"], "area": "infra", "area_votes": {"infra": 4, "docs": 1}, "body": "# Description\r\n\r\nThis is similar to #1691, but for the Perlmutter cluster. Both use the Slurm scheduler, but Perlmutter has Nvidia GPUs and a slightly different setup for conda. Tested with `scripts/perlmutter/launcher.sh -u wizeng -d /global/homes/w/wizeng/oumi_launcher -j scripts/perlmutter/jobs/example_job.sh`.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1507\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-22T14:39:54Z"}
{"number": 1944, "title": "[tiny] Update .gitignore", "files": [".gitignore"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nThis is causing a bug with the Oumi launcher, because the oumi/src/oumi/models folder isn't being copied to remote clusters. We have output/ in the gitignore for storing model checkpoints.\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-21T03:21:44Z"}
{"number": 1936, "title": "Add Pre-Populated GitHub Issue Link On Failures", "files": ["src/oumi/cli/cli_utils.py", "src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 2}, "body": "# Description\r\n\r\nWe want to make users more aware that when they run into any types of issues they should open an issue. This PR appends a pre-populated Github issue link after any type of failure occurs for the user to optionally open if they feel it should be tracked by the team.\r\n\r\nExample: `oumi launch up -c configs/examples/grpo_verl_countdown/gcp_job.yaml --cluster grpo-verl-countdown`\r\n**BEFORE CHANGES**\r\n\"image\"\r\n\r\n**AFTER CHANGERS**\r\n\"Screenshot\r\n\"Screenshot\r\n\r\n**LONGER ERROR EXAMPLE**\r\n\"Screenshot\r\n\"Screenshot\r\n\r\nNOTE: For any jobs used with the launcher, rich console logs that leverage links won't work. Instead a check is made and if its a remote job we just display the template for the user to use instead of populating any fields\r\n\r\n\"Screenshot\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2025-08-20T18:07:31Z"}
{"number": 1934, "title": "Judge CLI | Display Overall Score", "files": ["src/oumi/cli/judge.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n**Judge CLI: Display Overall Score**\r\n\r\nOutput of command:\r\n```\r\noumi judge conversations \\\r\n  --config truthfulness \\\r\n  --input /completions.jsonl\r\n```\r\n...is shown below (Overall Score: 50.00%):\r\n\r\n\"image\"\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1327\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-18T21:28:31Z"}
{"number": 1933, "title": "Replace mentions of deprecated `huggingface-cli` tool with `hf`", "files": ["configs/examples/berry_bench/evaluation/eval.yaml", "configs/examples/berry_bench/evaluation/gcp_job.yaml", "configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/grpo_verl_countdown/gcp_job.yaml", "configs/examples/grpo_verl_geometry3k/gcp_job.yaml", "configs/examples/letter_counting/evaluation/eval.yaml", "configs/examples/letter_counting/evaluation/gcp_job.yaml", "configs/examples/letter_counting/grpo/gcp_job.yaml", "configs/examples/letter_counting/grpo/train.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/projects/aya/evaluation/eval.yaml", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/chatqa/gcp_job.yaml", "configs/projects/dcvlr/README.md", "configs/projects/halloumi/8b_train.yaml", "configs/projects/halloumi/gcp_job.yaml", "configs/projects/wc50m/configs/gcp_base_ultrachat.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_frontier_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_gcp_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_0_5b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b_deep/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_34b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_3b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_7b/lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_0_5b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b_deep/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_34b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_3b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_7b/full_lambda_job.yaml", "configs/recipes/glm4/inference/air_gguf_macos_infer.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/inference/70b_infer.yaml", "configs/recipes/llama3_1/inference/8b_infer.yaml", "configs/recipes/llama3_1/inference/8b_rvllm_infer.yaml", "configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/polaris_job.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/llama3_2/inference/1b_infer.yaml", "configs/recipes/llama3_2/inference/1b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/1b_vllm_infer.yaml", "configs/recipes/llama3_2/inference/3b_infer.yaml", "configs/recipes/llama3_2/inference/3b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/3b_vllm_infer.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/llama3_3/evaluation/70b_eval.yaml", "configs/recipes/llama3_3/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_3/inference/70b_infer.yaml", "configs/recipes/llama3_3/inference/70b_vllm_infer.yaml", "configs/recipes/llama3_3/inference/nemotron_super_49b_gguf_macos_infer.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml", "configs/recipes/llama4/evaluation/scout_instruct_eval.yaml", "configs/recipes/llama4/inference/scout_instruct_gguf_macos_infer.yaml", "configs/recipes/llama4/inference/scout_instruct_infer.yaml", "configs/recipes/llama4/inference/scout_instruct_vllm_infer.yaml", "configs/recipes/llama4/sft/scout_base_full/gcp_job.yaml", "configs/recipes/llama4/sft/scout_base_full/train.yaml", "configs/recipes/llama4/sft/scout_instruct_full/gcp_job.yaml", "configs/recipes/llama4/sft/scout_instruct_full/train.yaml", "configs/recipes/llama4/sft/scout_instruct_lora/train.yaml", "configs/recipes/llama4/sft/scout_instruct_qlora/train.yaml", "configs/recipes/phi4/evaluation/reasoning_plus_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/full_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_gcp_job.yaml", "configs/recipes/qwen3/evaluation/14b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/30b_a3b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/32b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/8b_gcp_job.yaml", "configs/recipes/qwen3/inference/30b_a3b_instruct_gguf_macos_infer.yaml", "configs/recipes/qwen3/sft/14b_lora/gcp_job.yaml", "configs/recipes/qwen3/sft/30b_a3b_lora/gcp_job.yaml", "configs/recipes/qwen3/sft/32b_lora/gcp_job.yaml", "configs/recipes/qwen3/sft/8b_full/gcp_job.yaml", "configs/recipes/qwen3_coder/inference/30b_a3b_instruct_gguf_macos_infer.yaml", "configs/recipes/qwq/evaluation/gcp_job.yaml", "configs/recipes/qwq/sft/full_gcp_job.yaml", "configs/recipes/qwq/sft/lora_gcp_job.yaml", "configs/recipes/qwq/sft/qlora_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job.yaml", "configs/recipes/vision/internvl3/sft/full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_rvllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/train.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/full/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/full/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/lora/gcp_job.yaml", "configs/recipes/vision/phi4/sft/full/gcp_job.yaml", "configs/recipes/vision/phi4/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/full/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/full/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/lora/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/full/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/lora/gcp_job.yaml", "docs/development/dev_setup.md", "docs/get_started/quickstart.md", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - MiniMath-R1-1.5B.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "scripts/frontier/jobs/example_job.sh", "scripts/polaris/jobs/download_model_from_hf.sh", "src/experimental/configs/projects/zephyr/evaluation/eval.yaml", "src/experimental/configs/projects/zephyr/sft/full_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/full_train.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_train.yaml", "tests/scripts/predownload_for_github_gpu_tests.sh"], "area": "configs", "area_votes": {"configs": 174, "docs": 6, "infra": 2, "other": 5}, "body": "# Description\r\n\r\nExample warning message: \r\n```\r\n\u26a0\ufe0f  Warning: 'huggingface-cli download' is deprecated. Use 'hf download' instead.\r\n```\r\n\r\nI did the following find-replaces in our codebase:\r\n- `huggingface-cli download` -> 'hf download'\r\n- `huggingface-cli login` -> 'hf auth login'\r\n- `huggingface-cli upload-large-folder` -> 'hf upload-large-folder'\r\n\r\nAdditional code changes were made by the linter, not me.\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-18T22:55:15Z"}
{"number": 1931, "title": "Replace Luma link with YouTube recording for August webinar", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2025-08-15T18:36:26Z"}
{"number": 1930, "title": "Update README.md - updated after Aug webinar", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Will replace Luma link with YouTube once the edited video is posted\r\n\r\n", "merged_at": "2025-08-14T18:59:46Z"}
{"number": 1929, "title": "Ryan arman add analysis summary", "files": ["src/oumi/core/analyze/dataset_analyzer.py", "tests/unit/core/analyze/test_dataset_analyzer.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\nAdding computation of summary statistics for message level and conversation level\r\nAlso raises an exception when analysis results are not ready instead of silently calling analyze_dataset \r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1496\r\nTowards OPE-1370\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-15T18:45:06Z"}
{"number": 1927, "title": "Update README.md to highlight `gpt-oss` support", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "The title says it all", "merged_at": "2025-08-13T02:22:48Z"}
{"number": 1926, "title": "[tiny] Cleanup redundant specifications of lora_dropout 0.0", "files": ["configs/projects/coalm/405b_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_train.yaml", "configs/recipes/falcon_h1/dpo/falcon_h1_0_5b/qlora_dpo.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/dpo/1b_qlora_dpo.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml", "configs/recipes/llama4/sft/scout_instruct_lora/train.yaml", "configs/recipes/llama4/sft/scout_instruct_qlora/train.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_train.yaml", "configs/recipes/qwen3/sft/14b_lora/train.yaml", "configs/recipes/qwen3/sft/30b_a3b_lora/train.yaml", "configs/recipes/qwen3/sft/32b_lora/train.yaml", "configs/recipes/qwq/sft/lora_train.yaml", "configs/recipes/qwq/sft/qlora_train.yaml", "notebooks/Oumi - Finetuning Tutorial.ipynb"], "area": "configs", "area_votes": {"configs": 31, "docs": 1}, "body": "# Description\r\n\r\nIn https://github.com/oumi-ai/oumi/pull/1124, we changed the default value of `lora_dropout` from 0.05 to 0. We can now clean up this field from our yaml configs to reduce bloat.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1490\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-12T17:51:09Z"}
{"number": 1924, "title": "Improve e2e eval tests", "files": ["configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "tests/e2e/test_eval_e2e.py", "tests/e2e/test_train_e2e.py", "tests/scripts/e2e_tests_job.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\n- Replace eval task for multi-modal models from `mmmu_val` to `mmmu_val_computer_science` to be in line with `mmlu_college_computer_science` for non-MM models. This also reduces runtime for E2E tests.\r\n- Delete redundant tests. We don't need to test Llama 1B/3B/8B all for eval if they're the same family.\r\n- Add parameter to support using vLLM inference engine for eval\r\n- Re-enable train_mm_llama3_2_vision_11b_full\r\n- Reduce num steps to reduce runtime\r\n\r\nTested that all e2e tests pass\r\n\r\n## Related issues\r\n\r\nTowards OPE-1483\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-13T05:59:45Z"}
{"number": 1923, "title": "GGUF configs, MacOS LlamaCPP configs", "files": [".gitignore", "configs/examples/macos_gguf/README.md", "configs/recipes/glm4/inference/air_gguf_infer.yaml", "configs/recipes/glm4/inference/air_gguf_macos_infer.yaml", "configs/recipes/glm4/inference/air_vllm_infer.yaml", "configs/recipes/llama3_3/inference/nemotron_super_49b_gguf_infer.yaml", "configs/recipes/llama3_3/inference/nemotron_super_49b_gguf_macos_infer.yaml", "configs/recipes/llama3_3/inference/nemotron_super_49b_vllm_infer.yaml", "configs/recipes/llama4/inference/scout_instruct_gguf_infer.yaml", "configs/recipes/llama4/inference/scout_instruct_gguf_macos_infer.yaml", "configs/recipes/qwen3/inference/30b_a3b_instruct_gguf_infer.yaml", "configs/recipes/qwen3/inference/30b_a3b_instruct_gguf_macos_infer.yaml", "configs/recipes/qwen3/inference/30b_a3b_instruct_vllm_infer.yaml", "configs/recipes/qwen3_coder/inference/30b_a3b_instruct_gguf_infer.yaml", "configs/recipes/qwen3_coder/inference/30b_a3b_instruct_gguf_macos_infer.yaml", "configs/recipes/qwen3_coder/inference/30b_a3b_instruct_vllm_infer.yaml"], "area": "configs", "area_votes": {"infra": 1, "docs": 1, "configs": 14}, "body": "# Description\r\n\r\n\r\n\r\n\r\nAdd Oumi configs for GGUFs and the LlamaCPP engine, making it easier for users to understand how to run inference in Oumi using these model types.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-16T21:41:05Z"}
{"number": 1921, "title": "Deprecate experimental ring attention support", "files": ["src/oumi/core/trainers/oumi_trainer.py", "src/oumi/models/layers/ring_attention.py", "src/oumi/models/layers/zigzag.py", "src/oumi/models/layers/zigzag_utils.py"], "area": "other", "area_votes": {"training": 1, "other": 3}, "body": "# Description\r\n\r\n\r\n\r\n- Deprecate experimental ring attention support\r\n- This is causing issues with the latest flash-attn / transformers version. Removing since unused.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-09T00:03:49Z"}
{"number": 1919, "title": "Add august webinar to the readme.", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nAdd the August webinar link to our README.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-08T16:02:59Z"}
{"number": 1918, "title": "Unpin `flash-attn` version", "files": ["configs/examples/grpo_verl_countdown/gcp_job.yaml", "configs/examples/grpo_verl_geometry3k/gcp_job.yaml", "configs/recipes/vision/phi3/sft/full/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/full/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/lora/gcp_job.yaml", "configs/recipes/vision/phi4/sft/full/gcp_job.yaml", "configs/recipes/vision/phi4/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/full/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/full/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/lora/gcp_job.yaml", "tests/scripts/e2e_tests_job.yaml"], "area": "configs", "area_votes": {"configs": 12}, "body": "# Description\r\n\r\nSince we upgraded the PyTorch version, the pinned flash-attn version 2.7.4.post1 doesn't have a prebuilt wheel and has to be built from source. Tested that the error we previously encountered about an unknown symbol doesn't show up after unpinning.\r\n\r\nTested with e2e tests.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1336\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-07T19:19:19Z"}
{"number": 1917, "title": "Modify document ingestion to read bytes", "files": ["src/oumi/core/synthesis/document_ingestion.py", "tests/unit/core/synthesis/test_document_ingestion.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-08-06T23:20:05Z"}
{"number": 1914, "title": "Add Conversation-Level Analysis", "files": ["src/oumi/core/analyze/__init__.py", "src/oumi/core/analyze/dataset_analyzer.py", "src/oumi/core/analyze/length_analyzer.py", "src/oumi/core/analyze/sample_analyzer.py", "src/oumi/core/configs/analyze_config.py", "tests/unit/core/analyze/test_dataset_analyzer.py", "tests/unit/core/configs/test_analyze_config.py", "tests/unit/core/test_length_analyzer.py", "tests/unit/core/test_registry.py"], "area": "evaluation", "area_votes": {"evaluation": 3, "configs": 1}, "body": "# Description\r\nThis PR adds dual-level conversation analysis - enabling both message-level and conversation-level analysis of datasets.\r\n\r\nMain changes: \r\n* SampleAnalyzer has a analyze_sample method (instead of analyze_message) which computes both message level and conversation level metrics and returns tuple[list[MessageAnalysisResult], ConversationAnalysisResult]\r\n* LengthAnalyzer implements SampleAnalyzer in a way that conversation level metrics are the aggregate of message level metrics for all of the metrics (char, word, sentence) except for token. For token, it tokenizes the conversation directly using the datast.tokenize\r\n* dataset is passed to the LengthAnalyzer so it can use it for tokenziation\r\n* we can remove the tokenizer in a follow up PR since we are going to use the dataset directly\r\n* All of the tests are updated accordingly\r\n* The check for sample_count has been moved to analyze_config\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1455\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-08T22:37:11Z"}
{"number": 1913, "title": "Update dataset planner to use dependency injection", "files": ["src/oumi/core/synthesis/dataset_planner.py", "tests/unit/core/synthesis/test_dataset_planner.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-08-06T22:41:19Z"}
{"number": 1905, "title": "[tiny] Pin exact omegaconf version", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nSee attached Linear issue for details. tl;dr installing oumi with uv often won't work because it is stricter than pip and won't download prerelease packages (i.e. omegaconf) with our current version specifications. Pinning an exact omegaconf version should fix this.\r\n\r\n## Related issues\r\n\r\nFixes OPE-1476\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-08-05T18:43:17Z"}
{"number": 1904, "title": "Update README.md with v0.3 release", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Adding v0.3 release to GH news\r\n", "merged_at": "2025-08-05T03:32:13Z"}
{"number": 1901, "title": "Temporarily disable python 3.13 support to unblock pip release", "files": [".github/workflows/install_test.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-04T23:17:24Z"}
{"number": 1900, "title": "[bug] use number of slurm visible gpus instead of hardcoded", "files": ["src/oumi/cli/distributed_run.py", "tests/unit/cli/test_cli_distributed_run.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-04T21:29:56Z"}
{"number": 1896, "title": "[tiny] remove deprecated param from quantization notebook", "files": ["notebooks/Oumi - Quantization Tutorial.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [tiny] remove deprecated param from quantization notebook\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-02T03:39:27Z"}
{"number": 1894, "title": "[Quantization] tutorial quantization (part 8)", "files": ["notebooks/Oumi - Quantization Tutorial.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nadd quantization tutorial, including quantization tool and inference demo with quantized models.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards: OPE-1425\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-01T21:45:44Z"}
{"number": 1892, "title": "[docs] Add quantization docs to index, minor cleanups", "files": ["docs/index.md", "docs/user_guides/quantization.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-31T23:16:16Z"}
{"number": 1891, "title": "[docs] Skip :noindex duplicate warnings", "files": ["docs/conf.py"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- When generating the docs, we get a lot of warnings that are not particularly useful and clutter the output\r\n- This PR adds a log filter to skip logging them.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-31T23:14:17Z"}
{"number": 1889, "title": "Add notebook for bulk inference", "files": ["notebooks/Oumi - Bulk Inference of LLM APIs.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1413\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-08-01T15:27:30Z"}
{"number": 1887, "title": "Judge API V2 | Lower inference temperatures", "files": ["configs/projects/judges/doc_qa/completeness.yaml", "configs/projects/judges/doc_qa/groundedness.yaml", "configs/projects/judges/doc_qa/relevance.yaml", "configs/projects/judges/generic/format_compliance.yaml", "configs/projects/judges/generic/instruction_following.yaml", "configs/projects/judges/generic/safety.yaml", "configs/projects/judges/generic/topic_adherence.yaml", "configs/projects/judges/generic/truthfulness.yaml"], "area": "configs", "area_votes": {"configs": 8}, "body": "# Description\r\n\r\n\r\n\r\nLower judgment inference temperatures to 0 \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1327\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-31T17:28:14Z"}
{"number": 1885, "title": "Fix pyNVML not supported error", "files": ["src/oumi/utils/device_utils.py", "tests/unit/utils/test_device_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\nSome devices cause pynvml to raise a not supported error, right now we log these as exceptions and return None, but instead we should log them with debug and exclude them from returning. This way we return everything we can without raising exceptions for something out of our control.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1877\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-30T23:56:46Z"}
{"number": 1880, "title": "Update hf vision dataset class to also load local datasets", "files": ["src/oumi/datasets/vision_language/huggingface.py", "tests/unit/datasets/test_huggingface_vision_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Update hf vision dataset class to also load local datasets\r\n- If provided a local path, load from disk using `_load_local_dataset`, else load remote dataset\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-30T15:57:35Z"}
{"number": 1879, "title": "[tiny] Enable rich logging by default", "files": ["src/oumi/utils/logging.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Enable rich logging by default\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-28T23:15:50Z"}
{"number": 1874, "title": "Update README.md to add news of adding Qwen 3 Recipe", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\nAdd a line in the News about the new Qwen 3 Recipe\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-25T22:47:23Z"}
{"number": 1873, "title": "Add Qwen3 235B inference configuration", "files": ["configs/recipes/qwen3/inference/235b_a22b_together_infer.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\nAdd Qwen3-235B-A22B-2507 inference config using Together.ai\r\n\r\nModel on Together.AI: Qwen/Qwen3-235B-A22B-Instruct-2507-tput\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-25T22:24:03Z"}
{"number": 1872, "title": "remove webinar ad, add webinar recording link", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nRemove links to July webinar signup page, and add link to YouTube recording", "merged_at": "2025-07-25T18:04:03Z"}
{"number": 1869, "title": "Update mlflow dependency to >=3.1.4", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "## Summary\r\n- Updates the mlflow dependency in pyproject.toml from `>=2.21.2,<2.22.0` to `>=3.1.4`\r\n- This is a minor dependency update to allow for newer versions of mlflow", "merged_at": "2025-07-25T15:23:39Z"}
{"number": 1859, "title": "[tiny] Install torchdata from source if python >= 3.13", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- torchdata does not provide wheels for python >= 3.13.\r\n- Proceed as usual with python < 3.13, but install from source if python >= 3.13\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-23T17:37:27Z"}
{"number": 1855, "title": "[tiny] update type annotations for python3.9 support", "files": ["src/oumi/utils/verl_model_merger.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- New syntax (using | ) is only support in python 3.10+\r\n- Switch to using `Union` to unblock 3.9 support\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-23T04:18:46Z"}
{"number": 1854, "title": "[Quantization] add bnb quantizer (part 5)", "files": ["src/oumi/builders/__init__.py", "src/oumi/builders/quantizers.py", "src/oumi/quantize/__init__.py", "src/oumi/quantize/awq_quantizer.py", "src/oumi/quantize/bnb_quantizer.py", "src/oumi/quantize/constants.py"], "area": "other", "area_votes": {"other": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\nadd more quantizers: gguf and bnb\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards: OPE-1425\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-24T17:45:28Z"}
{"number": 1853, "title": "Add github workflow with install test matrix", "files": [".github/workflows/gpu_install_test.yaml", ".github/workflows/install_test.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Add a workflow to test oumi installation for various environments (Python, cuda versions, torch)\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-24T00:43:50Z"}
{"number": 1851, "title": "Add  Dataset Analyzer", "files": ["src/oumi/core/analyze/__init__.py", "src/oumi/core/analyze/dataset_analyzer.py", "src/oumi/core/analyze/sample_analyzer.py", "tests/unit/core/analyze/test_dataset_analyzer.py"], "area": "evaluation", "area_votes": {"evaluation": 2}, "body": "# Description\r\nAdds dataset analysis functionality with unit tests for the DatasetAnalyzer class and compute_sample_level_analysis utility function\r\n\r\nNew Files\r\n* src/oumi/core/analyze/dataset_analyzer.py: High-level orchestrator for dataset analysis\r\n* tests/unit/core/analyze/test_dataset_analyzer.py: tests for DatasetAnalyzer\r\nEnhanced Files\r\n* src/oumi/utils/analysis_utils.py: Added compute_sample_level_analysis function for message-level analysis\r\n* tests/unit/utils/test_analysis_utils.py: Added unit tests\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1407\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-24T21:15:43Z"}
{"number": 1850, "title": "[Quantization] CLI integration (part 4)", "files": ["src/oumi/__init__.py", "src/oumi/cli/alias.py", "src/oumi/cli/main.py", "src/oumi/cli/quantize.py", "src/oumi/quantize/__init__.py"], "area": "cli", "area_votes": {"cli": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\ncli integration\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards: OPE-1425\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-23T00:26:11Z"}
{"number": 1849, "title": "Adjust settings.json (type checking & auto-format)", "files": [".vscode/settings.json"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n*Changes*\r\n- Avoid setting the type checker to accomodate users using cursor\r\n- Set formatting by default\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-23T00:07:15Z"}
{"number": 1847, "title": "[vision] Add vision dpo dataset class", "files": ["data/dataset_examples/vision_language_dpo_format.jsonl", "src/oumi/core/datasets/__init__.py", "src/oumi/core/datasets/base_dpo_dataset.py", "src/oumi/core/datasets/vision_language_dpo_dataset.py", "src/oumi/datasets/__init__.py", "src/oumi/datasets/vision_language/__init__.py", "src/oumi/datasets/vision_language/vision_dpo_jsonlines.py", "tests/unit/datasets/test_vision_dpo_jsonlines_dataset.py"], "area": "data", "area_votes": {"data": 4}, "body": "# Description\r\n\r\n\r\n\r\n**Changes**\r\n- Add a base vision DPO dataset class, similar to the language DPO dataset class\r\n  - Note that both DPO datasets (text, and now vision-language) are still experimental, and do not use the `Conversation` format. \r\n- Add a first vision DPO dataset, to consume `jsonl` files.  \r\n  - Add an example data file\r\n- Fix a type annotation issue in the DPO text dataset  \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-22T17:09:30Z"}
{"number": 1844, "title": "Judge API V2 | Documentation & Notebook", "files": ["README.md", "docs/get_started/tutorials.md", "docs/index.md", "docs/user_guides/evaluate/generative_benchmarks.md", "docs/user_guides/judge/built_in_judge.md", "docs/user_guides/judge/built_in_judges.md", "docs/user_guides/judge/cli_usage.md", "docs/user_guides/judge/custom_infer.md", "docs/user_guides/judge/custom_prompt.md", "docs/user_guides/judge/judge.md", "docs/user_guides/judge/judge_config.md", "notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Simple Judge.ipynb", "src/oumi/cli/main.py", "tests/unit/cli/test_cli_main.py"], "area": "docs", "area_votes": {"docs": 14, "cli": 1}, "body": "# Description\r\n\r\n\r\n\r\nCurrent changes:\r\n- Updated Judge API documentation from Judge V1 to Judge V2\r\n- Dropped 2 notebooks (these are for Judge V1)\r\n- Added a notebook for Judge V2 (Simple Judge) and updated references\r\n- Removed experimental reg key (`OUMI_EXPERIMENTAL_JUDGE_V2`). Judge V2 can now be used without this\r\n\r\nFuture changes ToDo:\r\n- Rename Judge V2 to Judge\r\n- Delete legacy judge code and configs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1327\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-23T16:06:34Z"}
{"number": 1843, "title": "[Quantization] add awq quantizer (part 3)", "files": ["src/oumi/quantize/__init__.py", "src/oumi/quantize/awq_quantizer.py", "src/oumi/quantize/base.py", "src/oumi/quantize/constants.py", "src/oumi/quantize/utils.py"], "area": "other", "area_votes": {"other": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n add awq quantizer\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards: OPE-1425\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-22T00:06:42Z"}
{"number": 1842, "title": "Fix a pyright error for synthesis", "files": ["src/oumi/core/synthesis/document_ingestion.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdd a pyright exemption for an optional synthesis import.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-21T21:25:09Z"}
{"number": 1841, "title": "[bug] Fix a bug where inference engines would wait for one politeness policy cycle after all requests were finished.", "files": ["src/oumi/inference/adaptive_concurrency_controller.py", "src/oumi/inference/adaptive_semaphore.py", "src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_adaptive_concurrency_controller.py", "tests/unit/inference/test_adaptive_semaphore.py", "tests/unit/inference/test_llama_cpp_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\nThis PR updates our concurrency logic to use a \"Polite\" semaphore. From a producer-consumer standpoint, the semaphore is producing \"start times\", which are consumed on subscription. The semaphore will then force the holder to wait if necessary given the politeness policy. When the subscriber releases the semaphore, their release time is appended to the queue.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-23T21:33:52Z"}
{"number": 1839, "title": "[Quantization] adding base class (part 2)", "files": ["src/oumi/core/configs/__init__.py", "src/oumi/quantize/base.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nadding base class\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards: OPE-1425\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-21T22:42:59Z"}
{"number": 1835, "title": "Update docs with information on adaptive inference", "files": ["docs/user_guides/infer/configuration.md", "docs/user_guides/infer/infer.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\n\n\n\nAdaptive inference changes have been merged in, this PR updates the docs to help explain to users what it is and how it works. It also provides updates on our \"save and resume\" functionality in inference.\n\n\n## Related issues\n\n\n\n\nFixes OPE-1411\n\n\n\n## Before submitting\n\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-07-21T22:57:02Z"}
{"number": 1832, "title": "docs: add troubleshooting note for Conda Terms of Service error in de\u2026", "files": ["docs/development/dev_setup.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\nThis commit adds a troubleshooting note to the development setup documentation to help users resolve the \"CondaToSNonInteractiveError: Terms of Service have not been accepted for the following channels...\" error.\r\nThis issue can block new users from creating or activating the necessary Conda environment for Oumi development.\r\n\r\nThe note provides copy-pasteable commands for accepting the Terms of Service or removing problematic channels, addressing possible installation blockers that may not be familiar to all contributors. This should help speed up onboarding and reduce confusion during the initial installation process.\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-24T18:27:08Z"}
{"number": 1831, "title": "Refine sentence to clarify that Oumi cannot be installed on Intel Macs", "files": ["docs/get_started/installation.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\nUpdating the [#Requirements](https://oumi.ai/docs/en/latest/get_started/installation.html#requirements) section on the Installation page.\r\n\r\nThe original sentence used the word \"there,\" which could be ambiguous about the location where Oumi cannot be installed. This change replaces \"there\" with \"on those machines\" to explicitly specify that installation is not possible on Intel Macs due to PyTorch dropping support for that platform. This improves clarity for readers and reduces potential confusion.\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-16T23:15:53Z"}
{"number": 1828, "title": "Judge API V2 | Built-In Judges", "files": ["configs/projects/judges/doc_qa/completeness.yaml", "configs/projects/judges/doc_qa/groundedness.yaml", "configs/projects/judges/doc_qa/relevance.yaml", "configs/projects/judges/generic/format_compliance.yaml", "configs/projects/judges/generic/instruction_following.yaml", "configs/projects/judges/generic/topic_adherence.yaml", "configs/projects/judges/generic/truthfulness.yaml", "configs/projects/judges/qa/relevance.yaml", "src/oumi/core/configs/judge_config_v2.py", "tests/integration/cli/test_judge_v2_e2e.py"], "area": "configs", "area_votes": {"configs": 9}, "body": "# Description\r\n\r\n\r\n\r\nIntroducing 4 generic built-in judges (applicable to any task type, i.e. their inputs are `user request` and `model response`) and 3 built-in judges specifically for Doc Q&A (our signature use case) with inputs: `context`, `question`, and `answer`.\r\n\r\n**_Generic Judges_**\r\n- Format Compliance (JSON/XML/HTML/.., keys required, custom constraints)\r\n- Instruction Following\r\n- Topic Adherence\r\n- Truthfulness \r\n\r\n**_DocQA Judges_** \r\n- Groundedness\r\n- Relevance\r\n- Completeness\r\n\r\n_Notes_\r\n- **_Why did we select these 4 generic judges?_** These should cover the most popular task types identified by [PromptEvals](https://arxiv.org/pdf/2504.14738). Note that there are a few more use cases that are popular, but are either hard to generalize (e.g., stylistic constraints), or should be indirectly covered by these 3 (length constraints and language adherence can be covered by the \"Format Compliance\" judge).\r\n- **_Why not use generic judges for the 3 DocQA use cases?_** We potentially could, but I strongly believe these would be very inefficient, since we could not explicitly define and refer to the \"context\". Note that the assesment of these 3 judges can be very different if no context is explicitly considered. In addition, some basic testing I did for generalizing these (\"if there is a context in the request, then only consider this for ...\") did NOT prove to be very efficient. Since DocQA is one of our signature use cases and these specific 3 dimensions are NOT well-captured by generic judges, I opted for specialized ones (@oelachqar let me know if this is OK).\r\n- **_How well do these judges work? Have we done extensive testing?_** Nope. These are just a starting point with minimum investment and testing from my side (I spend a day on these). If (i) these are deemed high priority and (ii) @brragorn has bandwidth to invest in refining/improving these, we can iterate and incrementally make them more efficient and powerful (without changing the inputs/outputs, which would break users). \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1327\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-17T23:20:48Z"}
{"number": 1826, "title": "[4] Add support for Transform Attributes", "files": ["src/oumi/core/synthesis/attribute_transformation.py", "tests/unit/core/synthesis/test_attribute_transformation.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\n\n\n\nTo be able to output datasets in a specific format (i.e. training, eval, etc.), we need to be able to support \"Transformed\" Attributes, which are effectively attributes of various types (str, list[str], dict[str, str], and Conversation) which can be \"filled in\" using the other attributes in the sample.\n\nThis PR adds the class necessary to handle these attributes, but does not integrate it into the synthesis flow.\n\n\n## Related issues\n\n\n\n\nFixes OPE-1385\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-07-23T15:21:44Z"}
{"number": 1823, "title": "[0] Add attribute formatter for samples", "files": ["src/oumi/core/synthesis/attribute_formatter.py", "tests/unit/core/synthesis/test_attribute_formatter.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\nThe AttributeFormatter allows us to apply more advanced formatting logic using attribute values from samples in a dataset (think columns).\r\n\r\nFor the below examples, `attribute_id` is basically `column_name` in a CSV-style dataset.\r\n\r\nSamples are formatted with varying logic:\r\n{attribute_id} for the value of a given column\r\n{attribute_id} for the name of a permutable attribute column\r\n{attribute_id.description} for the description of a permutable attribute column\r\n{attribute_id.value} for the specific value of a permutable attribute for that sample\r\n{attribute_id.value.description} for the description of that value\r\n\r\nTo properly format strings of this type with samples, we need to construct the necessary information into objects (_AttributeInfo and _AttributeValueInfo), which will allow `resolve_placeholders` to do the rest of the work for us.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1355\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-16T19:35:29Z"}
{"number": 1820, "title": "Followup for comments from #1817", "files": ["configs/recipes/qwen3/evaluation/0.6b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/1.7b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/14b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/4b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/8b_gcp_job.yaml", "configs/recipes/qwen3/sft/0.6b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/1.7b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/4b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/8b_full/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 9}, "body": "# Description\r\n\r\n\r\n\r\nSFT: \r\n- Removed unnecessary `HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download Qwen/Qwen3-4B` (for model sizes 8b and below)\r\n- Increased disk size for Qwen3 8b \r\n\r\nEvaluation:\r\n- Decrease number of GPU requested from 4 to 1 for model sizes 14b and below\r\n- Tested 14b to ensure it runs\r\n\r\nAddresses comments from #1817 \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1346\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-14T18:54:12Z"}
{"number": 1819, "title": "misc. cleanup", "files": ["configs/projects/aya/sft/train.yaml", "configs/recipes/llama3_2/inference/1b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/1b_vllm_infer.yaml", "configs/recipes/llama3_2/inference/3b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/3b_vllm_infer.yaml", "configs/recipes/llama3_3/inference/70b_vllm_infer.yaml", "configs/recipes/phi3/sft/lora_train.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_rvllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml", "configs/recipes/vision/llava_7b/inference/vllm_infer.yaml", "configs/recipes/vision/phi3/inference/vllm_infer.yaml", "configs/recipes/vision/phi4/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_5_vl_3b/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/sglang_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/vllm_infer.yaml", "configs/recipes/vision/smolvlm/inference/vllm_infer.yaml"], "area": "configs", "area_votes": {"configs": 17}, "body": "# Description\r\n\r\n- Remove batch_size from inference configs that don't use it (anything but native)\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-07-13T23:04:59Z"}
{"number": 1818, "title": "Add `slurm_init.sh` script and quickstart Slurm job", "files": ["configs/examples/grpo_verl_countdown/slurm_job.yaml", "configs/examples/grpo_verl_gsm8k/slurm_job.yaml", "configs/examples/misc/slurm_init.sh", "configs/examples/misc/slurm_ray_init.sh", "configs/recipes/smollm/sft/135m/gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_train.yaml", "configs/recipes/smollm/sft/135m/slurm_job.yaml", "configs/recipes/smollm/sft/135m/train.yaml", "src/oumi/launcher/clients/slurm_client.py"], "area": "configs", "area_votes": {"configs": 8, "launcher": 1}, "body": "# Description\r\n\r\nThe script is similar to `sky_init.sh`. Example output of slurm init script:\r\n```\r\n------------------------------------------\r\nSlurm job ID: 801\r\nSlurm job name: grpo-verl-gsm8k\r\nSlurm task PID: 1832035\r\nJob start time: Sat Jul 12 01:54:26 UTC 2025\r\nSlurm job nodelist: oumi-compute[002,001]\r\nCurrent dir: /home/wizeng/oumi_launcher/20250711_215400585499\r\n\r\nHead node: oumi-compute002\r\nMaster address: 172.26.135.6\r\nNumber of nodes: 2\r\nNumber of tasks per node: 1\r\nNumber of CPUs per node: 8\r\nNumber of GPUs per node: 4\r\nCUDA_VISIBLE_DEVICES: 0,1,2,3\r\n------------------------------------------\r\n```\r\n\r\n## Related issues\r\n\r\nTowards OPE-1373\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-07-13T23:01:50Z"}
{"number": 1817, "title": "Added more Qwen3 recipes for training, evaluation, and inference", "files": ["configs/recipes/qwen3/README.md", "configs/recipes/qwen3/evaluation/0.6b_eval.yaml", "configs/recipes/qwen3/evaluation/0.6b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/1.7b_eval.yaml", "configs/recipes/qwen3/evaluation/1.7b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/14b_eval.yaml", "configs/recipes/qwen3/evaluation/14b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/4b_eval.yaml", "configs/recipes/qwen3/evaluation/4b_gcp_job.yaml", "configs/recipes/qwen3/evaluation/8b_eval.yaml", "configs/recipes/qwen3/evaluation/8b_gcp_job.yaml", "configs/recipes/qwen3/inference/0.6b_infer.yaml", "configs/recipes/qwen3/inference/1.7b_infer.yaml", "configs/recipes/qwen3/inference/14b_infer.yaml", "configs/recipes/qwen3/inference/4b_infer.yaml", "configs/recipes/qwen3/inference/8b_infer.yaml", "configs/recipes/qwen3/sft/0.6b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/0.6b_full/train.yaml", "configs/recipes/qwen3/sft/1.7b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/1.7b_full/train.yaml", "configs/recipes/qwen3/sft/14b_lora/gcp_job.yaml", "configs/recipes/qwen3/sft/14b_lora/train.yaml", "configs/recipes/qwen3/sft/4b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/4b_full/train.yaml", "configs/recipes/qwen3/sft/8b_full/gcp_job.yaml", "configs/recipes/qwen3/sft/8b_full/train.yaml"], "area": "configs", "area_votes": {"docs": 1, "configs": 25}, "body": "# Description\r\n\r\n\r\n\r\nAdded more Qwen3 recipes for training, evaluation, and inference. I have tested all SFT configs for a few training steps to make sure they don't OOM. For inference and eval configs, I tested 3 out of 5 configs (usually 0.6b, 4b, 14b) to see that they run as expected. To test each config, I launched a new cluster and ran training for a few steps eg:\r\n\r\n```bash\r\noumi launch up -c configs/recipes/qwen3/sft/4b_full/gcp_job.yaml --cluster qwen3-4b-full\r\noumi launch down --cluster qwen3-4b-full\r\n```\r\n\r\nBefore checking in the PR, I added back `oumi://` prefixes.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1346\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-12T01:48:54Z"}
{"number": 1815, "title": "feat: add analyze config classes and exports", "files": ["src/oumi/core/configs/__init__.py", "src/oumi/core/configs/analyze_config.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\nThis is the first PR in a series of smaller, reviewable pull requests that break down the larger analyze functionality. The complete changes can be found in the full feature branch: [main...ryan-arman-anlyze_v0](https://github.com/oumi-ai/oumi/compare/main...ryan-arman-anlyze_v0)\r\n\r\nThis PR introduces the foundational configuration classes needed for the dataset analysis functionality.\r\n\r\nChanges:\r\nAdded: src/oumi/core/configs/analyze_config.py\r\n- DatasetAnalyzeConfig - Main configuration class for dataset analysis\r\n- SampleAnalyzeConfig - Configuration for individual analyzer plugins\r\n\r\nUpdated: src/oumi/core/configs/__init__.py\r\n- Added imports for the new config classes\r\n- Added exports to __all__ list for public API\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1407\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-12T00:53:05Z"}
{"number": 1812, "title": "Add verl Countdown Slurm config", "files": ["configs/examples/grpo_verl_countdown/gcp_job.yaml", "configs/examples/grpo_verl_countdown/slurm_job.yaml", "configs/examples/grpo_verl_countdown/train.yaml", "configs/examples/grpo_verl_geometry3k/gcp_job.yaml", "configs/examples/grpo_verl_geometry3k/train.yaml", "configs/examples/grpo_verl_gsm8k/slurm_job.yaml", "configs/recipes/vision/molmo/grpo/train.yaml"], "area": "configs", "area_votes": {"configs": 7}, "body": "# Description\r\n\r\n- Add Slurm Countdown for config that mirrors the GCP config\r\n- Swap from deprecated `log_prob_micro_batch_size` param to `log_prob_micro_batch_size_per_gpu`. With the former, there's a bug where if `log_prob_micro_batch_size` is less than the number of GPUs, training will hang until it OOMs.\r\n\r\n## Related issues\r\n\r\nFixes OPE-1371\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-07-11T22:11:52Z"}
{"number": 1810, "title": "Add experimental option to use the rich logging handler", "files": ["src/oumi/utils/logging.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n#### Changes\r\n- Our CLI needs a refresh visually. One of the lowest effort improvements is to enable the rich logging handler\r\n- This is currently experimental, and disabled by default\r\n- Users can enable it by setting `export OUMI_ENABLE_RICH_LOGGING=\"1\"`\r\n- When oumi's log level is set to `DEBUG`, we additionally log a lot more information (more detailed stack traces, locals, etc)\r\n\r\n#### OUMI_ENABLE_RICH_LOGGING=\"1\"\r\n\"image\"\r\n\r\n\r\n#### OUMI_ENABLE_RICH_LOGGING=\"0\"\r\n\"image\"\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-11T17:11:41Z"}
{"number": 1809, "title": "[tiny] Update train config comment header", "files": ["configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "configs/examples/grpo_tldr/train.yaml", "configs/examples/grpo_verl_countdown/train.yaml", "configs/examples/grpo_verl_geometry3k/train.yaml", "configs/examples/letter_counting/grpo/train.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/chatqa/chatqa_stage1_train.yaml", "configs/projects/chatqa/chatqa_stage2_train.yaml", "configs/projects/dcvlr/starter_kit/molmo-d-train-openr1.yaml", "configs/projects/dcvlr/starter_kit/molmo-o-train-openr1.yaml", "configs/projects/dcvlr/starter_kit/qwenvl-openr1.yaml", "configs/projects/halloumi/8b_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_train.yaml", "configs/recipes/falcon_e/dpo/falcon_e_1b_instruct/dpo.yaml", "configs/recipes/falcon_e/sft/falcon_e_1b/full_train.yaml", "configs/recipes/falcon_e/sft/falcon_e_1b_instruct/full_train.yaml", "configs/recipes/falcon_e/sft/falcon_e_3b/full_train.yaml", "configs/recipes/falcon_e/sft/falcon_e_3b_instruct/full_train.yaml", "configs/recipes/falcon_h1/dpo/falcon_h1_0_5b/qlora_dpo.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_0_5b/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b_deep/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_34b/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_3b/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_7b/full_train.yaml", "configs/recipes/gpt2/pretraining/mac_train.yaml", "configs/recipes/gpt2/pretraining/train.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml", "configs/recipes/phi3/dpo/mac_train.yaml", "configs/recipes/phi3/dpo/nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/nvidia_80g_train.yaml", "configs/recipes/phi3/dpo/train.yaml", "configs/recipes/phi3/sft/lora_train.yaml", "configs/recipes/phi3/sft/mac_lora_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/full_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_train.yaml", "configs/recipes/qwen3/sft/30b_a3b_lora/train.yaml", "configs/recipes/qwen3/sft/32b_lora/train.yaml", "configs/recipes/qwq/sft/full_train.yaml", "configs/recipes/qwq/sft/lora_train.yaml", "configs/recipes/qwq/sft/qlora_train.yaml", "configs/recipes/smollm/sft/135m/quickstart_train.yaml", "configs/recipes/smollm/sft/135m/train.yaml", "configs/recipes/vision/internvl3/sft/full/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/train.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/molmo/grpo/train.yaml", "configs/recipes/vision/molmo/sft/molmo_d_full/train.yaml", "configs/recipes/vision/molmo/sft/molmo_o_full/train.yaml", "configs/recipes/vision/phi3/sft/full/completions_only_train.yaml", "configs/recipes/vision/phi3/sft/full/train.yaml", "configs/recipes/vision/phi3/sft/lora/train.yaml", "configs/recipes/vision/phi4/sft/full/train.yaml", "configs/recipes/vision/phi4/sft/lora/train.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/full/train.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/lora/train.yaml", "configs/recipes/vision/qwen2_5_vl_7b/sft/full/train.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/full/train.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/lora/train.yaml", "configs/recipes/vision/smolvlm/sft/full/train.yaml", "configs/recipes/vision/smolvlm/sft/lora/train.yaml", "src/experimental/configs/projects/zephyr/sft/full_train.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_train.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_nvidia_24g_train.yaml"], "area": "configs", "area_votes": {"configs": 94, "other": 3}, "body": "# Description\r\n\r\nSince we have many types of training configs now, this PR makes the comment more generic. This PR is just a large find+replace.\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-07-11T17:01:46Z"}
{"number": 1806, "title": "Add support for document ingestion during data synthesis", "files": ["src/oumi/core/synthesis/planner.py", "tests/unit/core/synthesis/test_planner.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\n\n\n\nIntegrates document ingestion into the dataset planning process, allowing users to specify attributes in their dataset plan which will be filled with documents and/or document segments, which can then be usable during generation.\n\n\n## Related issues\n\n\n\n\nFixes OPE-1297\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-07-11T22:39:40Z"}
{"number": 1805, "title": "Judge API V2 | Enable prompt variable replacement by YAML", "files": ["src/oumi/core/configs/params/judge_params.py", "src/oumi/judges_v2/base_judge.py", "src/oumi/judges_v2/simple_judge.py", "src/oumi/utils/placeholders.py", "tests/e2e/test_simple_judge.py", "tests/unit/core/configs/params/test_judge_params.py", "tests/unit/judges/test_base_judge.py"], "area": "other", "area_votes": {"configs": 1, "other": 3}, "body": "# Description\r\n\r\n\r\n\r\nAllow placeholders / variables (format: `{variable}`) in prompts (both `prompt_template` and `system_instruction` fields) to be replaced by the `template_variables` dict that is defined in YAML.\r\nWhy we need this? See [here](https://oumi-ai.slack.com/archives/C081RNZUBCZ/p1752149810332809?thread_ts=1752148996.735389&cid=C081RNZUBCZ)\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1327\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-11T16:37:02Z"}
{"number": 1804, "title": "Update mflow support in oumi trainer", "files": ["src/oumi/core/trainers/oumi_trainer.py", "tests/unit/core/trainers/test_oumi_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Update mflow support in the oumi trainer:\r\n  - Log metrics\r\n  - Re-use existing run if it exists. if not, create and manage a new run\r\n \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-10T15:15:57Z"}
{"number": 1802, "title": "Added utm_source parameters", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nAdded UTM source parameters to webinar Lu.ma links. This helps us track which are the most successful advertising mediums.", "merged_at": "2025-07-09T17:29:42Z"}
{"number": 1801, "title": "qwen 2.5 config updates", "files": ["configs/recipes/qwen2_5/sft/3b_full/train.yaml", "configs/recipes/qwen2_5/sft/7b_full/train.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/full/train.yaml", "configs/recipes/vision/qwen2_5_vl_7b/sft/full/train.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\nQwen 2.5 and Qwen 2.5 VL config updates: faster, more efficient training, supports long context. Optimized for single node, 8xH100 80GB GPU training.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-11T22:49:57Z"}
{"number": 1800, "title": "Webinar announcement and other news", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nNotice for our July 24 webinar at top of README\r\n\r\nUpdated news items for June-July\r\n", "merged_at": "2025-07-09T04:56:21Z"}
{"number": 1797, "title": "Add support for Example Sources in Synthesis", "files": ["src/oumi/core/synthesis/planner.py", "tests/unit/core/synthesis/test_planner.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\n\n\n\nIntegrates ExampleSources into the dataset planner, allowing the planner to enumerate through them when creating the dataset.\n\n\n## Related issues\n\n\n\n\nFixes OPE-1357\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-07-08T22:58:12Z"}
{"number": 1795, "title": "Fixed issue with final conversations not consistently being saved", "files": ["src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/unit/inference/test_base_inference_engine.py"], "area": "inference", "area_votes": {"inference": 5}, "body": "# Description\n\n\n\nConversations were not being saved in the core public `infer()` method, resulting in some scenarios where only some of the conversations were being written in the case where inference was stopped and restarted, rather than ALL conversations.\n\n\n## Related issues\n\n\n\n\nTowards OPE-1307\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-07-02T21:44:46Z"}
{"number": 1792, "title": "Fixed various tutorial notebooks", "files": ["notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "src/oumi/core/evaluation/backends/alpaca_eval.py"], "area": "docs", "area_votes": {"docs": 3, "evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\n1. vLLM inference notebook: Adjust the VLLMInferenceEngine config such that it can load on 4 A100-40GB GPUs\r\n2. Evaluation with Oumi notebook: Load models in torch_dtype_str=\"bfloat16\"\r\n3. alpaca_evals.py: Remove import alpaca.evaluate. evaluate is actually a function in alpaca_eval/main.py, not a submodule\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\nFixes OPE-1348\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-07-10T23:47:45Z"}
{"number": 1789, "title": "Update our contributing guidelines.", "files": ["CONTRIBUTING.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nMinor update to CONTRIBUTING.md\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-30T23:13:24Z"}
{"number": 1788, "title": "Removed collator in finetuning tutorial notebook", "files": ["notebooks/Oumi - Finetuning Tutorial.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nRemoved collator in finetuning tutorial notebook as temp fix for missing 'labels' ValueError bug. Intended as a temporary fix for [OPE-1342](https://linear.app/oumi/issue/OPE-1342).\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-30T16:13:43Z"}
{"number": 1782, "title": "Judge API V2 | Fix judge config from repo path", "files": ["src/oumi/core/configs/judge_config_v2.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\nMinor Fix: judge config being loaded from repo path\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1327\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-26T11:17:46Z"}
{"number": 1781, "title": "Update launch.md", "files": ["docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Fix subversion handling in launch.md by adding required quotes\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-25T19:12:39Z"}
{"number": 1780, "title": "Add adaptive semaphore to enable future adaptive throughput scenarios", "files": ["src/oumi/inference/adaptive_semaphore.py", "tests/unit/inference/test_adaptive_semaphore.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\n\n\n\nBounded Semaphores are a lock mechanism allowing a finite amount of \"permits\" to allow so many \"waiters\" to access a given resource.\n\nTraditionally, bounded semaphores are a fixed size - once initialized, they keep that capacity.\n\nThis PR introduced an Adaptive Semaphore - one which can change its size allocation. For size decreases, nothing special really happens - the existing permits stay valid, but no new permits are granted.\n\nFor size increases, the semaphore will immediately grant  waiters access to the resource.\n\n\n## Related issues\n\n\n\n\nTowards OPE-1307\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-06-25T22:21:05Z"}
{"number": 1779, "title": "[tiny] Remove `use_liger` argument", "files": ["src/oumi/core/configs/training_config.py", "tests/e2e/test_train_e2e.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\nThis parameter for SFTConfig is now deprecated. This PR also adds a new test case to integration test cases so that a Liger kernel model is covered.\r\n\r\n## Related issues\r\n\r\nFixes OPE-1117\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-06-21T00:55:40Z"}
{"number": 1778, "title": "[tiny] Fix gradient checkpointing for Oumi trainer", "files": ["src/oumi/core/trainers/oumi_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\nThe current code fails for PEFT models, which also expose a gradient_checkpointing_enable() function. This PR makes the check less strict.\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-06-21T00:11:39Z"}
{"number": 1777, "title": "Pin the version of lm_eval to prevent a breaking change in the 4.9 release", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\nLM Harness released a new version today causing our CPU tests to regress. Pinning to the previous version until we have bandwidth to upgrade.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-20T08:20:14Z"}
{"number": 1775, "title": "[tiny] Pin flash-attn version", "files": ["configs/examples/grpo_verl_countdown/gcp_job.yaml", "configs/examples/grpo_verl_geometry3k/gcp_job.yaml", "configs/recipes/vision/phi3/sft/full/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/full/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/lora/gcp_job.yaml", "configs/recipes/vision/phi4/sft/full/gcp_job.yaml", "configs/recipes/vision/phi4/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/full/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/full/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/lora/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 12}, "body": "# Description\r\n\r\nThe latest version, 0.8.0.post2, results in an import error on our GCP jobs. Tested that this version pinning fixes the issue.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1336\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-06-19T17:08:08Z"}
{"number": 1774, "title": "Upgrade accelerate and peft", "files": ["pyproject.toml", "tests/e2e/test_train_e2e.py", "tests/scripts/e2e_tests_job.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n- Upgrade accelerate and peft to latest minor versions\r\n- Fix integration test script by pinning flash-attn version (see OPE-1336)\r\n- Update integration test script to remove redundant trl GRPO test and reflect correct machine type for the other\r\n\r\nVerified that integration tests pass.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1337\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-06-18T23:41:06Z"}
{"number": 1772, "title": "[bug] safetensors v0.6.0rc0 is causing a regression, prevent upgrading", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- safetensors v0.6.0rc0 is causing a regression, prevent upgrading\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-17T20:36:17Z"}
{"number": 1764, "title": "Additional news items", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2025-06-16T17:02:10Z"}
{"number": 1763, "title": "Updates to Oumi news for Falcon releases", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\nAdding to README relevant news from the past two months", "merged_at": "2025-06-16T16:04:14Z"}
{"number": 1761, "title": "Automatically tail SkyPilot logs", "files": ["src/oumi/cli/launch.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\nIf the cloud is a SkyPilot cloud and we don't run in detach mode, replace the default \"Running job\" polling message with tailing the SkyPilot logs. This is in line with SkyPilot's default behavior. IMO this doesn't need to be configurable as I've never run a job and not wanted to see the logs yet.\r\n\r\nManually tested.\r\n\r\n## Related issues\r\n\r\nFixes OPE-322\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2025-06-13T21:35:41Z"}
{"number": 1760, "title": "Enable vLLM for trl GRPO jobs", "files": ["configs/examples/grpo_tldr/train.yaml", "configs/examples/letter_counting/grpo/gcp_job.yaml", "configs/examples/letter_counting/grpo/train.yaml", "docs/user_guides/train/configuration.md", "src/oumi/core/configs/params/grpo_params.py"], "area": "configs", "area_votes": {"configs": 4, "docs": 1}, "body": "# Description\r\n\r\n- Added `vllm_mode` param to GRPOParams which controls how to integrate with vLLM.\r\n- Delete deprecated `vllm_device` param (unused in our configs).\r\n- Enabled vLLM for both trl GRPO jobs. Note they don't use the newly added param as it hasn't been pushed to the PyPI yet.\r\n\r\nTested that the jobs work. The tldr sample job training time went from 2min to 30 seconds.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1107\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-06-13T22:54:22Z"}
{"number": 1757, "title": "Judge API V2 | minor nit", "files": ["src/oumi/judges_v2/base_judge.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\nNit\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-12T18:35:28Z"}
{"number": 1756, "title": "Update infer.md to fix a broken link", "files": ["docs/user_guides/infer/infer.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "the link goes to https://github.com/oumi-ai/oumi/blob/main/%3CGitHub%3E%20notebooks/Oumi%20-%20Using%20vLLM%20Engine%20for%20Inference.ipynb\r\n\r\nwhich is broken, it should go to https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20Using%20vLLM%20Engine%20for%20Inference.ipynb\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-12T17:11:14Z"}
{"number": 1755, "title": "Update `oumi distributed torchrun` to fallback to `oumi train -c cfg.yaml ....` on a single-node with 1 GPU", "files": ["src/oumi/cli/distributed_run.py", "tests/unit/cli/test_cli_distributed_run.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Tested `oumi distributed torchrun -m oumi train  -c configs/recipes/llama3_2/sft/1b_full/train.yaml ` on `A100:1`\r\n-- Configs with such special cases  should be updated **after** the next Oumi PyPI package is shipped.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1315\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-11T21:31:39Z"}
{"number": 1754, "title": "add DCVLR logo to readme", "files": ["configs/projects/dcvlr/README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nadd DCVLR logo to readme\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-12T16:29:42Z"}
{"number": 1753, "title": "[bugfix] Allow prerelease when building docker image", "files": ["Dockerfile"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- We need to allow prerelease as we rely on an experimental version of omegaconf\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-10T22:31:42Z"}
{"number": 1752, "title": "Update link to Oumi banner image in README", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\"Screenshot\r\n\r\nOur banner currently doesn't display on PyPI. I changed the link to an absolute URL, which I've seen other repos use.\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-06-10T22:34:30Z"}
{"number": 1751, "title": "docs: add a badge and link to the social network Twitter", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-11T16:05:59Z"}
{"number": 1750, "title": "Dcvlr", "files": ["configs/projects/dcvlr/README.md", "configs/projects/dcvlr/starter_kit/README.md", "configs/projects/dcvlr/starter_kit/evaluate.sh", "configs/projects/dcvlr/starter_kit/molmo-d-train-openr1.yaml", "configs/projects/dcvlr/starter_kit/molmo-o-train-openr1.yaml", "configs/projects/dcvlr/starter_kit/qwenvl-openr1.yaml", "configs/recipes/vision/qwen2_5_vl_7b/sft/full/train.yaml"], "area": "configs", "area_votes": {"docs": 2, "configs": 5}, "body": "# Description\r\n\r\n\r\n\r\n\r\nDCVLR Starter Kit\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-10T18:25:22Z"}
{"number": 1748, "title": "[docs] update dcvlr readme", "files": ["configs/projects/dcvlr/README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [docs] update dcvlr readme\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-10T17:23:47Z"}
{"number": 1747, "title": "[bug] fix rank/local rank parsing for docker env", "files": ["src/oumi/core/distributed.py", "tests/unit/core/test_distributed.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Some environments set `RANK` / `LOCAL_RANK` to \"\", or \"-1\". Update `get_distributed_info` to treat those values as `0`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-10T04:05:45Z"}
{"number": 1743, "title": "Add docker release workflow", "files": [".github/workflows/release_docker.yaml", "Dockerfile"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1273, OPE-1216\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-10T03:06:23Z"}
{"number": 1742, "title": "[bug] update trainer to save processor when training with fsdp", "files": ["src/oumi/core/trainers/hf_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n- [bug] update trainer to save processor when training with fsdp\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-09T19:56:21Z"}
{"number": 1740, "title": "Add model revision param", "files": ["configs/recipes/falcon_e/dpo/falcon_e_1b_instruct/dpo.yaml", "configs/recipes/falcon_e/sft/falcon_e_1b/full_train.yaml", "configs/recipes/falcon_e/sft/falcon_e_1b_instruct/full_train.yaml", "configs/recipes/falcon_e/sft/falcon_e_3b/full_train.yaml", "configs/recipes/falcon_e/sft/falcon_e_3b_instruct/full_train.yaml", "docs/user_guides/train/configuration.md", "src/oumi/builders/models.py", "src/oumi/core/configs/params/model_params.py"], "area": "configs", "area_votes": {"configs": 6, "docs": 1, "other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add model revision param --\r\n  - This used to be passed inside model_kwargs\r\n  - Given this is important for reproducibility, adding as a first class param\r\n- This update is backward compatible \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-09T19:56:32Z"}
{"number": 1736, "title": "Update dev_setup.md to add additional instructions", "files": ["docs/development/dev_setup.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "mention that the results can be seen on the wandb website and to tear down the cluster when finished\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-05T22:58:17Z"}
{"number": 1734, "title": "Update inference to resume from temporary result file when possible", "files": ["src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/integration/infer/test_infer.py", "tests/unit/inference/test_base_inference_engine.py", "tests/unit/inference/test_generation_params.py", "tests/unit/inference/test_remote_inference_engine.py", "tests/unit/inference/test_vllm_inference_engine.py"], "area": "inference", "area_votes": {"inference": 5}, "body": "# Description\r\n\r\n\r\n\r\nInference jobs write results to a temporary file throughout, but we should also be resuming from said files when restarting the job.\r\n\r\nThis PR loads the scratch file before inference, skips any inputs that have already completed, runs inference, then reads all results from the scratch file, cleans up the scratch file, then returns.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1307\r\nFixes OPE-1257\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-20T16:25:42Z"}
{"number": 1733, "title": "[tiny] Update phi3-vision configs to use oumi trainer", "files": ["configs/recipes/vision/phi3/sft/full/completions_only_train.yaml", "configs/recipes/vision/phi3/sft/full/train.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n-  Update phi3-vision configs to use oumi trainer\r\n- Temporary workaround until this is is fixed:\r\n  - https://github.com/huggingface/transformers/issues/34690\r\n  - https://github.com/microsoft/PhiCookBook/issues/223\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-05T21:07:16Z"}
{"number": 1732, "title": "Add lmms-lab/multimodal-open-r1-8k-verified dataset", "files": ["src/oumi/datasets/__init__.py", "src/oumi/datasets/vision_language/__init__.py", "src/oumi/datasets/vision_language/lmms_lab_multimodal_open_r1.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add lmms-lab/multimodal-open-r1-8k-verified dataset\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-06T00:14:07Z"}
{"number": 1731, "title": "[tiny][bug] Add missing molmo feature", "files": ["configs/recipes/vision/molmo/sft/molmo_d_full/train.yaml", "configs/recipes/vision/molmo/sft/molmo_o_full/train.yaml", "src/oumi/core/configs/internal/supported_models.py"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\n\r\n\r\n- Add missing molmo feature `attention_mask` with `InternalFeatureFirstDimAction.KEEP`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-05T20:02:20Z"}
{"number": 1730, "title": "[tiny][bug] make git cmd optional", "files": ["src/oumi/utils/git_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- `git` cmd is used to log the git commit id.\r\n- If `git` is not installed, the exception is `FileNotFoundError`\r\n- Since reporting the commit id is best effort, this PR extends the exceptions list to add `FileNotFoundError`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1319\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-05T20:02:10Z"}
{"number": 1729, "title": "Readme for Falcon-E and note on extra dependencies required", "files": ["configs/recipes/falcon_e/README.md", "configs/recipes/falcon_h1/README.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\nAdded a readme for the Falcon-E recipes. Added a note on how to set up dependencies for Falcon-{H1, E} to get it working locally", "merged_at": "2025-06-05T17:50:00Z"}
{"number": 1728, "title": "Minor bugfixes for 2 clouds in launcher code", "files": ["src/oumi/launcher/clusters/polaris_cluster.py", "src/oumi/launcher/clusters/slurm_cluster.py"], "area": "launcher", "area_votes": {"launcher": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1281\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-05T22:17:01Z"}
{"number": 1727, "title": "Added doc for new QLoRA param", "files": ["docs/user_guides/train/configuration.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\nTo get Falcon-H1 model running with the `state-spaces/mamba` and `Dao-AILab/causal-conv1d` libraries, I had to add an additional `bitsandbytes` option to avoid quantizing one of the parameter matrices. This adds documentation for that.\r\n\r\n- [X] This PR only changes documentation.", "merged_at": "2025-06-05T17:16:50Z"}
{"number": 1726, "title": "Add generic vision dataset", "files": ["src/oumi/datasets/__init__.py", "src/oumi/datasets/vision_language/__init__.py", "src/oumi/datasets/vision_language/huggingface.py", "tests/unit/datasets/test_huggingface_vision_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\nThis PR introduces `HuggingFaceVisionDataset`, a generic dataset class for vision datasets stored on the HuggingFaceHub. It supports single image, single turn conversations (1 image, 1 user query, 1 assistant response). Images can be urls, raw bytes, or base64 encoded strings.\r\n\r\nUsage:\r\n```python\r\nfrom oumi.datasets import HuggingFaceVisionDataset\r\n\r\ndataset = HuggingFaceVisionDataset(\r\n    hf_dataset_path=\"\",\r\n    image_column=\"image\",\r\n    question_column=\"question\",\r\n    answer_column=\"answer\",\r\n    tokenizer=tokenizer,\r\n    processor=processor,\r\n    system_prompt=\"You are a helpful assistant.\"\r\n)\r\n```\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-05T19:08:01Z"}
{"number": 1724, "title": "Update inference to always write intermediate results to file.", "files": ["src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 4}, "body": "# Description\n\n\n\nSome inference jobs may fail midway either due to GPU memory issues, rate limits, etc.\n\nThis PR updates inference to *always* write intermediate results to some scratch file, ensuring that no money is wasted during inference runs.\n\n\n## Related issues\n\n\n\n\nTowards OPE-1307\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-06-05T17:10:15Z"}
{"number": 1723, "title": "Fix Falcon H1 dependency setup", "files": ["configs/recipes/falcon_h1/README.md", "configs/recipes/falcon_h1/evaluation/falcon_h1_0_5b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b_deep/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_34b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_3b/lambda_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_7b/lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_0_5b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b_deep/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_34b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_3b/full_lambda_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_7b/full_lambda_job.yaml"], "area": "configs", "area_votes": {"docs": 1, "configs": 12}, "body": "# Description\r\n\r\n- Switch from GCP to Lambda cluster for training jobs, as GCP's CUDA version of 12.2 results in errors with certain CUDA ops needed for SSMs.\r\n- Update some package installs to new recommendations from Falcon team. I had to install a specific dill version to prevent a version dependency issue.\r\n- Switch 0.5B training job to not use oumi distributed to avoid a bug\r\n\r\nI've tested the 0.5B training and evaluation jobs to confirm they work.\r\n\r\n## Related issues\r\n\r\nFixes OPE-1314\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2025-06-05T00:00:05Z"}
{"number": 1720, "title": "Frontier: Fix -n param in launcher script", "files": ["scripts/frontier/launcher.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- `-n` is the number of MPI tasks. Currently we launch 1 task per node i.e., `-N` and `-n` should be equal\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1280\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-04T21:08:34Z"}
{"number": 1719, "title": "Adds DPO + QLoRA example for Falcon-H1", "files": ["configs/recipes/falcon_e/dpo/falcon_e_1b_instruct/dpo.yaml", "configs/recipes/falcon_h1/dpo/falcon_h1_0_5b/qlora_dpo.yaml", "configs/recipes/llama3_2/dpo/1b_qlora_dpo.yaml", "src/oumi/core/configs/params/peft_params.py"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\nAdded simple recipe for DPO + QLoRA fine-tuning. Will change dataset to make this a more interesting example.", "merged_at": "2025-06-05T16:26:21Z"}
{"number": 1717, "title": "Judge API V2 | Core Functionality", "files": ["src/oumi/__init__.py", "src/oumi/core/configs/__init__.py", "src/oumi/core/configs/judge_config_v2.py", "src/oumi/judge_v2.py", "src/oumi/judges_v2/__init__.py", "src/oumi/judges_v2/base_judge.py", "src/oumi/judges_v2/simple_judge.py", "tests/e2e/test_judges.py", "tests/e2e/test_simple_judge.py", "tests/unit/judges/test_base_judge.py", "tests/unit/judges/test_simple_judge.py"], "area": "other", "area_votes": {"configs": 1, "other": 3}, "body": "# Description\r\n\r\n\r\n\r\nJudge API refactoring.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1327\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-11T21:20:40Z"}
{"number": 1715, "title": "Update installation.md to fix subversion handling by adding required \u2026", "files": ["docs/get_started/installation.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "\u2026quotations\r\n\r\nfixes an issue where subversion values fail to work without being wrapped in quotations\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-04T00:04:52Z"}
{"number": 1711, "title": "[tiny] fix pre-commits checks on a fresh install", "files": ["src/oumi/core/evaluation/backends/alpaca_eval.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\n- `alpaca_eval` is an optional dependency, and if not installed users will get a type check error\r\n- This PR adds the proper type ignore to mitigate the issue (not the most elegant fix, open to suggestions)\r\n \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-03T16:28:57Z"}
{"number": 1709, "title": "Update dev_setup.md to correct the order of steps", "files": ["docs/development/dev_setup.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "The repo should be cloned first before being able to run the \"make install-miniconda\"\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-03T00:50:50Z"}
{"number": 1706, "title": "[vision] Add option to process images individually", "files": ["src/oumi/core/collators/vision_language_sft_collator.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add option to process images individually:\r\n  - Some processors (e.g. Molmo) do not support processing a batch of images\r\n  - This PR adds an option to process each image individually instead\r\n- Also add more docstrings / explanations of each option\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-02T23:37:54Z"}
{"number": 1703, "title": "Add sample inference configs for `HuggingFaceTB/SmolVLM-Instruct`", "files": ["configs/recipes/vision/smolvlm/inference/infer.yaml", "configs/recipes/vision/smolvlm/inference/vllm_infer.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add vLLM and HF inference configs for `HuggingFaceTB/SmolVLM-Instruct`\r\n-- Docs will be updated separately\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-697\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-29T22:09:32Z"}
{"number": 1701, "title": "[tiny] Remove deprecated `use_async_dataset` from configs", "files": ["configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "configs/projects/coalm/405b_train.yaml", "configs/projects/coalm/70b_train.yaml", "configs/projects/coalm/8b_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_train.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml", "configs/recipes/llama4/sft/scout_base_full/train.yaml", "configs/recipes/llama4/sft/scout_instruct_full/train.yaml", "configs/recipes/llama4/sft/scout_instruct_lora/train.yaml", "configs/recipes/llama4/sft/scout_instruct_qlora/train.yaml", "configs/recipes/phi4/sft/reasoning_plus/full_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_train.yaml", "configs/recipes/qwen3/sft/30b_a3b_lora/train.yaml", "configs/recipes/qwen3/sft/32b_lora/train.yaml", "configs/recipes/qwq/sft/full_train.yaml", "configs/recipes/qwq/sft/lora_train.yaml", "configs/recipes/qwq/sft/qlora_train.yaml"], "area": "configs", "area_votes": {"configs": 48}, "body": "# Description\r\n\r\nThis is a deprecated unused parameter, and this PR deletes all usages of it. A future PR will delete the parameter altogether for a future Oumi version release.\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-05-29T00:59:21Z"}
{"number": 1700, "title": "Add data synthesis config, params, and unit tests", "files": ["src/oumi/core/configs/params/synthesis_params.py", "src/oumi/core/configs/synthesis_config.py", "tests/unit/core/configs/params/test_synthesis_params.py", "tests/unit/core/configs/test_synthesis_config.py"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\nAdd initial skeleton for data synthesis configs & params\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1288\r\nFixes OPE-1290\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-06-05T15:54:27Z"}
{"number": 1699, "title": "Implement Falcon H1", "files": ["configs/recipes/falcon_h1/README.md", "configs/recipes/falcon_h1/evaluation/falcon_h1_0_5b/eval.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_0_5b/gcp_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b/eval.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b/gcp_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b_deep/eval.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_1_5b_deep/gcp_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_34b/eval.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_34b/gcp_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_3b/eval.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_3b/gcp_job.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_7b/eval.yaml", "configs/recipes/falcon_h1/evaluation/falcon_h1_7b/gcp_job.yaml", "configs/recipes/falcon_h1/inference/0_5b_infer.yaml", "configs/recipes/falcon_h1/inference/1_5b_deep_infer.yaml", "configs/recipes/falcon_h1/inference/1_5b_infer.yaml", "configs/recipes/falcon_h1/inference/34b_infer.yaml", "configs/recipes/falcon_h1/inference/3b_infer.yaml", "configs/recipes/falcon_h1/inference/7b_infer.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_0_5b/full_gcp_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_0_5b/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b/full_gcp_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b_deep/full_gcp_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_1_5b_deep/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_34b/full_gcp_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_34b/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_3b/full_gcp_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_3b/full_train.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_7b/full_gcp_job.yaml", "configs/recipes/falcon_h1/sft/falcon_h1_7b/full_train.yaml", "docs/resources/models/supported_models.md", "src/oumi/cli/alias.py"], "area": "configs", "area_votes": {"docs": 2, "configs": 30, "cli": 1}, "body": "This PR adds support for FalconH1 \u2014 the new Hybrid Falcon series of models developed at the Technology Innovation Institute.\r\n\r\n**_Notes on config testing:_**\r\nFor all model sizes, the provided configs (e.g., `batch_size_per_device`, `grad_accum`, `lr`) have been tested in successful SFT training runs to validate runtime correctness and stability.\r\nWhile we did not perform exhaustive hyperparameter sweeps, the settings are **production-validated**, derived from **internal throughput experiments** and prior experience with this model family and dataset type.\r\n\r\nLet me know if further clarification is needed.", "merged_at": "2025-05-28T23:12:11Z"}
{"number": 1696, "title": "Add CoALM dataset class", "files": ["src/oumi/datasets/__init__.py", "src/oumi/datasets/sft/__init__.py", "src/oumi/datasets/sft/coalm.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add CoALM dataset class\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-22T23:25:09Z"}
{"number": 1694, "title": "[GRPO] Update letter counting notebook", "files": ["notebooks/Oumi - Train a Letter Counting Model using GRPO.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nThis includes the changes made by Jeremy in #1692, and the showcases the accuracy improving after training for 500 steps. I'm not sure why the eval results on the initial Llama 3B model changed so much. I don't think our eval function has changed since this notebook was created.\r\n\r\nWandB: https://wandb.ai/lema-academic/huggingface/runs/m5ja36bf?nw=nwuserwizeng\r\n\r\n## Related issues\r\n\r\nFixes OPE-1122\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-05-21T23:05:46Z"}
{"number": 1690, "title": "docs: Add GRPO/verl documentation", "files": ["configs/examples/grpo_verl_geometry3k/train.yaml", "docs/about/acknowledgements.md", "docs/resources/recipes.md", "docs/user_guides/train/configuration.md", "docs/user_guides/train/train.md", "docs/user_guides/train/training_methods.md", "src/oumi/datasets/grpo/rewards/countdown_rewards.py", "tests/unit/datasets/grpo/rewards/test_countdown_rewards.py"], "area": "docs", "area_votes": {"configs": 1, "docs": 5, "data": 1}, "body": "# Description\r\n\r\nThis PR adds documentation for GRPO, including both the trl and verl trainers.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1192\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-05-17T03:36:13Z"}
{"number": 1688, "title": "Add `torchao` version to `pyproject.toml`", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- The `transformers` library depends on `torchao` and I was getting errors when old/incompatible `torchao` version is installed: \r\n```\r\n    from ...modeling_utils import PreTrainedModel\r\n  File \"/home/user/miniconda3/envs/oumi/lib/python3.11/site-packages/transformers/modeling_utils.py\", line 53, in \r\n    from torchao.quantization import Int4WeightOnlyConfig\r\nImportError: cannot import name 'Int4WeightOnlyConfig' from 'torchao.quantization' (/home/user/miniconda3/envs/oumi/lib/python3.11/site-packages/torchao/quantization/__init__.py)\r\n\r\nThe above exception was the direct cause of the following exception:\r\n\r\nTraceback (most recent call last):\r\n  File \"/home/user/miniconda3/envs/oumi/bin/oumi\", line 8, in \r\n    sys.exit(run())\r\n```\r\n\r\n-- Configuring `torchao` version explicitly for better determinism\r\n \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-14T18:54:57Z"}
{"number": 1687, "title": "Add Geometry3K VLM dataset", "files": ["src/oumi/datasets/vision_language/__init__.py", "src/oumi/datasets/vision_language/geometry3k.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n-- Viewer: https://huggingface.co/datasets/hiyouga/geometry3k/viewer/default/train?views%5B%5D=train\r\n-- Tested with 1 GCP training run.\r\n-- https://github.com/volcengine/verl/blob/main/examples/data_preprocess/geo3k.py\r\n\r\nTowards OPE-1222, OPE-1229\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-14T18:30:12Z"}
{"number": 1686, "title": "[tiny] Update training environments doc", "files": ["docs/user_guides/train/environments/environments.md", "docs/user_guides/train/environments/local.md", "docs/user_guides/train/environments/notebooks.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-05-14T17:13:08Z"}
{"number": 1684, "title": "feat: Updated BaseConfig class for non primitive fields", "files": ["src/oumi/core/configs/base_config.py", "tests/unit/core/configs/test_base_config.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\nThis is a PR for the issue [1628](https://github.com/oumi-ai/oumi/issues/1628). The `to_yaml` does not warn when trying to save non primitive data types (such as functions). This leads to errors while loading from saved yaml files.\r\n\r\n\r\n\r\nI have added a check to gracefully check for non primitive types recursively. If present, they are removed from the config. The removed types and their paths are specified to the user via logs.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\nhttps://github.com/oumi-ai/oumi/issues/1628\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-11-03T19:12:31Z"}
{"number": 1683, "title": "[very nit] center oumi logo in the cli", "files": ["src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Center oumi logo when printed in the CLI -- just adding a newline after the logo since we have one before\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n![image](https://github.com/user-attachments/assets/55f08dc3-6f50-4db4-b0c3-9037663adce8)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-13T17:35:54Z"}
{"number": 1681, "title": "chore: edited the link to the stars badge", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-13T16:04:12Z"}
{"number": 1680, "title": "[Feature] Save evaluation config as YAML in output_dir #1546", "files": ["src/oumi/core/evaluation/utils/save_utils.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\nThis PR adds functionality to serialize and save the `EvaluationConfig` used during `oumi evaluate` into the output directory as a YAML file (`evaluation_config.yaml`). This ensures configuration details are stored with evaluation outputs, improving reproducibility and experiment transparency.\r\n\r\n**Key changes:**\r\n\r\n- Added `_convert_to_serializable` utility to handle serialization of complex types (Enums, nested custom classes), addressing limitations in default YAML serialization.\r\n- Integrated config saving into `save_evaluation_output` pipeline.\r\n- Verified output directory contains complete `evaluation_config.yaml`.\r\n\r\n## Related Issues\r\n\r\nFixes #1546 ([https://github.com/oumi-ai/oumi/issues/1546](https://github.com/oumi-ai/oumi/issues/1546))\r\n\r\n## Testing\r\n\r\n- [x] Manually verified `oumi evaluate` generates correct `evaluation_config.yaml`.\r\n\r\n## Before submitting\r\n\r\n- [x] This PR does not only change documentation.\r\n- [x] I have read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md).\r\n- [x] I have linked the related issue in the section above.\r\n- [x] Tested locally that `oumi evaluate` generates the correct `evaluation_config.yaml`.\r\n- [ ] Tests added or updated where needed\r\n\r\n## Reviewers\r\n\r\n- wizeng23: [William Zeng](https://github.com/wizeng23)\r\n", "merged_at": "2025-05-15T05:04:39Z"}
{"number": 1679, "title": "Fix a test breakage caused by a new Click version (8.2.0)", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\nClick published a new version today with a breaking change: https://pypi.org/project/click/8.2.0/\r\n\r\nTyper doesn't pin their Click version, and has an outstanding PR to fix this issue: https://github.com/fastapi/typer/pull/1145\r\n\r\nThis is a temporary change to prevent breakage in Oumi until Typer has updated their code accordingly.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-11T15:58:10Z"}
{"number": 1674, "title": "Add Phi4 reasoning plus configs", "files": ["configs/recipes/phi4/README.md", "configs/recipes/phi4/evaluation/reasoning_plus_eval.yaml", "configs/recipes/phi4/evaluation/reasoning_plus_gcp_job.yaml", "configs/recipes/phi4/inference/reasoning_plus_infer.yaml", "configs/recipes/phi4/sft/reasoning_plus/full_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/full_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/lora_train.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_gcp_job.yaml", "configs/recipes/phi4/sft/reasoning_plus/qlora_train.yaml", "configs/recipes/qwq/sft/qlora_train.yaml", "docs/resources/models/supported_models.md", "src/oumi/cli/alias.py"], "area": "configs", "area_votes": {"docs": 2, "configs": 10, "cli": 1}, "body": "# Description\r\n\r\nThis includes FFT/Lora/QLora configs. I've also added an alias to these configs.\r\n\r\n## Related issues\r\n\r\nFixes OPE-1226\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-05-07T03:57:27Z"}
{"number": 1673, "title": "Migrate to `logger.warning` usage", "files": ["src/oumi/core/tokenizers/utils.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\nThis small PR resolves the `logger.warn()` deprecation warnings:\r\n```python\r\n/home/runner/work/oumi/oumi/src/oumi/core/tokenizers/utils.py:106: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead\r\n```\r\n\r\n## Related issues\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2025-05-06T19:01:32Z"}
{"number": 1671, "title": "Fix broken tests due to precommit violations", "files": ["src/oumi/core/configs/params/grpo_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\nFixed by running\r\n```\r\npre-commit run --all-files\r\n```\r\n\r\nFailed run: https://github.com/oumi-ai/oumi/actions/runs/14849096322/job/41689217638\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-06T01:13:58Z"}
{"number": 1670, "title": "Set explicit permissions for our test workflows.", "files": [".github/workflows/doctests.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 3}, "body": "# Description\r\n\r\n\r\n\r\nThis PR sets the permissions for our workflows so they don't inherit from the default org permission set. This should ultimately be a no-op for the workflows as they all require only `read` permissions.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-05T23:01:40Z"}
{"number": 1669, "title": "[Documentation] Custom Evaluations (PR 2-of-2)", "files": ["docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/evaluation_config.md", "docs/user_guides/evaluate/generative_benchmarks.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Custom Evaluations (2 of 2)\r\n\r\nThe second part updates the evaluation documentation to introduce custom evals (in the overview section), properly link to custom evals & update the evaluation config page to include custom backend as an option. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-06T02:00:44Z"}
{"number": 1664, "title": "[Documentation] Custom Evaluations (PR 1-of-2)", "files": ["docs/user_guides/evaluate/custom_evals.md", "docs/user_guides/evaluate/evaluate.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Custom Evaluations (1 of 2)\r\n\r\nNote: This only adds the page for custom evaluations. I also need to update other parts of the evaluation documentation to properly link to this & introduce custom evals (in the overview section and wherever else is applicable).\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-01T23:12:14Z"}
{"number": 1659, "title": "[verl] Populate verl config from Oumi config", "files": ["src/oumi/core/configs/params/training_params.py", "src/oumi/core/trainers/verl_grpo_trainer.py", "src/oumi/core/trainers/verl_trainer_config.yaml", "src/oumi/train.py", "tests/unit/core/trainers/test_verl_grpo_trainer.py"], "area": "training", "area_votes": {"configs": 1, "training": 3}, "body": "# Description\r\n\r\n- Add a `verl_config_overrides` field in TrainingParams to allow setting any verl config value. This is similar to what `trainer_kwargs` does for HF/TRL trainers, and gives users flexibility to pass in fields we don't have in Oumi configs.\r\n- Replace `TrainingParams` with `TrainingConfig` in verl trainer initialization since values in the latter are needed for verl.\r\n- Populate verl config from some Oumi config fields and user-provided overrides.\r\n\r\nFuture work\r\n- Add more Oumi -> verl config field converters. This PR only adds a few, but adding more is time-consuming. There are 200 verl config values, and I need to read into the verl codebase for many as similarly-named params in Oumi/trl/verl are not the same. For example, `max_prompt_length` in Oumi/trl will truncate prompts longer than that value, but in verl, it will result in an error for longer prompts.\r\n- Add new fields in Oumi configs for verl param values we believe are important enough to get their own Oumi param field, as opposed to having be passed through `verl_config_overrides`.\r\n- Add countdown training/gcp job configs.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1172\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-04-30T18:12:39Z"}
{"number": 1654, "title": "Add support for repetition_penalty in GrpoParams", "files": ["src/oumi/core/configs/params/grpo_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nThis PR adds support for the `repetition_penalty` parameter in the `GrpoParams` class.\r\n\r\n`repetition_penalty` is a generation parameter commonly used in language models to discourage or encourage the repetition of tokens. By default, it is set to 1.0 (no penalty). Values >1.0 reduce repetition in generated text, and values <1.0 increase it.\r\n\r\nThis change allows users to fine-tune output repetition behavior during generation through Oumi\u2019s generation interface, bringing it in line HuggingFace TRL GRPO.\r\n\r\nTested that TRL GRPO training works, and also tested that regular training isn't affected.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1655 \r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-05-06T00:12:23Z"}
{"number": 1649, "title": "[bug] fix build errors", "files": ["scripts/demo.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- fix build errors, accidentally caused by #1647 \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-21T09:53:25Z"}
{"number": 1648, "title": "[tiny] Update cli help shorthand", "files": ["src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Update CLI to use `-h` in addition to `--help`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-757\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-21T19:13:49Z"}
{"number": 1647, "title": "Add demo script", "files": ["scripts/.gitignore", "scripts/demo.py"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- We frequently need to demo / walkthrough usage of the oumi pip package.\r\n  - This PR adds a simple workflow that's fast enough to run on a laptop < 5 minutes \r\n- The demo lives in the scripts folder, so is not part of the oumi package \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-21T03:10:33Z"}
{"number": 1644, "title": "[Evaluation] Convenience function for standard config retrieval", "files": ["configs/apis/anthropic/eval_claude_3_5_sonnet.yaml", "configs/apis/anthropic/eval_claude_3_7_sonnet.yaml", "configs/apis/anthropic/infer_claude3_7.yaml", "configs/apis/anthropic/infer_claude_3_5_sonnet.yaml", "configs/apis/anthropic/infer_claude_3_7_sonnet.yaml", "configs/apis/gemini/eval_gemini_1_5_pro.yaml", "configs/apis/gemini/infer_gemini_1_5_pro.yaml", "configs/apis/openai/eval_gpt_4o.yaml", "configs/apis/openai/eval_gpt_o1_preview.yaml", "configs/apis/openai/infer_gpt_4o.yaml", "configs/apis/openai/infer_gpt_o1_preview.yaml", "configs/apis/vertex/eval_llama_3_1_405b.yaml", "configs/apis/vertex/eval_llama_3_3_70b.yaml", "configs/apis/vertex/infer_llama_3_1_405b.yaml", "configs/apis/vertex/infer_llama_3_3_70b.yaml", "src/oumi/cli/alias.py"], "area": "configs", "area_votes": {"configs": 15, "cli": 1}, "body": "# Description\r\n\r\n\r\n\r\nSecond attempt to implement this convenience function (previous one [here](https://github.com/oumi-ai/oumi/pull/1632))\r\n\r\nObjective: Convenience function to instantiate `EvaluationConfig`s for custom evals\r\n- only for the most common/popular hosted models\r\n- objective is to be used from users/notebooks that do NOT want to understand our configuration options; they want something that works as-is out-of-the-box without any complexity (ie. single step: model name -> config)\r\n\r\nAfter iterating on this with Matt: the fn will be implemented with the code snippet instead:\r\n```\r\nremote_config_path = try_get_config_name_for_alias(model_name, AliasType.EVAL)\r\nlocal_config_path = cli_utils.resolve_and_fetch_config(remote_config_path)\r\nconfig = EvaluationConfig.from_yaml(local_config_path)\r\nconfig.tasks[0].task_name=\"my_custom_fn\"\r\n```\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-19T01:28:37Z"}
{"number": 1641, "title": "Update dev_setup.md", "files": ["docs/development/dev_setup.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nInclude Llama 4\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-04-16T20:17:36Z"}
{"number": 1640, "title": "Rename cnn_mnist_example to cnn_mnist_tutorial", "files": ["notebooks/Oumi - Training CNN on Custom Dataset.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nThis is so the output directory is properly tracked by the gitignore\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-04-16T17:33:50Z"}
{"number": 1638, "title": "[Remote Inference] Error checking for `api_key`", "files": ["src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nChanges\r\n- Do NOT include the AUTH BEARER header in the request, if API KEY is NOT available\r\n- Fast fail if the API KEY is NOT available but required (`api_key_env_varname` is set)\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-16T16:37:04Z"}
{"number": 1636, "title": "[Remote Inference][GCP] Constructing `api_url` from the Project ID and Region", "files": ["src/oumi/inference/gcp_inference_engine.py", "tests/unit/inference/test_gcp_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nRemote Inference for GCP: Constructing the `api_url` from the Project ID and Region.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-16T17:46:48Z"}
{"number": 1631, "title": "Update trl to 0.16", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nIt claims to make GRPO 6x faster: [github](https://github.com/huggingface/trl/releases/tag/v0.16.0). Tested all e2e tests\r\n\r\n## Related issues\r\n\r\nFixes OPE-1157\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-04-11T20:25:11Z"}
{"number": 1630, "title": "[Remote Inference] Update Default Params", "files": ["src/oumi/core/configs/params/generation_params.py", "src/oumi/core/configs/params/remote_params.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/gcp_inference_engine.py", "src/oumi/inference/gemini_inference_engine.py", "src/oumi/inference/openai_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_anthropic_inference_engine.py", "tests/unit/inference/test_gcp_inference_engine.py", "tests/unit/inference/test_gemini_inference_engine.py", "tests/unit/inference/test_generation_params.py", "tests/unit/inference/test_openai_inference_engine.py"], "area": "inference", "area_votes": {"configs": 2, "inference": 5}, "body": "# Description\r\n\r\n\r\n\r\nRemote Inference: Update Defaults\r\n\r\n**_Generation_**\r\n- max_new_tokens: 256 -> 1024\r\n- temperature: 0.0 -> 1.0 (_only for OpenAi o1_)\r\n\r\n**_Remote Inference_**\r\n- connection_timeout: 20 -> 300\r\n- (num_workers, politeness_policy): (1,0) -->\r\n  - Anthropic (5,60)\r\n  - GCP (10,60)\r\n  - Gemini (2,60)\r\n  - OpenAI (50,60)\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-11T18:24:25Z"}
{"number": 1624, "title": "Add additional_model_kwargs and additional_trainer_kwargs to train function", "files": ["src/oumi/__init__.py", "src/oumi/train.py"], "area": "training", "area_votes": {"training": 1}, "body": "\r\n\r\nThis pull request includes several changes to the `train` function and related methods in the `oumi` module to support additional model and trainer keyword arguments. The motivation was to allow passing non-serializable keyword arguments into the trainer class (and the model class). The most important changes are as follows:\r\n\r\n### Enhancements to `train` function:\r\n\r\n* Modified the `train` function in `src/oumi/__init__.py` to accept `additional_model_kwargs` and `additional_trainer_kwargs` parameters and pass them to the `oumi.train.train` function.\r\n\r\n### Enhancements to `train` function in `src/oumi/train.py`:\r\n\r\n* Updated the `_create_optional_training_kwargs` function to accept an `additional_trainer_kwargs` parameter and include it in the returned dictionary. [[1]](diffhunk://#diff-883bb6b1dff705e291235aa01c36f1b4d1a80d1907c27b30bcf299f7c63972d2R192) [[2]](diffhunk://#diff-883bb6b1dff705e291235aa01c36f1b4d1a80d1907c27b30bcf299f7c63972d2R208-R216)\r\n* Modified the `train` function to accept `additional_model_kwargs` and `additional_trainer_kwargs` parameters and use them when building the model and creating optional training kwargs. [[1]](diffhunk://#diff-883bb6b1dff705e291235aa01c36f1b4d1a80d1907c27b30bcf299f7c63972d2L278-R284) [[2]](diffhunk://#diff-883bb6b1dff705e291235aa01c36f1b4d1a80d1907c27b30bcf299f7c63972d2R363)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1623 by allowing user to pass a `preprocess_logits_for_metrics` function to `additional_trainer_kwargs` in the `train` function.\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-09T23:59:32Z"}
{"number": 1622, "title": "Lazy load skypilot", "files": ["src/oumi/launcher/clients/sky_client.py", "src/oumi/launcher/clouds/sky_cloud.py", "src/oumi/launcher/clusters/sky_cluster.py", "tests/unit/launcher/clouds/test_sky_cloud.py", "tests/unit/launcher/test_launcher.py"], "area": "launcher", "area_votes": {"launcher": 3}, "body": "# Description\r\n\r\n\r\n\r\nThis change updates all skypilot entry points to lazy load the `sky` library. This prevents DB locking that can occur when using skypilot in a multithreaded environment.\r\n\r\nI also added a test to ensure that `sky` is not loaded when importing the `oumi.launcher` module.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1605\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-08T17:31:37Z"}
{"number": 1621, "title": "Update llama4 GCP jobs for non-dev environments.", "files": ["configs/recipes/llama4/sft/scout_base_full/gcp_job.yaml", "configs/recipes/llama4/sft/scout_instruct_full/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\nAdd the `oumi://` prefix to the llama4 GCP configs to ensure the path exists when running from a non-dev environment.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-07T20:10:11Z"}
{"number": 1618, "title": "Add alias functionality for train, launch, evaluate, and infer in the oumi CLI", "files": ["src/oumi/cli/alias.py", "src/oumi/cli/evaluate.py", "src/oumi/cli/infer.py", "src/oumi/cli/launch.py", "src/oumi/cli/train.py", "tests/unit/cli/test_cli_alias.py", "tests/unit/cli/test_cli_evaluate.py", "tests/unit/cli/test_cli_infer.py", "tests/unit/cli/test_cli_launch.py", "tests/unit/cli/test_cli_train.py"], "area": "cli", "area_votes": {"cli": 5}, "body": "# Description\r\n\r\n\r\n\r\n\r\nThis PR adds aliases to the CLI to work as shorthand for commonly used configs.\r\n\r\nFor example, the old command:\r\n```\r\noumi train -c oumi://configs/recipes/llama4/sft/scout_base_full/train.yaml\r\n```\r\ncan now be shortened to\r\n```\r\noumi train -c llama4-scout\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-06T21:32:13Z"}
{"number": 1617, "title": "[configs] Add llama4 eval configs", "files": ["configs/recipes/llama4/evaluation/scout_instruct_eval.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-06T20:44:38Z"}
{"number": 1616, "title": "[configs] Add llama4 inference configs, tweak training configs", "files": ["README.md", "configs/recipes/llama4/inference/maverick_instruct_together_infer.yaml", "configs/recipes/llama4/inference/scout_instruct_infer.yaml", "configs/recipes/llama4/inference/scout_instruct_together_infer.yaml", "configs/recipes/llama4/inference/scout_instruct_vllm_infer.yaml", "configs/recipes/llama4/sft/scout_instruct_full/train.yaml", "configs/recipes/llama4/sft/scout_instruct_lora/train.yaml", "configs/recipes/llama4/sft/scout_instruct_qlora/train.yaml", "docs/resources/models/supported_models.md"], "area": "configs", "area_votes": {"docs": 2, "configs": 7}, "body": "# Description\r\n\r\n\r\n\r\n- Add llama4 inference configs for Native, VLLM, and Together.ai\r\n- Tweak training configs\r\n- Add LoRa training config\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-06T17:31:44Z"}
{"number": 1615, "title": "Add configs for Llama4 Scout and Maverick", "files": ["configs/recipes/llama4/README.md", "configs/recipes/llama4/sft/scout_base_full/gcp_job.yaml", "configs/recipes/llama4/sft/scout_base_full/train.yaml", "configs/recipes/llama4/sft/scout_instruct_full/gcp_job.yaml", "configs/recipes/llama4/sft/scout_instruct_full/train.yaml", "configs/recipes/llama4/sft/scout_instruct_qlora/train.yaml", "pyproject.toml"], "area": "configs", "area_votes": {"docs": 1, "configs": 5, "infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add configs for Llama4 Scout and Maverick\r\n- Tested on single-node 8xA100:80GB on GCP\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-06T03:20:08Z"}
{"number": 1613, "title": "[HallOumi] Update inference notebook", "files": ["configs/projects/halloumi/halloumi_inference_notebook.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n- Install oumi from source, because this colab uses the newly added `to_str()` fn\r\n- Add a cell suggesting to install vllm\r\n- Limit model max length or else it surpasses kv cache size\r\n", "merged_at": "2025-04-07T07:05:56Z"}
{"number": 1612, "title": "[HallOumi] README: Adding Clarity - differentiating between generative and classifier", "files": ["configs/projects/halloumi/README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nHallOumi README: Adding Clarity  - better differentiating between generative and classifier\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-04T23:47:58Z"}
{"number": 1611, "title": "Add a news section to the readme", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdd a `news` section to our main readme.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-04T23:35:05Z"}
{"number": 1610, "title": "Update exposure (readme etc.) of Phi4, Qwen2.5", "files": ["README.md", "docs/resources/recipes.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\nUpdates links to visible repo locations (`readme.md`, `recipes.md`) for using our newly supported **Phi4** and **Qwen2.5** _VLMs_.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-04T23:09:46Z"}
{"number": 1609, "title": "[HallOumi] Evaluation notebook improvements", "files": ["configs/projects/halloumi/halloumi_eval_notebook.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nHallOumi: Eval notebook - minor improvements\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-04T20:28:24Z"}
{"number": 1608, "title": "[HallOumi] Inference notebook for non-generative model", "files": ["configs/projects/halloumi/halloumi_classifier_inference_notebook.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nHallOumi: Inference notebook for non-generative model\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-04T22:43:02Z"}
{"number": 1607, "title": "feat (env): include mlflow in displayed dependencies", "files": ["src/oumi/cli/env.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\nThis PR adds `mlflow` to the list of core packages shown when running the `oumi env` command. This change allows users to see the installed version of `mlflow` as part of the environment report, which is helpful for debugging.\r\n\r\n## Related issues\r\n\r\n#1600 \r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\n@oumi-ai/oumi-staff\r\n", "merged_at": "2025-04-04T20:11:42Z"}
{"number": 1604, "title": "[HallOumi Readme] Link to inference notebook", "files": ["configs/projects/halloumi/README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n[HallOumi Readme] Link to inference notebook\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-04T06:04:27Z"}
{"number": 1602, "title": "LoRA training for Qwen2.5-VL", "files": ["configs/recipes/vision/qwen2_5_vl_3b/sft/full/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/full/train.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/lora/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/lora/train.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\nAdds recipe/configs for training with SFT and LoRA Qwen2.5 vision-language model.\r\n\r\n- The recipe was tested with `flash_attention` and `sdpa` as the attention mechanism and with a single or multiple A100 GPUs (data parallel). The dataset used was `merve/vqav2-small` along with the rest of the hyperparameters shown in the included train.yaml.\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # OPE-1101\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-04T00:08:18Z"}
{"number": 1601, "title": "docs: update datasets.md", "files": ["docs/resources/datasets/datasets.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "\r\n\r\n# Description\r\n\r\n\r\n\r\nverions -> versions\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-03T22:06:18Z"}
{"number": 1598, "title": "[HallOumi] Inference notebook for generative model", "files": ["configs/projects/halloumi/halloumi_inference_notebook.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nHallOumi: Inference notebook for generative model\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-04T05:30:27Z"}
{"number": 1596, "title": "Fix build", "files": ["src/oumi/builders/data.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Small fix to unblock the cpu build.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-03T03:10:22Z"}
{"number": 1593, "title": "Update HallOumi configs", "files": ["configs/projects/halloumi/8b_train.yaml", "configs/projects/halloumi/README.md", "configs/projects/halloumi/gcp_job.yaml", "configs/projects/halloumi/img/halloumi_table.png"], "area": "configs", "area_votes": {"configs": 3, "docs": 1}, "body": "# Description\r\n\r\nThis adds additional useful information, and makes them consistent with our other config. By using the `oumi://` config prefix, users can run the config without cloning the repo.\r\n\r\nAdditionally, this replaces the table image with a markdown table.\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-04-03T03:57:39Z"}
{"number": 1592, "title": "Update README.md", "files": ["configs/projects/halloumi/README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-02T05:50:01Z"}
{"number": 1590, "title": "Halloumi notebook ruff formatting", "files": ["configs/projects/halloumi/halloumi_eval_notebook.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-02T05:42:51Z"}
{"number": 1588, "title": "Create halloumi_eval_notebook.ipynb", "files": ["configs/projects/halloumi/halloumi_eval_notebook.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-02T04:22:22Z"}
{"number": 1587, "title": "Add training configs for HallOumi", "files": ["configs/projects/halloumi/8b_train.yaml", "configs/projects/halloumi/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-04-02T04:02:45Z"}
{"number": 1586, "title": "LoRA configs for Phi4 Multimodal", "files": ["configs/recipes/vision/phi4/sft/full/gcp_job.yaml", "configs/recipes/vision/phi4/sft/full/train.yaml", "configs/recipes/vision/phi4/sft/lora/gcp_job.yaml", "configs/recipes/vision/phi4/sft/lora/train.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\n- Adds LoRA configs for Phi4 Multimodal SFT training.\r\n-- Minor: slightly touches some comments/names in Phi4 Multimodal full training. \r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards: OPE-1102\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-02T19:26:07Z"}
{"number": 1585, "title": "Create README.md", "files": ["configs/projects/halloumi/README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-02T01:23:37Z"}
{"number": 1584, "title": "[GRPO] Implement `transform_conversation()` in `BaseExperimentalGrpoDataset`", "files": ["src/oumi/core/datasets/base_grpo_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\nThe default behavior of this function parallels the default behavior for `transform()`. This fixes the broken GRPO TLDR example.\r\n\r\n## Related issues\r\n\r\nFixes OPE-1176\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-04-01T23:38:00Z"}
{"number": 1583, "title": "[Evaluation] Support for confidence intervals", "files": ["src/oumi/core/evaluation/metrics.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nSupport for confidence intervals\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-01T21:23:35Z"}
{"number": 1582, "title": "PromptResponseDataset: make model_output optional", "files": ["src/oumi/datasets/sft/prompt_response.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\nPromptResponseDataset: make model_output optional\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-01T04:29:40Z"}
{"number": 1579, "title": "Add support for conversational datasets from huggingface", "files": ["src/oumi/datasets/__init__.py", "src/oumi/datasets/sft/__init__.py", "src/oumi/datasets/sft/huggingface.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\n\n\n\nAdd support for conversational datasets from huggingface\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-04-01T02:23:23Z"}
{"number": 1577, "title": "Upgrade `transformers` to `v4.49.*`", "files": ["configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/eval.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/phi3/evaluation/eval.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/gcp_job.yaml", "pyproject.toml", "tests/e2e/test_eval_e2e.py", "tests/scripts/launch_tests.sh"], "area": "configs", "area_votes": {"configs": 10, "infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Upgrade to `v4.49.*` , which is needed for Qwen 2.5 VL support\r\n-- Updated more eval configs to use `shard_for_eval: True` for compatibility (see OPE-931), and for consistency with larger model sizes.\r\n-- Upgrade to `v4.50` may be less straightforward (observed some errors/issues) to be handled later separately.\r\n-- Tested: e2e integration tests` tests/scripts/launch_tests.sh` + some manual tests, including inference\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-984, OPE-931, OPE-1175\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-01T18:36:58Z"}
{"number": 1576, "title": "[Configs] add support for loading config from yaml string", "files": ["src/oumi/core/configs/base_config.py", "tests/unit/core/configs/test_config.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\nConfigs: Add support for loading config from yaml string\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-31T19:48:35Z"}
{"number": 1575, "title": "[Evaluation] Custom functions: support for 2 output types", "files": ["src/oumi/core/evaluation/evaluator.py", "tests/unit/core/evaluation/test_evaluator.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nCustom functions: support for 2 output types\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1097\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-28T19:08:08Z"}
{"number": 1570, "title": "[GRPO] Reorganize letter counting configs", "files": ["configs/examples/letter_counting/grpo/gcp_job.yaml", "configs/examples/letter_counting/grpo/train.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/eval.yaml", "configs/recipes/qwq/evaluation/eval.yaml", "src/oumi/core/configs/evaluation_config.py", "src/oumi/datasets/grpo/letter_count.py", "src/oumi/datasets/grpo/rewards/count_letters_rewards.py"], "area": "configs", "area_votes": {"configs": 7, "data": 2}, "body": "# Description\r\n\r\n- Changed directory structure in preparation for future PR to add letter counting evaluation.\r\n- Changed model for GRPO letter counting to Deepseek R1 distilled Qwen 1.5B, as reasoning models should have better performance.\r\n- Removed some unnecessary `shard_for_eval` params for smaller models.\r\n- Fixed broken documentation link.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1122\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-03-26T19:58:47Z"}
{"number": 1569, "title": "e2e eval integration tests updates", "files": ["configs/recipes/phi3/evaluation/eval.yaml", "tests/e2e/test_eval_e2e.py", "tests/e2e/test_train_e2e.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Reset `shard_for_eval` to `False` for e2e tests using 1 GPU (leads to errors otherwise)\r\n-- Optimize some test params e.g., move `iterative_logs` to test specs (currently hardcoded)\r\n-- Update Phi3 eval config to use `bfloat16` \r\n-- Tested e2e eval tests on `A100:1` and `A100:4`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-909\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-25T23:17:08Z"}
{"number": 1567, "title": "[Evaluation] Bug: Multiple GPUs attempt to save in the same folder", "files": ["src/oumi/core/evaluation/evaluator.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nBug: Multiple GPUs attempt to save in the same folder\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1151\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-25T00:44:09Z"}
{"number": 1566, "title": "Increment `pillow` version for compatibility with Python 3.13", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Increment version to support Python 3.13\r\n-- Manually tested with few VLMs (Llama Vision, Qwen), no problems found.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes OPE-1119\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-24T23:01:11Z"}
{"number": 1565, "title": "Fix printing errors in oumi env for non-string values.", "files": ["src/oumi/cli/env.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n`device_count()` has no type annotation and can return a number, which typer cannot parse. This CL casts erroneous values to strings before printing.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-24T04:10:12Z"}
{"number": 1562, "title": "[Remote Inference] Supported params are ignored", "files": ["src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/gcp_inference_engine.py", "src/oumi/inference/gemini_inference_engine.py", "src/oumi/inference/openai_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/remote_vllm_inference_engine.py", "src/oumi/inference/sambanova_inference_engine.py", "src/oumi/inference/sglang_inference_engine.py", "tests/unit/inference/test_anthropic_inference_engine.py", "tests/unit/inference/test_deepseek_inference_engine.py", "tests/unit/inference/test_gcp_inference_engine.py", "tests/unit/inference/test_gemini_inference_engine.py", "tests/unit/inference/test_generation_params.py", "tests/unit/inference/test_inference_engine_init.py", "tests/unit/inference/test_openai_inference_engine.py", "tests/unit/inference/test_parasail_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py", "tests/unit/inference/test_sambanova_inference_engine.py", "tests/unit/inference/test_sglang_inference_engine.py", "tests/unit/inference/test_together_inference_engine.py"], "area": "inference", "area_votes": {"inference": 9}, "body": "# Description\r\n\r\n\r\n\r\n\r\nUpdates:\r\n1) ~~Validating that all params are supported, before the remote API call~~. \r\n2) Adding 2 missing params (`min_p`, `stop_token_ids`) in `_convert_conversation_to_api_input` (base remote engine)\r\n3) Open AI: Dropping `logit_bias` for \"o1-preview\" (this model was not working before)\r\n4) Instead of always using the model params that were used when initializing the inference engine (`self._model_params`), this change allows users to update the model with a new inference config (`inference_config.model`) after the engine is instantiated. This allows users to use the same OpenAI engine for gpt-4, but later (without re-instantiating the engine) they can pass in a new config and run inference for OpenAI's `o1-preview`.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1123\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-28T20:33:35Z"}
{"number": 1560, "title": "Logging message update in `log_number_of_model_parameters`", "files": ["src/oumi/utils/torch_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Convert `Parameters` tom lowercase  for some cases to make message look better.\r\n-- No-op.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-20T16:56:38Z"}
{"number": 1558, "title": "Remove UV install from notebooks as this breaks colab", "files": ["notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Build your own Custom Evaluation (Reliability Classifier).ipynb", "notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - MiniMath-R1-1.5B.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 16}, "body": "# Description\r\n\r\n\r\n\r\nThis is a temporary workaround for #1554 while we investigate why `uv` is installing the incorrect pypi package.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1554\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-19T04:41:39Z"}
{"number": 1557, "title": "[Evaluations] Custom evals: Adding support for `eval_kwargs`", "files": ["src/oumi/core/evaluation/evaluator.py", "tests/unit/core/evaluation/test_evaluator.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nCustom evals: Adding support for `eval_kwargs`\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1097\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-20T15:35:28Z"}
{"number": 1553, "title": "Misc no-op code cleanups", "files": ["src/oumi/core/feature_generators/vision_language_conversation_feature_generator.py", "src/oumi/core/types/conversation.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n--  Reduce code duplication in `vision_language_conversation_feature_generator.py`\r\n-- Remove noisy type aliases in conversation.py\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-991\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-18T18:54:56Z"}
{"number": 1552, "title": "[Evaluation] Updates in hallucination notebook", "files": ["notebooks/Oumi - Build your own Custom Evaluation (Hallucination Classifier).ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nUpdates in hallucination notebook (requested by Manos for clarity).\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-19T21:07:29Z"}
{"number": 1548, "title": "Render the oumi env command as a shell block in bug reports.", "files": [".github/ISSUE_TEMPLATE/bug-report.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\nAfter switching to Rich formatting, we should render the `oumi env` output in a shell block.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-17T17:53:03Z"}
{"number": 1547, "title": "Update the CLI to replace all prints with Rich prints.", "files": ["src/oumi/cli/cli_utils.py", "src/oumi/cli/infer.py", "src/oumi/cli/judge.py", "src/oumi/cli/train.py"], "area": "cli", "area_votes": {"cli": 4}, "body": "# Description\r\n\r\n\r\n\r\n- All `print(...)` statements in the CLI have been replaced with an alternative from Rich.\r\n- Added a loading spinner when importing python deps for train\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-306\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-17T17:20:50Z"}
{"number": 1544, "title": "Update oumi evaluate to use rich formatting.", "files": ["src/oumi/cli/evaluate.py", "tests/unit/cli/test_cli_evaluate.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\nUpdate oumi evaluate to use Rich formatting.\r\n\r\nWe make a best-effort to parse the results of evaluation and visualize them in the CLI. See unit tests for how we handle non-standard metrics.\r\n![Screenshot 2025-03-15 at 7 01 56\u202fPM](https://github.com/user-attachments/assets/1012f879-c1fc-4b5f-8e4d-8743de0c0ea1)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-306\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-16T22:36:42Z"}
{"number": 1541, "title": "Update Oumi Env to use Rich formatting", "files": ["src/oumi/cli/cli_utils.py", "src/oumi/cli/env.py", "src/oumi/cli/main.py", "tests/unit/cli/test_cli_env.py", "tests/unit/cli/test_cli_utils.py"], "area": "cli", "area_votes": {"cli": 3}, "body": "# Description\r\n\r\n\r\n\r\nThis PR comes in two small parts:\r\n- Updates the `oumi env` command to use rich formatting\r\n- Updates our ASCII logo\r\nSee the screenshot below for the new experience:\r\n![Screenshot 2025-03-14 at 10 58 51\u202fAM](https://github.com/user-attachments/assets/995a0e34-26ac-4005-ae39-f60a0468c7a7)\r\n\r\nvs old experience:\r\n\r\n\"Screenshot\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-306\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-14T18:55:21Z"}
{"number": 1537, "title": "Auto-populate and validate params specific to `vision_language_sft` collator in `TrainingConfig`", "files": ["src/oumi/core/configs/training_config.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Setup and validate params for `vision_language_sft` collator. The collator expects VLM SFT dataset to only produce one column: 'conversation_json' (JSON-encoded `Conversation`)! This PR auto-configures misc other params for  `vision_language_sft`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-355, OPE-1109\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-13T18:02:15Z"}
{"number": 1534, "title": "[docs] add security.md", "files": ["SECURITY.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add security.md notice, based on [GitHub example](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-11T22:12:48Z"}
{"number": 1533, "title": "Upload scripts used in a Weekly Walkthrough", "files": ["scripts/examples/batch_inference/README.md", "scripts/examples/batch_inference/bulk_infer.py", "scripts/examples/batch_inference/infer.yaml", "scripts/examples/batch_inference/smollm_infer.yaml"], "area": "infra", "area_votes": {"docs": 1, "infra": 3}, "body": "# Description\r\n\r\n\r\n\r\nAdding a script used in a recent walkthrough video for Oumi.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-11T19:38:26Z"}
{"number": 1532, "title": "Minor logging improvements in `BaseMapDataset`", "files": ["src/oumi/core/datasets/base_map_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Use effective `dataset_name` value\r\n-- Only print `dataset_name`/`dataset_path` if non-empty to reduce clutter.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-11T18:28:53Z"}
{"number": 1531, "title": "Update VisionLanguageConversationFeatureGenerator", "files": ["src/oumi/core/feature_generators/vision_language_conversation_feature_generator.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add type hints\r\n-- Fix potential issue with passing multi-image  conversations to processor\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-355\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-11T20:19:08Z"}
{"number": 1529, "title": "[tiny] Clean up datasets code", "files": ["src/oumi/core/configs/params/data_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\nRemove duplicate code and clean up comments\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-03-11T07:09:27Z"}
{"number": 1528, "title": "Qwen2.5 VL: Replace \"from source\" install with `transformers>=0.49`", "files": ["configs/recipes/vision/qwen2_5_vl_3b/sft/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-11T02:39:01Z"}
{"number": 1526, "title": "[Evaluation] Renaming `evaluation_platform` \u2192 `evaluation_backend`", "files": ["configs/projects/aya/evaluation/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/eval.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/llama3_3/evaluation/70b_eval.yaml", "configs/recipes/phi3/evaluation/eval.yaml", "configs/recipes/qwq/evaluation/eval.yaml", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_alpaca_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "src/experimental/configs/projects/zephyr/evaluation/eval.yaml", "src/oumi/core/configs/params/evaluation_params.py"], "area": "configs", "area_votes": {"configs": 21, "other": 1}, "body": "# Description\r\n\r\n\r\n\r\nRenaming `evaluation_platform` \u2192 `evaluation_backend` for compatibility with PR-1484. Now that we released a new PyPI package (0.1.8) we can start using the new field name in our yaml files (ie., `evaluation_backend`) that is compatible with the latest release. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1097\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-11T02:55:52Z"}
{"number": 1523, "title": "Added 3 Pixmo vision-language datasets", "files": ["src/oumi/datasets/__init__.py", "src/oumi/datasets/vision_language/__init__.py", "src/oumi/datasets/vision_language/pixmo_ask_model_anything.py", "src/oumi/datasets/vision_language/pixmo_cap.py", "src/oumi/datasets/vision_language/pixmo_cap_qa.py", "tests/integration/datasets/test_sft_vision_datasets_load_datasets.py", "tests/unit/datasets/test_pixmo.py"], "area": "data", "area_votes": {"data": 3}, "body": "# Description\r\n\r\n\r\n\r\nAdded 3 Pixmo vision-language datasets.\r\nSince some of the image urls return 404 and 429, the integration test was modified to only test a subset of the training set.\r\nFor reference, added the yaml files used to test manually using skypilot but these should be deleted.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1401\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-04-10T00:37:30Z"}
{"number": 1522, "title": "Evaluation: Inference optimizations", "files": ["src/oumi/core/evaluation/backends/alpaca_eval.py", "src/oumi/core/evaluation/evaluator.py", "tests/unit/core/evaluation/test_backend_alpaca_eval.py", "tests/unit/core/evaluation/test_evaluator.py"], "area": "evaluation", "area_votes": {"evaluation": 2}, "body": "# Description\r\n\r\n\r\n\r\nEnable re-using the same inference engine across all tasks. \r\n\r\n**_Exception_**: In the case of LM Harness: We need to destroy the engine, since LM Harness uses its own inference engine; a 2nd engine will allocate unnecessary resources, potentially increasing the memory footprint. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1097\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-11T01:42:02Z"}
{"number": 1518, "title": "Add QwQ full fine-tune and QLoRA configs", "files": ["configs/recipes/qwq/sft/full_gcp_job.yaml", "configs/recipes/qwq/sft/full_train.yaml", "configs/recipes/qwq/sft/lora_train.yaml", "configs/recipes/qwq/sft/qlora_gcp_job.yaml", "configs/recipes/qwq/sft/qlora_train.yaml"], "area": "configs", "area_votes": {"configs": 5}, "body": "# Description\r\n\r\nShould be similar to the existing LoRA config, with some modifications to the hyperparameters. Tested on GCP.\r\n\r\n## Related issues\r\n\r\nTowards #1408\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-03-10T19:16:07Z"}
{"number": 1517, "title": "Mark `BaseMapDataset` as `typing.Sized`", "files": ["src/oumi/core/datasets/base_map_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Type annotation change for clarity. Should be no-op otherwise..\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-06T19:46:44Z"}
{"number": 1516, "title": "Switch eval yaml configs to use evaluation_platform", "files": ["configs/projects/aya/evaluation/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/eval.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/llama3_3/evaluation/70b_eval.yaml", "configs/recipes/phi3/evaluation/eval.yaml", "configs/recipes/qwq/evaluation/eval.yaml", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_alpaca_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "src/experimental/configs/projects/zephyr/evaluation/eval.yaml", "src/oumi/core/configs/params/evaluation_params.py", "tests/unit/core/evaluation/test_save_utils.py"], "area": "configs", "area_votes": {"configs": 21, "other": 1}, "body": "# Description\r\n\r\n`evaluation_platform` was renamed to `evaluation_backend` throughout our codebase in #1484. This causes an error in all our eval configs at main, which use the YAML config from HEAD but the Oumi package from PyPi (which hasn't picked up this rename).\r\n\r\nThis PR:\r\n- Renames `evaluation_backend` back to `evaluation_platform` to fix the breakage\r\n- Updates `EvaluationTaskParams` so that `evaluation_backend` is not a required field. This will cause a breakage on main, as our configs don't provide this value anymore. Also updates post_init logic in this class to expect one of the two fields to be populated, not none or both.\r\n- Fixes a test\r\n\r\nAfter the next PyPI push, we can revert this change.\r\n\r\nTested that there's no errors both when using the PyPI package and using source.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1097\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-03-06T04:07:07Z"}
{"number": 1515, "title": "Add QwQ eval/infer configs", "files": ["configs/recipes/qwq/evaluation/eval.yaml", "configs/recipes/qwq/evaluation/gcp_job.yaml", "configs/recipes/qwq/inference/infer.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\nCopies the config from `configs/recipes/deepseek_r1/sft/distill_qwen_32b`, which is a reasoning model based on the same architecture (Qwen 2.5 32B).\r\n\r\nTested on GCP.\r\n\r\n## Related issues\r\n\r\nTowards #1408\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-03-05T23:39:31Z"}
{"number": 1514, "title": "Add Qwen QwQ Lora config", "files": ["configs/recipes/qwq/sft/lora_gcp_job.yaml", "configs/recipes/qwq/sft/lora_train.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\nCopies the config from `configs/recipes/deepseek_r1/sft/distill_qwen_32b`, which is a reasoning model based on the same architecture (Qwen 2.5 32B).\r\n\r\nTested on GCP.\r\n\r\n## Related issues\r\n\r\nTowards #1408\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-03-05T22:50:56Z"}
{"number": 1513, "title": "[Evaluation] Instantiating an inference engine (if needed) when running custom evaluations", "files": ["src/oumi/core/evaluation/evaluator.py", "tests/unit/core/evaluation/test_evaluator.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nInstantiating an inference engine (if needed) when running custom evaluations\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1097\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-06T02:08:26Z"}
{"number": 1510, "title": "Resolve warning about `--dispatch batches` deprecated param", "files": ["src/oumi/core/configs/params/training_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Use the new field accelerator_config \r\n-- Tested with `configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml` and `configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-985\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-04T22:34:32Z"}
{"number": 1509, "title": "Update sample GRPO script to validate num_generations", "files": ["configs/examples/grpo_tldr/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Check if global batch size is divisible by num_generations . Otherwise, training may fail (depending on `trl` version)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-993\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-04T21:41:57Z"}
{"number": 1508, "title": "Update docker image and build script", "files": ["Dockerfile", "scripts/docker/build_docker.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Fix syntax error with docker build script\r\n- Update to use the latest oumi pip package\r\n- Update build script to include a runtime test\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1495\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-05T01:46:25Z"}
{"number": 1506, "title": "[Evaluation] Save Utils: Moving, fixes, and unit tests", "files": ["src/oumi/core/evaluation/evaluator.py", "src/oumi/core/evaluation/utils/platform_prerequisites.py", "src/oumi/core/evaluation/utils/save_utils.py", "src/oumi/evaluation/__init__.py", "tests/unit/core/evaluation/test_save_utils.py"], "area": "evaluation", "area_votes": {"evaluation": 3}, "body": "# Description\r\n\r\n\r\n\r\nEvaluation | Save Utils\r\n- Moving 2 evaluation util files from `oumi/evaluation` to  `oumi/core/evaluation/utils`, since all the evaluation-related code has been moved there.\r\n- Fixing an issue in `save_utils` occurring when overwriting a pre-existing dir.\r\n- Adding unit tests for save_utils.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1097\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-04T02:35:19Z"}
{"number": 1505, "title": "Minor logging improvements", "files": ["src/oumi/core/distributed.py", "src/oumi/train.py"], "area": "training", "area_votes": {"training": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1107\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-04T00:48:26Z"}
{"number": 1504, "title": "Update the GPU Tests badge to use results from main", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nAfter manually running our GPU tests, I've updated the test badge to use results only from the `main` branch.\r\n\r\nYou can preview the badge here: https://github.com/oumi-ai/oumi/actions/workflows/gpu_tests.yaml/badge.svg?branch=main\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-03T23:26:54Z"}
{"number": 1503, "title": "Add a schedule for our GPU, CPU, and doc tests", "files": [".github/workflows/ci_tests.yaml", ".github/workflows/doctests.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 4}, "body": "# Description\r\n\r\n\r\n\r\nIn #1498 I overlooked that subactions would not be listed in the results page for their respective actions. This means that while my CI action ran all tests, they weren't reported for GPU and CPU tests, respectively.\r\n\r\nThis PR adds a simple workaround, just scheduling GPU, CPU, and doc tests to run once every 8 hours. Additionally, I've enabled the workflow so it can be triggered manually when needed.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-03T23:05:54Z"}
{"number": 1500, "title": "Support for deprecated input param (` evaluation_platform`)", "files": ["src/oumi/core/configs/params/evaluation_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\nSupport for deprecated input param (` evaluation_platform`) in order to not break any users that have taken a dependency on this. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1097\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-03T21:07:02Z"}
{"number": 1498, "title": "Add recurring tests to keep our test badges updated.", "files": [".github/workflows/ci_tests.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 3}, "body": "# Description\r\n\r\n\r\n\r\nThis PR adds a new `ci_tests` workflow that triggers both our CPU and GPU tests in parallel on the `main` branch.\r\n\r\nCurrently our test badges in our README reflect the latest tests run from pull requests. This means that problematic PRs will cause our tests to be listed as failing even when tests pass on `main`.\r\n\r\nFollow up work:\r\n- Update our badges to look for only tests run on the `main` branch.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-03T22:40:24Z"}
{"number": 1496, "title": "Fix `log_samples` not propagating from `eval_kwargs`", "files": ["src/oumi/core/evaluation/backends/lm_harness.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "Fix issue where `log_samples` kwarg is not able to be passed to lm_harness\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-03T16:19:56Z"}
{"number": 1494, "title": "Update documentation formatting for BaseModel", "files": ["src/oumi/core/models/base_model.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\nThe docstring for the forward method was malformed.\r\n\r\nBefore:\r\n![Screenshot 2025-03-01 at 9 53 11\u202fAM](https://github.com/user-attachments/assets/9342741c-4c40-4be7-bf15-9188b57f91ae)\r\n\r\nAfter:\r\n![Screenshot 2025-03-01 at 9 52 59\u202fAM](https://github.com/user-attachments/assets/412d2054-f110-49f0-90c1-c79f5c813440)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-02T23:38:58Z"}
{"number": 1493, "title": "Fix chat template issue for nested content parts used for VLMs", "files": ["src/oumi/datasets/chat_templates/llama3-instruct.jinja", "src/oumi/datasets/chat_templates/llava.jinja", "src/oumi/datasets/chat_templates/phi3-instruct.jinja", "src/oumi/datasets/chat_templates/qwen2-vl-instruct.jinja", "src/oumi/inference/vllm_inference_engine.py", "tests/unit/datasets/test_chat_templates.py"], "area": "data", "area_votes": {"data": 4, "inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update chat templates to use `content` or `text` field in `ContentItem`, depending on what's available  to make it compatible with Oumi and OpenAI conversations\r\n-- Revert temporary hacks in VLLM inference engines introduced in https://github.com/oumi-ai/oumi/pull/1486\r\n\r\nTested with VLLM inference engines. All chat template related unit tests pass. \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-1090\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-03-01T00:25:52Z"}
{"number": 1492, "title": "Minor updates to `oumi env`", "files": ["src/oumi/cli/env.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Include `CUDA_VISIBLE_DEVICES` and `sglang` version into env report.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-28T00:38:11Z"}
{"number": 1490, "title": "Define `GrpoParams` under configs", "files": ["configs/examples/grpo_tldr/gcp_job.yaml", "configs/examples/grpo_tldr/train.yaml", "src/oumi/core/configs/__init__.py", "src/oumi/core/configs/params/grpo_params.py", "src/oumi/core/configs/params/training_params.py"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Include initial set of params including  vLLM-related params.\r\n-- Tested vLLM completions on 1 GPU.  Distributed  multi-GPU setup still needs work.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-993\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-27T17:18:04Z"}
{"number": 1488, "title": "[tiny] Update async_eval.yaml comments to reference correct class", "files": ["configs/recipes/gpt2/evaluation/async_eval.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-27T02:15:01Z"}
{"number": 1487, "title": "Fix a bug where overriding remote_params fails via the CLI (oumi infer)", "files": ["src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nDuring inference, `oumi infer` would fail if the user specified any remote_params via the CLI for wrapper engines (engines that inherit from the RemoteInferenceEngine). This is because the merged config specified at inference time had `None` values for required API fields.\r\n\r\nUpdated the infer method to gracefully fallback to previous values if `None` values are detected at runtime.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-27T05:28:28Z"}
{"number": 1485, "title": "Add sample DDP/GCP config for GRPO trainer", "files": ["configs/examples/grpo_tldr/gcp_job.yaml", "configs/examples/grpo_tldr/train.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-993\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-26T20:52:27Z"}
{"number": 1483, "title": "Update oumi infer to fall back to interactive mode if no input path is specified.", "files": ["src/oumi/cli/infer.py", "tests/unit/cli/test_cli_infer.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nThis is a Quality of Life change for the CLI. If the user doesn't specify interactive inference AND doesn't specify an input path, we'll default to interactive inference.\r\n\r\nAdditionally, we'll always prevent interactive inference if the user specifies an input file. This is in preparation for removing this flag. We should instead use the presence of the `input_file` as an indicator for whether or not we run interactive inference.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-26T02:42:09Z"}
{"number": 1480, "title": "Add notebook for fine-tuning MiniMath-R1-1.5B", "files": ["notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - MiniMath-R1-1.5B.ipynb"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nAdd notebook outlining the training and evaluation steps for MiniMath-R1-1.5B\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-25T00:34:36Z"}
{"number": 1479, "title": "Create an inference config for Claude Sonnet 3.7", "files": ["configs/apis/anthropic/infer_claude3_7.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\nRunnabled by anyone, anywhere, with the command:\r\n\r\n`oumi infer -c oumi://configs/api/anthropic/infer_claude3_7.yaml -i`\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-24T23:44:47Z"}
{"number": 1478, "title": "[tiny]Update trl to 0.14", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nThis is needed for the GRPO trainer., which was added in 0.14.0. We'd ideally like to upgrade to the latest, 0.15.1, which seems to have a decent number of updates to GRPOTrainer. However, 0.15.0 changes the SFTTrainer to only accept HF datasets or a ConstantLengthDataset. Our custom PyTorch datasets thus no longer work, and additional time is needed to do a migration to fix this.\r\n\r\nRelease history:\r\n- 0.13.0: 12/16/24\r\n- 0.14.0: 1/29/25\r\n- 0.15.0: 2/13/25\r\n- 0.15.1: 2/18/25\r\n\r\nTRL release notes: https://github.com/huggingface/trl/releases. At a glance, there doesn't seem to be any other breaking changes for this version update.\r\n\r\nTested by running `configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml` and `configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml` as a sanity check.\r\n\r\n## Related issues\r\n\r\nTowards #1351\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-25T00:44:59Z"}
{"number": 1476, "title": "Fix local models to not break the registry.", "files": ["src/oumi/core/registry/registry.py", "tests/unit/core/test_registry.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\nPreviously we were loading external deps using an arbitrary module name. This had the benefit of preventing of preventing module conflicts, but the downside of python not being able to recognize when a module was already imported.\r\n\r\nFor custom models, this led to a scenario where a module would be imported twice, causing python to try to register the model 2x with Oumi. This is currently a fatal error, and crashes out.\r\n\r\nThe new change temporarily appends the directory of the python module to the user's sys path (this is reset when the python session ends). This means local references to the module will now resolve properly. The main downside here is that users must be careful that their *.py files don't conflict with any other packages they need (like `pathlib`, `time`, etc).\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1475\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-24T19:43:33Z"}
{"number": 1472, "title": "Update the RemoteInferenceEngine to appropriately handle openai format batch prediction endpoints.", "files": ["src/oumi/inference/gemini_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_gemini_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Description\r\n\r\n\r\n\r\nIf a user specifies an API endpoint that isn't the base endpoint (e.g. `https://api.openai.com/v1`) batch prediction will fail. Unfortunately, our wrapper engines like the OpenAIInferenceEngine specify paths of the form: `https://api.openai.com/v1/chat/completions`.\r\n\r\nAdditionally, not every engine supports batch requests (such as gemini). Updated the Gemini engine to simply return an error for `infer_batch`.\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1465\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-23T02:58:02Z"}
{"number": 1468, "title": "Updated all CLI endpoints to support oumi:// prefix", "files": ["src/oumi/cli/evaluate.py", "src/oumi/cli/judge.py", "src/oumi/cli/launch.py", "src/oumi/cli/train.py", "tests/unit/cli/test_cli_evaluate.py", "tests/unit/cli/test_cli_infer.py", "tests/unit/cli/test_cli_judge.py", "tests/unit/cli/test_cli_launch.py", "tests/unit/cli/test_cli_train.py"], "area": "cli", "area_votes": {"cli": 4}, "body": "# Description\r\n\r\n\r\n\r\nEach CLI entrypoint calls the ```cli_utils.fetch_and_resolve_config``` function before proceeding with the existing logic. This resolves the path if it includes ```oumi://``` prefix and otherwise returns the existing path unchanged.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1444 \r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n@taenin\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-28T23:46:38Z"}
{"number": 1466, "title": "Enable pre-release install for uv in pyproject.toml", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nUnlike pip, uv disallows prelease packages by default, which we need for `omegaconf==2.4.0dev3` for `oumi>=0.1.5`. This PR fixes that in our pyproject.toml.\r\n\r\n## Related issues\r\n\r\nFixes #1462\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2025-02-22T02:23:59Z"}
{"number": 1464, "title": "Remove a temp workaround in `pad_sequences` on the left side", "files": ["src/oumi/utils/torch_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Use built-in param after migration to `pytorch 2.5`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-644\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-21T01:23:03Z"}
{"number": 1463, "title": "Minor cleanup of oumi fetch", "files": ["src/oumi/cli/cli_utils.py", "src/oumi/cli/fetch.py", "src/oumi/cli/infer.py", "tests/unit/cli/test_cli_fetch.py", "tests/unit/cli/test_cli_utils.py"], "area": "cli", "area_votes": {"cli": 3}, "body": "# Description\r\n\r\n\r\n\r\nI've slightly refactored the `oumi fetch` logic to live in `cli_utils` to make testing a bit easier.\r\nI've also removed the `output_dir` argument from the `oumi infer` cli as it was unclear that it was for downloading configs. \r\n\r\nI also updated `oumi fetch` to fetch configs if the user omits the `oumi://` prefix.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nRelated to #1386\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-20T23:28:19Z"}
{"number": 1454, "title": "[Tiny] Update default training params for Qwen2-VL-2B-Instruct", "files": ["configs/recipes/vision/qwen2_vl_2b/inference/infer.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\nShort experiments when training `Qwen2-VL-2B-Instruct` with `merve/vqav2-small` hint that decreasing the `max_grad_norm` results in more stable training, and increasing the  `weight decay` reduces overfitting.  \r\n\r\nWe update these two hyper-parameters here.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # Towards OPE-946\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-20T05:34:04Z"}
{"number": 1453, "title": "[Tiny] Add more warnings for \"special\" requirements of Qwen2.5-VL", "files": ["configs/recipes/vision/qwen2_5_vl_3b/README.md", "configs/recipes/vision/qwen2_5_vl_3b/inference/infer.yaml", "configs/recipes/vision/qwen2_5_vl_3b/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/train.yaml"], "area": "configs", "area_votes": {"docs": 1, "configs": 3}, "body": "# Description\r\n\r\nAdds explicit user warnings on the configuration and readme files regarding `Qwen2.5-VL-3B` and its dependency on the latest dev `transformers` version.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # Towards: OPE-988\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-20T05:50:11Z"}
{"number": 1451, "title": "Update oumi launch status to show clusters with no running jobs.", "files": ["src/oumi/cli/launch.py", "tests/unit/cli/test_cli_launch.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\nPreviously, running `oumi launch status` would omit running clusters if they had no jobs available.\r\n\r\nThis change lists running clusters, regardless of their child job status.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-19T23:15:43Z"}
{"number": 1450, "title": "Renamed CALM to CoALM", "files": ["configs/projects/coalm/405b_train.yaml", "configs/projects/coalm/70b_infer.yaml", "configs/projects/coalm/70b_train.yaml", "configs/projects/coalm/8b_infer.yaml", "configs/projects/coalm/8b_train.yaml", "configs/projects/coalm/README.md", "configs/projects/coalm/images/dataset.png", "configs/projects/coalm/images/results.png"], "area": "configs", "area_votes": {"configs": 7, "docs": 1}, "body": "# Description\n\n\n\nCALM was renamed to CoALM.\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-02-19T22:35:54Z"}
{"number": 1449, "title": "Add support for Docmatix dataset to multimodal training script", "files": ["scripts/benchmarks/minimal_multimodal_training.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- The dataset includes multi-image examples, good for testing this feature.\r\n-- Add support for `--subset` param to the script.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-355\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-19T23:00:56Z"}
{"number": 1443, "title": "Require an inference config for oumi infer.", "files": ["src/oumi/cli/cli_utils.py", "src/oumi/cli/fetch.py", "src/oumi/cli/infer.py", "tests/unit/cli/test_cli_fetch.py", "tests/unit/cli/test_cli_infer.py", "tests/unit/cli/test_cli_utils.py"], "area": "cli", "area_votes": {"cli": 3}, "body": "# Description\r\n\r\n\r\n\r\nMake the config parameter required for oumi infer.\r\n\r\nAlso updated the `oumi fetch` default directory to `~/.oumi/fetch`. Updated tests accordingly.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1442\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-19T01:01:54Z"}
{"number": 1441, "title": "Create a script to calculate memory used during training", "files": ["scripts/memcalc.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nThis is a Python script that will estimate the memory needed to run training. This is derived from the memory calculator Google Sheet Jeremy work. In the future, this can be integrated as a subcommand for the CLI, but leaving it as a separate script for now to reduce visibility as it's a WIP. The initial implementation works for GPT2.\r\n\r\n## Related issues\r\n\r\nTowards OPE-672\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-05-16T17:51:44Z"}
{"number": 1439, "title": "Added fetch command and modified infer command to resolve oumi://", "files": ["src/oumi/cli/cli_utils.py", "src/oumi/cli/fetch.py", "src/oumi/cli/infer.py", "src/oumi/cli/main.py", "tests/unit/cli/test_cli_fetch.py", "tests/unit/cli/test_cli_main.py", "tests/unit/cli/test_cli_utils.py"], "area": "cli", "area_votes": {"cli": 4}, "body": "# Description\r\n\r\n\r\n\r\nA new CLI endpoint ```fetch``` has been added to allows users to directly download configs from the github repo. The same function is then reused to allow the resolution of ```oumi://``` to download a the config file from the github repo and provide the address of this newly downloaded config file for further use by the ```infer``` command.\r\n\r\nA test has been added in the ```tests_cli_main.py``` to check for successful registration of the ```fetch``` entry-point.\r\n\r\nA new test file has been added named ```tests_cli_fetch.py``` that has three tests that cover the cases of default directory, explicit ```--output-dir``` passing and passing output directory via ```OUMI_DIR``` environment variable.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1374 (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-19T00:08:32Z"}
{"number": 1437, "title": "Set a better default for vllm inference GPU usage.", "files": ["src/oumi/inference/vllm_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nSet the local VLLM inference to use 90% of GPU vRAM.\r\n\r\n100% usage causes OOMs when small amounts of vRAM are witheld.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-16T23:39:11Z"}
{"number": 1435, "title": "Create `pad_to_max_dim_and_stack()` function in torch_utils", "files": ["src/oumi/utils/torch_utils.py", "tests/unit/utils/test_torch_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- The function pads input tensors to maximum size in each dimension, then stacks them\r\n-- This is similar to `pad_sequences()` except it can pad to more than one dimension, which is needed e.g., for `cross_attention_mask` which has two variable dimensions: `seq_len`, `num_images` for multi-image multimodal examples\r\n-- The integration of this new function into data pipeline is coming in a separate PR.\r\n-- Minor error logging improvements in padding utils\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-355\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-15T00:42:35Z"}
{"number": 1433, "title": "WildChat-50M Reproduction", "files": ["configs/projects/wc50m/README.md", "configs/projects/wc50m/configs/base_ultrachat.yaml", "configs/projects/wc50m/configs/gcp_base_ultrachat.yaml", "configs/projects/wc50m/results/baseline.csv", "configs/projects/wc50m/results/oumi.csv"], "area": "configs", "area_votes": {"docs": 1, "configs": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\nAdding a project containing the configs necessary to reproduce the UltraChat baseline from the WildChat-50M paper. Resolving https://linear.app/oumi/issue/OPE-1050.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-14T16:42:24Z"}
{"number": 1430, "title": "Enable ability to override list values in config via CLI", "files": ["docs/cli/commands.md", "docs/get_started/core_concepts.md", "docs/get_started/quickstart.md", "src/oumi/core/configs/base_config.py", "tests/unit/core/configs/test_config.py"], "area": "docs", "area_votes": {"docs": 3, "configs": 1}, "body": "# Description\r\n\r\n## Problem\r\n\r\nPreviously, we could not override config values that were within a list field via the CLI, ex. `oumi evaluate -c configs/recipes/smollm/evaluation/135m/eval.yaml --tasks[0].num_samples=1`, getting the error `omegaconf.errors.ConfigTypeError: Cannot merge DictConfig with ListConfig`. This is not a problem with using the wrong CLI syntax, but instead with how we were parsing the CLI overrides.\r\n\r\nCurrently, we create a config from the args list with `OmegaConf.from_cli(arg_list)`, and then merge it with the YAML config via `OmegaConf.merge(*all_configs)`. The YAML config is an OmegaConf structured config, meaning it's aware of the typing of our Config object, `EvaluationConfig` in this example. The problem is that the config from args list is created before it's merged with the YAML config, so it is not aware of the typing. In the above example with `--tasks[0].num_samples=1`, it's thus not aware that `tasks` is a List, and creates a dict instead: `{\"tasks\": {\"0\": {\"num_samples\": 1}}}`.\r\n\r\n## Solution\r\n\r\nThe solution is simply to call `config.merge_with_dotlist(arg_list)` instead, which merges each argument into the config individually. With this method, the `0` is properly interpreted as a list index instead of a dict key.\r\n\r\nI've tested that this does not impact existing behavior. I've also verified that many variants of specifying overrides, which are Pythonic and we expect to work, do work. Here's some example args for the config mentioned in the first example:\r\n\r\n```\r\n# Bracket notation\r\n--tasks[0].num_samples=1\r\n# Dot notation\r\n--tasks.0.num_samples=null\r\n# Override existing key in dict\r\n--tasks.0.eval_kwargs.num_fewshot=1\r\n# Add new key in dict\r\n--tasks.0.eval_kwargs.foo=bar\r\n# Merge dict values within the list entirely\r\n# Note this doesn't override the dict, as OmegaConf merges dicts by default\r\n--tasks.0.eval_kwargs=\"{'foo':'bar', 'baz':2}\"\r\n\r\n```\r\n\r\nNOTE: This PR doesn't support the ability to add new elements in the list. This is currently not supported by OmegaConf: https://github.com/facebookresearch/hydra/issues/1547#issuecomment-1235918733\r\n\r\n## Related issues\r\n\r\nFixes OPE-934\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2025-02-19T22:05:45Z"}
{"number": 1429, "title": "Provide example show to start SGLang server using Docker", "files": ["configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/1b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/3b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/sglang_infer.yaml"], "area": "configs", "area_votes": {"configs": 5}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Docker should be the recommended way to start SGLang, as Docker pulls pre-tested server configurations and avoids Python dependency conflicts.\r\n-- Tested locally\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-989\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-13T16:49:14Z"}
{"number": 1428, "title": "Update VLM sample `oumi infer -i` commands", "files": ["configs/recipes/vision/llama3_2_vision/inference/11b_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_rvllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml", "configs/recipes/vision/llava_7b/inference/infer.yaml", "configs/recipes/vision/llava_7b/inference/vllm_infer.yaml", "configs/recipes/vision/phi3/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_5_vl_3b/inference/infer.yaml", "configs/recipes/vision/qwen2_5_vl_3b/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/sglang_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/vllm_infer.yaml"], "area": "configs", "area_votes": {"configs": 12}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Include `--image` param, which is required to make these samples meaningful\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-13T01:56:52Z"}
{"number": 1425, "title": "Add calm recipe.", "files": ["configs/projects/calm/405b_train.yaml", "configs/projects/calm/70b_infer.yaml", "configs/projects/calm/70b_train.yaml", "configs/projects/calm/8b_infer.yaml", "configs/projects/calm/8b_train.yaml"], "area": "configs", "area_votes": {"configs": 5}, "body": "# Description\r\n\r\n\r\n\r\nAdd a recipe for the CALM model.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-12T21:20:23Z"}
{"number": 1423, "title": "Update notebooks to run on Colab", "files": ["notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 9}, "body": "# Description\r\n\r\nUpdate notebooks so they run on Colab (ex. that they contain all required setup instructions). Removed Colab link from the top for those that don't work on Colab.\r\n\r\nExample changes:\r\n- Add missing installs that are needed if starting from a clean env\r\n- Give options for smaller models/data to allow the notebook to complete in a reasonable time\r\n- Don't depend on configs in the repo\r\n\r\n## Related issues\r\n\r\nFixes OPE-1000\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-12T19:57:56Z"}
{"number": 1414, "title": "[Evaluation] LM Harness remote server support", "files": ["src/oumi/evaluation/lm_harness.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\nEnabling our LM Harness integration to support a remote/local server.\r\n\r\nTested as follows:\r\n1) Installed an LM Harness requirement locally\r\n```\r\npip install lm-eval[api]\r\n```\r\n2) Launched a basic vLLM server:\r\n```\r\npython -m vllm.entrypoints.openai.api_server \\\r\n    --model HuggingFaceTB/SmolLM2-135M-Instruct \\\r\n    --port 6864\r\n```\r\n3) executed the following `YAML` code\r\n```\r\nmodel:\r\n  model_name: \"HuggingFaceTB/SmolLM2-135M-Instruct\"\r\n  model_max_length: 2048\r\n  torch_dtype_str: \"bfloat16\"\r\n  trust_remote_code: True\r\n\r\ntasks:\r\n  - evaluation_platform: lm_harness\r\n    task_name: mmlu_college_computer_science\r\n\r\ninference_engine: REMOTE\r\ninference_remote_params:\r\n  api_url: \"http://localhost:6864/v1/completions\"\r\n  num_workers: 1\r\n  max_retries: 3\r\n  connection_timeout: 300\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1005\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-12T01:14:00Z"}
{"number": 1413, "title": "Create a script to save `Conversation`-s from SFT datasets into  `.jsonl` file", "files": ["scripts/datasets/save_conversations.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Create a simple script to save `Conversation`-s from SFT datasets into  `.jsonl` file\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-355\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-08T04:13:23Z"}
{"number": 1411, "title": "Add requirements header to configs and clean them up", "files": ["configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/examples/misc/hello_world_gcp_job.yaml", "configs/examples/misc/hello_world_polaris_job.yaml", "configs/examples/misc/vllm_polaris_job.yaml", "configs/projects/aya/evaluation/eval.yaml", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/chatqa/chatqa_stage1_train.yaml", "configs/projects/chatqa/chatqa_stage2_train.yaml", "configs/projects/chatqa/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_train.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/gpt2/inference/infer.yaml", "configs/recipes/gpt2/pretraining/gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/inference/70b_infer.yaml", "configs/recipes/llama3_1/inference/8b_infer.yaml", "configs/recipes/llama3_1/inference/8b_rvllm_infer.yaml", "configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/polaris_job.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/llama3_2/inference/1b_infer.yaml", "configs/recipes/llama3_2/inference/1b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/1b_vllm_infer.yaml", "configs/recipes/llama3_2/inference/3b_infer.yaml", "configs/recipes/llama3_2/inference/3b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/3b_vllm_infer.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/llama3_3/evaluation/70b_eval.yaml", "configs/recipes/llama3_3/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_3/inference/70b_infer.yaml", "configs/recipes/llama3_3/inference/70b_vllm_infer.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml", "configs/recipes/phi3/dpo/gcp_job.yaml", "configs/recipes/phi3/dpo/mac_train.yaml", "configs/recipes/phi3/dpo/nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/nvidia_80g_train.yaml", "configs/recipes/phi3/dpo/train.yaml", "configs/recipes/phi3/evaluation/eval.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/phi3/sft/lora_train.yaml", "configs/recipes/phi3/sft/mac_lora_train.yaml", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job.yaml", "configs/recipes/smollm/inference/135m_infer.yaml", "configs/recipes/smollm/sft/135m/train.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_rvllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/train.yaml", "configs/recipes/vision/llava_7b/inference/infer.yaml", "configs/recipes/vision/llava_7b/inference/vllm_infer.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/inference/vllm_infer.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/inference/infer.yaml", "configs/recipes/vision/qwen2_5_vl_3b/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/gcp_job.yaml", "configs/recipes/vision/qwen2_5_vl_3b/sft/train.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/sglang_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml", "docs/user_guides/evaluate/leaderboards.md", "docs/user_guides/launch/launch.md", "scripts/datasets/pretokenize/sky.yaml", "src/experimental/configs/projects/zephyr/evaluation/eval.yaml", "src/experimental/configs/projects/zephyr/sft/full_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/full_train.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_train.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_nvidia_24g_train.yaml", "tests/scripts/gcp_e2e_tests_job.yaml"], "area": "configs", "area_votes": {"configs": 170, "docs": 2, "infra": 1, "other": 7}, "body": "# Description\r\n\r\nThis adds a Requirements section to the top of all configs. This explicitly lists out the steps required to get the config to work, aside from installing Oumi.\r\n\r\nComprehensive list of all requirements added:\r\n- Log into HF for configs requiring a HF token\r\n- Log into WandB if needed\r\n- Set up Skypilot GCP for GCP configs\r\n- Set up ALCF and the username for Polaris configs, and call out that Polaris is only for the Oumi core team (to not confuse users)\r\n- Request access to required gated models/datasets (ex. Llama)\r\n- Install relevant packages (SGLang, flash-attn, vLLM) for non-job configs (ex. SGLang inference configs)\r\n- Optionally mount buckets if we mention it in the config\r\n\r\nMiscellaneous config comments cleanup:\r\n- Rename my-oumi-cluster to something unique to prevent collisions\r\n- Delete redundant comments\r\n- Fix comment on where Polaris working_dir is mounted\r\n- Create more detailed description for files missing it.\r\n\r\nConfig cleanup:\r\n- Delete `adapter_model: null`. This has no functional effect.\r\n\r\nFor reviewers: This PR only updates comments, aside from deleting `adapter_model: null` which is a no-op. Feel free to take a look at just a few files to get the general idea; no need to review them all. I manually double-checked every change in this PR.\r\n\r\n## Related issues\r\n\r\nFixes OPE-1022\r\nFixes OPE-1001\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-11T19:06:27Z"}
{"number": 1406, "title": "Add a Slurm cluster and cloud to the oumi launcher.", "files": ["src/oumi/cli/env.py", "src/oumi/launcher/clients/slurm_client.py", "src/oumi/launcher/clouds/__init__.py", "src/oumi/launcher/clouds/slurm_cloud.py", "src/oumi/launcher/clusters/slurm_cluster.py", "tests/unit/launcher/clients/data/sacct_full.txt", "tests/unit/launcher/clients/test_slurm_client.py", "tests/unit/launcher/clouds/test_slurm_cloud.py", "tests/unit/launcher/clusters/test_slurm_cluster.py"], "area": "launcher", "area_votes": {"cli": 1, "launcher": 3}, "body": "# Description\r\n\r\n\r\n\r\nThis PR adds a cluster and cloud implementation for slurm.\r\n\r\nIn oumi, we'll consider a \"cluster\" a logical pair of a `hostname` and a `user`, ex `taenin@myslurmhost`.\r\n\r\nThis means that running jobs as a second user on the same host would be represented with a second cluster in oumi. Ex: `janedoe@myslurmhost`\r\n\r\nUsers can set the `OUMI_SLURM_CONNECTIONS` env var to automatically connect to specific slurm hosts in the oumi CLI. This should be a comma-separated list of @ pairs.\r\n\r\nEx:\r\n```\r\nexport OUMI_SLURM_CONNECTIONS=\"taenin@myslurmhost,janedoe@myslurmhost\"\r\n```\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #1382\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-07T19:20:08Z"}
{"number": 1404, "title": "[tiny] Remove references to missing job configs in README", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nWe don't have quickstart job configs for the other cloud providers. Instead of adding them, I opted to override the cloud value instead:\r\n- The configs would have high overlap\r\n- This demonstrates how simple it is to switch clouds from the CLI\r\n- Takes up less real estate on the README page\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-07T01:32:36Z"}
{"number": 1402, "title": "Fix broken links in notebooks.", "files": ["notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 14}, "body": "# Description\r\n\r\n\r\n\r\nInspired by #1398, this PR fixes other `hhttp` typos.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-06T23:51:27Z"}
{"number": 1398, "title": "Fixed broken link in Oumi - A Tour.ipynb notebook", "files": ["notebooks/Oumi - A Tour.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "Fixed broken/incorrect link for \"training\" in introduction paragraph.\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\nI removed an extra \"h\" in the \"https\" for the \"training\" link.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-06T23:17:17Z"}
{"number": 1397, "title": "Remove `Docmatix` dataset references from docstrings VLM config examples", "files": ["configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- The use of the large dataset may require some extra infra work. Let's exclude it from examples for now.\r\n-- Comments-only change\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-1021\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-06T22:14:41Z"}
{"number": 1390, "title": "Remove uneeded env vars from job configs", "files": ["configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/smollm/sft/135m/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 30, "other": 1}, "body": "# Description\r\n\r\nACCELERATE_LOG_LEVEL and TOKENIZERS_PARALLELISM are set in https://github.com/oumi-ai/oumi/blob/06a6ad4296ecbb329b815285d6c152835f46c7f5/src/oumi/cli/cli_utils.py#L97. We can remove them from our configs to reduce fluff.\r\n\r\n## Related issues\r\n\r\nTowards OPE-1022\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-06T11:00:26Z"}
{"number": 1389, "title": "Create a client for communicating with a Slurm node via SSH.", "files": ["src/oumi/launcher/clients/slurm_client.py", "tests/unit/launcher/clients/data/sacct.txt", "tests/unit/launcher/clients/test_slurm_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description\r\n\r\n\r\n\r\nEstablish a client for communicating with a Slurm head node. This is the first PR in a series that will add Slurm support in `oumi launch`.\r\n\r\nWe launch jobs with\r\n- `sbatch ...`\r\n\r\nList jobs with\r\n- `sacct ...`\r\n\r\nAnd cancel jobs with\r\n- `scancel ...`\r\n\r\nVerified on our local Slurm cluster.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards #1382\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-07T00:11:06Z"}
{"number": 1388, "title": "Remove transformer version override for `HuggingFaceTB/SmolVLM-Instruct` in launcher script", "files": ["configs/recipes/vision/smolvlm/sft/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- No longer needed after  recent `transformers` upgrade\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-697\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-06T18:46:50Z"}
{"number": 1387, "title": "Cleanup `HuggingFaceM4/Docmatix` and `HuggingFaceM4/the_cauldron` multimodal datasets", "files": ["src/oumi/datasets/vision_language/__init__.py", "src/oumi/datasets/vision_language/docmatix.py", "src/oumi/datasets/vision_language/the_cauldron.py", "tests/integration/datasets/test_sft_vision_datasets_load_datasets.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add `DocmatixDataset` to  module's `__init__.py` file\r\n-- Make `DocmatixDataset` a subclass of `TheCauldron` for code reuse since they share data format\r\n-- Add `DocmatixDataset` to integration tests, and make them pass.\r\n-- Allow multiple image and only include images in the first turn of multi-turn conversations. \r\n\r\nSee also: https://github.com/oumi-ai/oumi/issues/916, https://github.com/oumi-ai/oumi/issues/915\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-748, OPE-747\r\nTowards OPE-1021, OPE-355\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-06T02:29:35Z"}
{"number": 1385, "title": "Update our FAQ for tips about installing oumi on Windows", "files": ["docs/faq/troubleshooting.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdd several errors users have encountered on windows.\r\nInstalling in WSL resolves these issues.\r\n\r\nThanks for the reports in Discord!\r\n\r\n![image](https://github.com/user-attachments/assets/5fa0639a-ce27-46bf-a99e-6d2c02a56ac5)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-05T22:43:45Z"}
{"number": 1384, "title": "Upgrade omegaconf to 2.4.0dev3", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\nThe last stable version of omegaconf is two years old and is causing dependency issues with azure. It's in maintenance mode, and there's no certainty on when the next stable version will release, so we'll upgrade to the dev version for now (which is a year old).\r\n\r\nTested that `pip install '.[azure]'` works in a fresh conda env, and that a simple training job works.\r\n\r\n## Related issues\r\n\r\nFixes #1377\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-05T21:23:54Z"}
{"number": 1383, "title": "Add proper labels and types to new Bugs and Feature Requests.", "files": [".github/ISSUE_TEMPLATE/bug-report.yaml", ".github/ISSUE_TEMPLATE/feature-request.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\nBugs and Feature Requests should be tagged with the appropriate types and labels.\r\n\r\nAdded the `triage` label to all newly logged issues.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-05T20:47:36Z"}
{"number": 1380, "title": "Update notebooks to improve their Colab experience", "files": ["README.md", "docs/index.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 17}, "body": "# Description\r\n\r\n- Add a link to open each notebook in Colab at the top. This is in case users encounter the notebook file in GitHub/VSCode as opposed to our docs.\r\n- Add commands to install Oumi to notebooks missing it.\r\n- Add instructions to add the HF token and sign the Llama agreement to notebooks missing it.\r\n- For notebooks requiring GPUs to run on Colab, add a notice to switch to the T4 GPU runtime. Note that some notebooks don't need to do this, ex. for the Oumi launcher.\r\n\r\nFuture work:\r\n- Test all notebooks to make sure they run on Colab. Some notebooks (ex. vLLM inference) will need modification to do so.\r\n- [minor] Fix header sizes, which are inconsistent across notebooks\r\n\r\n## Related issues\r\n\r\nTowards OPE-1000\r\nFixes OPE-998\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-05T19:40:01Z"}
{"number": 1378, "title": "Move code to disable caching in `model.config` to a helper function", "files": ["src/oumi/builders/models.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Move code to disable caching in `model.config` to a helper function for reuse\r\n-- Add an  additional snippet to also disable cache in `model.config.text_config`, which is often done for VLMs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-875\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-05T05:25:23Z"}
{"number": 1371, "title": "Update the link for the trending banner.", "files": ["README.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nLinking to https://trendshift.io/repositories/12865, the source of the banner.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-04T20:41:39Z"}
{"number": 1370, "title": "Have GitHub Trending image hyperlink to GitHub Trending page", "files": ["README.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\nBuilds on #1340. Tested on both staging docs and README\r\n\r\n## Related issues\r\n\r\nN/A\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-04T18:48:59Z"}
{"number": 1367, "title": "[nit] update default issue names", "files": [".github/ISSUE_TEMPLATE/bug-report.yaml", ".github/ISSUE_TEMPLATE/feature-request.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- update default issue names\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-04T17:26:25Z"}
{"number": 1366, "title": "Support HuggingFaceM4/the_cauldron dataset", "files": ["src/oumi/datasets/vision_language/__init__.py", "src/oumi/datasets/vision_language/the_cauldron.py", "tests/integration/datasets/test_sft_vision_datasets_load_datasets.py"], "area": "data", "area_votes": {"data": 1}, "body": "Related to #916\n\nAdd support for `HuggingFaceM4/the_cauldron` dataset for vision-language tasks.\n\n* **New Dataset Class**:\n  - Add `TheCauldronDataset` class in `src/oumi/datasets/vision_language/the_cauldron.py` to support `HuggingFaceM4/the_cauldron` dataset.\n  - Implement `transform_conversation` method to handle image and text data.\n  - Register the dataset using `@register_dataset` decorator.\n\n* **Initialization**:\n  - Import `TheCauldronDataset` class in `src/oumi/datasets/vision_language/__init__.py`.\n  - Add `TheCauldronDataset` to the `__all__` list.\n\n* **Documentation**:\n  - Add a section for `HuggingFaceM4/the_cauldron` dataset in `docs/resources/datasets/vl_sft_datasets.md`.\n  - Mention `TheCauldronDataset` as a supported vision-language dataset.\n  - Provide usage examples for both YAML configuration and Python API.\n\n", "merged_at": "2025-02-05T21:39:51Z"}
{"number": 1357, "title": "Adopt new Llama 3.1 HF names", "files": ["configs/examples/misc/vllm_polaris_job.yaml", "configs/projects/aya/evaluation/eval.yaml", "configs/projects/aya/sft/train.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/inference/70b_infer.yaml", "configs/recipes/llama3_1/inference/8b_infer.yaml", "configs/recipes/llama3_1/inference/8b_rvllm_infer.yaml", "configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "docs/user_guides/infer/common_workflows.md", "docs/user_guides/infer/configuration.md", "docs/user_guides/infer/inference_cli.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/train/configuration.md", "docs/user_guides/train/train.md", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "scripts/polaris/jobs/llama_tune.sh", "scripts/polaris/notebooks/Oumi - Tuning Llama.ipynb", "tests/unit/utils/test_torch_naming_heuristics.py"], "area": "configs", "area_votes": {"configs": 35, "docs": 8, "infra": 1}, "body": "# Description\r\n\r\nLlama 3.1 8B Instruct used to be named `meta-llama/Meta-Llama-3.1-8B-Instruct`, but it's now been renamed to `meta-llama/Llama-3.1-8B-Instruct`, which is how the models have been named in Llama 3.2 and after. The page for the former now redirects to the latter: https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct.\r\n\r\nWe make the following renames:\r\n- `meta-llama/Meta-Llama-3.1-8B` -> `meta-llama/Llama-3.1-8B`\r\n- `meta-llama/Meta-Llama-3.1-8B-Instruct` -> `meta-llama/Llama-3.1-8B-Instruct`\r\n- `meta-llama/Meta-Llama-3.1-70B-Instruct` -> `meta-llama/Llama-3.1-70B-Instruct`\r\n- `meta-llama/Meta-Llama-3.1-405B-Instruct` -> `meta-llama/Llama-3.1-405B-Instruct`\r\n\r\nI also updated references to Llama 2 or 3 to instead use the latest versions.\r\nTested one of the configs to verify it still works\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-02-04T02:15:29Z"}
{"number": 1349, "title": "Update sft_datasets.md", "files": ["docs/resources/datasets/sft_datasets.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nUpdated the documentation for sft_datasets with more precise instructions for how to add a new dataset and how to override dataset_name to utilize existing datasets.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-04T17:26:42Z"}
{"number": 1345, "title": "Update \"GPU Tests\" status badge in README page", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Using the link created by \"Create Status Badge\" action  for GPU Tests workflow\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-02T18:07:16Z"}
{"number": 1344, "title": "Update index.md - Add DeepSeek to supported models", "files": ["docs/index.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-02T15:38:56Z"}
{"number": 1343, "title": "Update README.md - Add DeepSeek to supported models", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-02T15:38:20Z"}
{"number": 1342, "title": "Support HuggingFaceM4/Docmatix dataset", "files": ["configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml", "src/oumi/datasets/vision_language/docmatix.py"], "area": "configs", "area_votes": {"configs": 4, "data": 1}, "body": "Related to #915\r\n\r\nAdd support for the HuggingFaceM4/Docmatix dataset for Document Visual Question Answering.\r\n\r\n* **New Dataset Class**\r\n  - Add `DocmatixDataset` class in `src/oumi/datasets/vision_language/docmatix.py` to handle the HuggingFaceM4/Docmatix dataset.\r\n  - Implement `transform_conversation` method to convert raw data into a `Conversation` object.\r\n  - Register the dataset using the `register_dataset` decorator.\r\n\r\n* **Configuration Updates**\r\n  - Update `configs/recipes/vision/llava_7b/sft/train.yaml` to include the HuggingFaceM4/Docmatix dataset under `data.train.datasets`.\r\n  - Update `configs/recipes/vision/phi3/sft/train.yaml` to include the HuggingFaceM4/Docmatix dataset under `data.train.datasets`.\r\n  - Update `configs/recipes/vision/qwen2_vl_2b/sft/train.yaml` to include the HuggingFaceM4/Docmatix dataset under `data.train.datasets`.\r\n  - Update `configs/recipes/vision/smolvlm/sft/train.yaml` to include the HuggingFaceM4/Docmatix dataset under `data.train.datasets`.\r\n\r\n\r\nTowards OPE-747\r\n", "merged_at": "2025-02-04T16:07:09Z"}
{"number": 1340, "title": "Update our README and docs with the github trending badge.", "files": ["README.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nI've added the badge from https://trendshift.io/repositories/12865 to both our README and docs main page.\r\n\r\n\r\nPreview of our docs:\r\n\"image\"\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nN/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-02T06:59:47Z"}
{"number": 1336, "title": "Update VLM cluster names in sample commands", "files": ["configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "tests/scripts/launch_tests.sh"], "area": "configs", "area_votes": {"configs": 7}, "body": "# Description\r\n\r\n\r\n\r\n\r\n--Make them more unique to prevent conflicts when same name is used for clusters with different specs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-01T23:57:11Z"}
{"number": 1335, "title": "[Notebooks] Minor improvements  in VLM and CNN notebooks", "files": ["notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-911\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-01T20:18:22Z"}
{"number": 1333, "title": "chore: update launcher.sh", "files": ["scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "\r\n# Description\r\n\r\n\r\n\r\neror -> error\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-01T15:06:48Z"}
{"number": 1328, "title": "Tweak `--mem-fraction-static` param in sample SGLang configs", "files": ["configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- The previous value `--mem-fraction-static=0.99` was selected to run Llama 3.2 Vision on GPU with 24GB (barely fits), and generally such a high limit is unsafe/not-recommended to use.  0.9 is more reasonable.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-31T22:58:23Z"}
{"number": 1327, "title": "[Notebooks] Add a note to Tour notebook to restart kernel after the first `pip install`", "files": ["notebooks/Oumi - A Tour.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add a note to Tour notebook to restart kernel after the first `pip install` as in https://github.com/oumi-ai/oumi/pull/1318\r\n\r\nhttps://github.com/oumi-ai/oumi/issues/1325\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-981\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X]  This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-31T21:58:06Z"}
{"number": 1326, "title": "Update `llama3-instruct` chat template to align with the original models template", "files": ["src/oumi/datasets/chat_templates/llama3-instruct.jinja", "tests/unit/datasets/test_chat_templates.py", "tests/unit/inference/test_sglang_inference_engine.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update `llama3-instruct` chat template for full compatibility with official template for text-only (multimodal will need more work)\r\n-- Add unit tests to maintain this equivalence.\r\n-- Parameterize Phi3 test to also cover `microsoft/Phi-3-mini-4k-instruct` (text only)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-957\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-01T14:28:44Z"}
{"number": 1324, "title": "Disallow using `DatasetParams` field names as keys in `DatasetParams.dataset_kwargs`", "files": ["src/oumi/core/configs/params/data_params.py", "tests/unit/core/configs/params/test_data_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- This may lead to obscure Python kwarg resolution error (function call with duplicate keyed arguments). Better to block it from the start.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-916\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-31T23:22:57Z"}
{"number": 1321, "title": "Update Phi3 to support multiple images", "files": ["src/oumi/core/datasets/vision_language_dataset.py", "src/oumi/datasets/chat_templates/phi3-instruct.jinja", "tests/unit/builders/test_models.py", "tests/unit/datasets/test_chat_templates.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update `phi3-instruct` chat template to support multiple images\r\n-- Add test to verify that Oumi's `phi3-instruct` template output is the same as for original built-in model template for text-only (can't do it for images as the original chat template is text-only)\r\n-- Add test to verify the `phi3-instruct` output for multimodal conversation is as expected\r\n-- Rename some vars in `vision_language_dataset.py` for clarify\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-575\r\nTowards OPE-355\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-31T02:52:30Z"}
{"number": 1320, "title": "Update parasail_inference_engine.py", "files": ["src/oumi/inference/parasail_inference_engine.py", "tests/unit/inference/test_parasail_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nParasail API uses .io, not .com\r\n```\r\ncurl -X POST https://api.parasail.io/v1/chat/completions \\\r\n    -H 'Authorization: Bearer ' \\\r\n    -H 'content-type: application/json' \\\r\n    -d '{\"model\": \"parasail-deepseek-r1\", \"messages\": [{\"role\": \"user\", \"content\": \"What is the capital of New York?\"}]}'\r\n```\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-31T00:00:30Z"}
{"number": 1319, "title": "Fix typo and update warning message for OUMI trainer", "files": ["src/oumi/builders/training.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-31T00:25:08Z"}
{"number": 1318, "title": "[Notebooks] Add a note that a notebook kernel restart may be needed after `pip install oumi`", "files": ["notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb"], "area": "docs", "area_votes": {"docs": 7}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add a note that a notebook kernel restart may be needed after the first `pip install oumi`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-31T02:39:55Z"}
{"number": 1317, "title": "[Notebooks] Update VLM notebook", "files": ["notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add hardware requirements section\r\n-- Add a note to restart notebook after the first `pip install` for the changes to take  effect. NOTE: I may replicate this notice in more notebooks after the initial review.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-911\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-30T20:24:03Z"}
{"number": 1315, "title": "remove duplicate keys in config example", "files": ["docs/resources/datasets/pretraining_datasets.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nstream and packing keys are duplicated in the example\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n", "merged_at": "2025-01-30T02:35:36Z"}
{"number": 1314, "title": "Update configuration.md", "files": ["docs/user_guides/train/configuration.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nAdding a list of supported optimizers to the docs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-29T22:14:25Z"}
{"number": 1313, "title": "Add HF_TOKEN instructions to Oumi Multimodal notebook", "files": ["notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-911\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-29T20:28:04Z"}
{"number": 1312, "title": "fix(docs): \"interested by joining\" to \"interested in joining\"", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Incorrect preposition \u201cby\u201d instead of \u201cin.\u201d\r\nApologies for the tiny one-word fix, but this was the only typo spotted! Kudos to the team for keeping everything else pristine.\r\n\r\n---\r\n\r\n# Description\r\n\r\nThis PR fixes a small documentation typo where the phrase \u201cinterested by joining\u201d was used instead of \u201cinterested **in** joining.\u201d Everything else looks great! Thank you for maintaining such high-quality documentation.\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)", "merged_at": "2025-01-29T20:02:10Z"}
{"number": 1311, "title": "Update logging and unit tests related to chat templates", "files": ["src/oumi/builders/models.py", "tests/unit/builders/test_models.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update logging and unit tests related to chat templates \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-957\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-29T19:57:39Z"}
{"number": 1310, "title": "Add more detailed comment headers to YAML configs", "files": ["configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/examples/misc/hello_world_gcp_job.yaml", "configs/examples/misc/hello_world_polaris_job.yaml", "configs/examples/misc/vllm_polaris_job.yaml", "configs/projects/aya/evaluation/eval.yaml", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/chatqa/chatqa_stage1_train.yaml", "configs/projects/chatqa/chatqa_stage2_train.yaml", "configs/projects/chatqa/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/gcp_job.yaml", "configs/recipes/deepseek_r1/inference/671b_together_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_llama_70b_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_llama_8b_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_1_5b_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b_infer.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_train.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/gpt2/inference/infer.yaml", "configs/recipes/gpt2/pretraining/gcp_job.yaml", "configs/recipes/gpt2/pretraining/mac_train.yaml", "configs/recipes/gpt2/pretraining/train.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/inference/70b_infer.yaml", "configs/recipes/llama3_1/inference/8b_infer.yaml", "configs/recipes/llama3_1/inference/8b_rvllm_infer.yaml", "configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/polaris_job.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/llama3_2/inference/1b_infer.yaml", "configs/recipes/llama3_2/inference/1b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/1b_vllm_infer.yaml", "configs/recipes/llama3_2/inference/3b_infer.yaml", "configs/recipes/llama3_2/inference/3b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/3b_vllm_infer.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/llama3_3/evaluation/70b_eval.yaml", "configs/recipes/llama3_3/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_3/inference/70b_infer.yaml", "configs/recipes/llama3_3/inference/70b_vllm_infer.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml", "configs/recipes/phi3/dpo/gcp_job.yaml", "configs/recipes/phi3/dpo/mac_train.yaml", "configs/recipes/phi3/dpo/nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/nvidia_80g_train.yaml", "configs/recipes/phi3/dpo/train.yaml", "configs/recipes/phi3/evaluation/eval.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/phi3/sft/lora_train.yaml", "configs/recipes/phi3/sft/mac_lora_train.yaml", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/smollm/evaluation/135m/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_alpaca_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/inference/135m_infer.yaml", "configs/recipes/smollm/sft/135m/gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_train.yaml", "configs/recipes/smollm/sft/135m/train.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_rvllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/train.yaml", "configs/recipes/vision/llava_7b/inference/infer.yaml", "configs/recipes/vision/llava_7b/inference/vllm_infer.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/inference/vllm_infer.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/sglang_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml", "src/experimental/configs/projects/zephyr/evaluation/eval.yaml", "src/experimental/configs/projects/zephyr/sft/full_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/full_train.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_train.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_nvidia_24g_train.yaml"], "area": "configs", "area_votes": {"configs": 183, "other": 7}, "body": "# Description\r\n\r\nAll configs will now have detailed information including the class they're loaded in, a link to documentation, and an example command to run.\r\n\r\nThis PR only modifies the comment headers of YAML config files. I double checked this is true for all files. I wouldn't recommend reviewing each line as it's a large PR, but instead to review an example header for one of each type of config (train/eval/infer/job.yaml) for its contents.\r\n\r\n## Related issues\r\n\r\nTowards OPE-894\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2025-01-31T09:34:04Z"}
{"number": 1308, "title": "Update config/notebook pip installs to use PyPI", "files": ["configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/projects/chatqa/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_gcp_job.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/gpt2/pretraining/gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_3/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/phi3/dpo/gcp_job.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/sft/135m/gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "scripts/polaris/notebooks/Oumi - Multinode Inference on Polaris.ipynb", "scripts/pretokenize/sky.yaml", "src/experimental/configs/projects/zephyr/sft/full_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_gcp_job.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "src/oumi/evaluation/alpaca_eval.py", "src/oumi/evaluation/platform_prerequisites.py"], "area": "configs", "area_votes": {"configs": 63, "docs": 3, "infra": 1, "other": 3, "evaluation": 2}, "body": "# Description\r\n\r\nTested a config and notebook.\r\n\r\n## Related issues\r\n\r\nFixes OPE-854\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2025-01-29T04:29:37Z"}
{"number": 1307, "title": "Tune VLM configs for  SmolVLM and Qwen2-VL", "files": ["configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Remove unnecessary dataset limits for SmolVLM and Qwen2-VL\r\n-- SmolVLM: `--exclude \"onnx/*`\r\n-- Tested on GCP\r\n-- The changes are similar to https://github.com/oumi-ai/oumi/pull/1287 and https://github.com/oumi-ai/oumi/pull/1306\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-951\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n\r\n", "merged_at": "2025-01-29T03:06:43Z"}
{"number": 1306, "title": "Tune VLM SFT configs", "files": ["configs/recipes/vision/llama3_2_vision/sft/11b_full/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/train.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- LLAVA: Switch to `use_torchdata=True`\r\n-- LLAMA: Remove unnecessary dataset `limit`s\r\n-- Tested on GCP\r\n-- The changes are similar to https://github.com/oumi-ai/oumi/pull/1287\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-951\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-29T02:40:07Z"}
{"number": 1305, "title": "[docs] Use `pip install oumi` over `pip install .`", "files": ["docs/development/docs_guide.md", "docs/get_started/installation.md", "docs/user_guides/evaluate/leaderboards.md", "docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Description\r\n\r\nUpdate a couple pip install references to install from PyPI. I verified all other references from installing from source, which seem to be correct.\r\n\r\nA future PR will update jobs/notebooks to install from PyPI as well\r\n\r\n## Related issues\r\n\r\nTowards OPE-854\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-29T02:06:40Z"}
{"number": 1304, "title": "Update README/docs to add new DeepSeek models", "files": ["README.md", "docs/resources/recipes.md", "docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_cli.md", "docs/user_guides/infer/inference_engines.md", "src/oumi/inference/vllm_inference_engine.py"], "area": "docs", "area_votes": {"docs": 5, "inference": 1}, "body": "# Description\r\n\r\nAlso includes minor doc fixes\r\n\r\n## Related issues\r\n\r\nTowards OPE-936\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-29T01:20:13Z"}
{"number": 1302, "title": "[docs][tiny] update inference engines reference", "files": ["docs/user_guides/infer/common_workflows.md", "docs/user_guides/infer/configuration.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Update inference engines reference\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-29T01:18:58Z"}
{"number": 1301, "title": "[tiny] use GitHub link for header", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-29T01:07:40Z"}
{"number": 1300, "title": "[docs] Add latest notebooks and update references", "files": ["README.md", "docs/development/dev_setup.md", "docs/get_started/tutorials.md", "docs/index.md", "docs/user_guides/infer/infer.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\n[docs] Add latest notebooks and update references\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-29T00:54:10Z"}
{"number": 1299, "title": "Minor updates to VLM Multimodal notebook", "files": ["notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Refactor memory cleanup into a helper function and call it in more places.\r\n-- Double the number of training steps (with the previous count I observed when adapter model still produces verbose outputs)\r\n-- Retain some cells' outputs (they can serve as illustration)\r\n-- More-consistent handling of variables .\r\n-- Other minor changes \r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-911\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-29T00:49:07Z"}
{"number": 1298, "title": "[tiny] Add docs auto-generated `.rst` files to gitignore", "files": [".gitignore"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-29T00:55:17Z"}
{"number": 1297, "title": "[notebooks] Update installation instructions for colab", "files": ["notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 7}, "body": "# Description\r\n\r\n\r\n\r\n-  Update installation instructions for colab\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-882\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T23:45:11Z"}
{"number": 1296, "title": "Update notebook intros", "files": ["notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Distill a Large Model.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 14}, "body": "# Description\r\n\r\n\r\n\r\n- Update notebook intros\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-948\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T23:24:55Z"}
{"number": 1294, "title": "[notebooks] Update intro & installation instruction", "files": ["notebooks/Oumi - A Tour.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add standard intro template for review\r\n- Add instructions for collar\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-908, OPE-948\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T23:15:52Z"}
{"number": 1293, "title": "[docs] Add more references to VL-SFT and SFT notebooks", "files": ["docs/resources/datasets/vl_sft_datasets.md", "docs/user_guides/train/training_methods.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- For better discoverability\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-911\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T23:05:59Z"}
{"number": 1292, "title": "Eval config change for `deepseek-ai/DeepSeek-R1-Distill-Llama-70B`", "files": ["configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Revert to `oumi evaluate`.  The updated config leads to CUDA OOM for this model\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-945, OPE-931\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T23:08:51Z"}
{"number": 1291, "title": "Add missing `-m oumi evaluate` argument in eval config", "files": ["configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Missed in accidentally in a previous change\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-945\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T22:16:27Z"}
{"number": 1289, "title": "Add Together Deepseek R1 inference config", "files": ["configs/recipes/deepseek_r1/inference/671b_together_infer.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\nManually tested\r\n\r\n## Related issues\r\n\r\nFixes OPE-936\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-28T22:01:22Z"}
{"number": 1288, "title": "[minor] vlm notebook minor updates (doc referencing, freeze visual backbone)", "files": ["docs/get_started/tutorials.md", "docs/index.md", "notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 3}, "body": "- It links the notebook in our documentation\r\n- Updates the notebook to freeze the vision part of Llama during SFT, and removes the `include_performance_metrics` which is less informative for PEFT/VisionLang models currently.\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # OPE-911\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T22:06:48Z"}
{"number": 1287, "title": "Update `microsoft/Phi-3-vision-128k-instruct` training config", "files": ["configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Switch to `use_torchdata=True` for both `OUMI` and `TRL_SFT` examples (previousuly only `OUMI`)\r\n-- Remove `limit`  dataset params.  It's uncommon to train on partial datasets except for demo/debugging.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-951\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T22:00:29Z"}
{"number": 1286, "title": "Misc eval configs cleanup", "files": ["configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "tests/e2e/test_eval_e2e.py"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update more models to use `oumi distributed accelerate launch` for consistency\r\n-- Enable multiGPU eval for DeepSeek 8B\r\n-- Add more e2e eval tests for these cases e.g., DeepSeek\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards  OPE-945, OPE-909\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T20:06:17Z"}
{"number": 1285, "title": "Fix citation", "files": ["CITATION.cff", "README.md", "docs/about/citations.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T19:32:35Z"}
{"number": 1283, "title": "Documentation: Judge (minor edits)", "files": ["docs/user_guides/judge/judge.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Judge (minor edits)\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T18:59:02Z"}
{"number": 1282, "title": "Update Oumi - A Tour.ipynb", "files": ["notebooks/Oumi - A Tour.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "Emoji update\r\nPunctuation and mechanics fixes\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T18:39:09Z"}
{"number": 1281, "title": "Update all documentation links to the new doc URL", "files": ["CONTRIBUTING.md", "README.md", "configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_gcp_job.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/llama3_1/README.md", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/llama3_3/evaluation/70b_eval.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml", "docs/development/docs_guide.md", "docs/user_guides/launch/launch.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with Oumi.ipynb", "notebooks/Oumi - Training CNN on Custom Dataset.ipynb", "notebooks/Oumi - Vision Language Models.ipynb", "src/experimental/configs/projects/zephyr/sft/full_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_gcp_job.yaml"], "area": "configs", "area_votes": {"docs": 10, "configs": 31, "other": 2}, "body": "# Description\r\n\r\n\r\n\r\nOur docs moved from `oumi.ai/docs/latest` -> `oumi.ai/docs/en/latest`.\r\n\r\nThis PR is a FIND-REPLACE of:\r\nFIND: `oumi.ai/docs/latest`\r\nREPLACE: `oumi.ai/docs/en/latest`\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-896\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T05:34:17Z"}
{"number": 1280, "title": "[tiny] update website link", "files": ["README.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n- update website link\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T05:31:22Z"}
{"number": 1279, "title": "Documentation: Inference -> List supported models", "files": ["docs/user_guides/infer/inference_engines.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdding lists of supported `model_name`s for remote inference engines\r\n\r\nListed supported models for the following Cloud APIs:\r\n- Anthropic\r\n- Google Vertex AI\r\n- Google Gemini API\r\n- OpenAI\r\n- DeepSeek\r\n\r\nThese 2 have too many to mention (did NOT add any specific models)\r\n- Together\r\n- Parasail.io\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-942\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T04:24:44Z"}
{"number": 1276, "title": "Add Deepseek R1 1.5B/32B configs", "files": ["configs/recipes/deepseek_r1/README.md", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_1_5b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_qwen_32b/gcp_job.yaml", "configs/recipes/deepseek_r1/inference/distill_llama_70b_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_llama_8b_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_1_5b_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_qwen_32b_infer.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_1_5b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_qwen_32b/lora_train.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "src/oumi/core/configs/params/generation_params.py"], "area": "configs", "area_votes": {"docs": 1, "configs": 32}, "body": "# Description\r\n\r\nOther possible configs include Qwen 7B/14B. I'm skipping for now since they're close in size to existing configs.\r\n\r\n## Related issues\r\n\r\nTowards OPE-936\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-28T19:58:50Z"}
{"number": 1275, "title": "Set up versioning for our documentation", "files": ["docs/conf.py"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nSets up a top-bar versioning drop down.\r\n\r\nNote: this doesn't work in local builds due to CORS errors when serving `make docs-serve`\r\n\r\nDuring build time, we'll read an environment variable to determine the current version.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-773\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T03:23:21Z"}
{"number": 1274, "title": "Documentation: Judge | minor edit (bold)", "files": ["docs/user_guides/judge/judge.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nManos ask\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T00:39:43Z"}
{"number": 1273, "title": "Add pypi release workflow", "files": [".github/workflows/release_pypi.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add pypi release workflow\r\n- After each new release, we will automatically upload to test-pypi\r\n- After manual verification, the workflow can be used to upload to pypi\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T00:29:28Z"}
{"number": 1272, "title": "Update launch.md", "files": ["docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nUpdating from `Code` -> `Jobs`\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T00:18:10Z"}
{"number": 1271, "title": "Update common_workflows.md - Clarify OpenAI is just an example", "files": ["docs/user_guides/infer/common_workflows.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Nit\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T00:10:25Z"}
{"number": 1270, "title": "Disable pre-release packages", "files": [".github/workflows/doctests.yaml", ".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 3}, "body": "# Description\r\n\r\n\r\n\r\n- Disallow pre-release packages.\r\n- The CI should match how users will install the package to catch any breakages early\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T00:09:52Z"}
{"number": 1269, "title": "Documentation: Evaluation page (update to highlight multi-modal)", "files": ["docs/user_guides/evaluate/evaluate.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Evaluation page update to highlight multi-modal\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-947\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T00:13:45Z"}
{"number": 1268, "title": "Update index.md to highlight beta stage", "files": ["docs/index.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T23:44:03Z"}
{"number": 1267, "title": "Update README.md to highlight beta stage", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Nit: underline \"beta\" to highlight more\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T23:45:15Z"}
{"number": 1266, "title": "[tiny] less strict requirements", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T23:22:39Z"}
{"number": 1265, "title": "update pypi release workflow to use trusted env", "files": [".github/workflows/release_gcp.yaml", ".github/workflows/release_pypi.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n- update pypi release workflow to use trusted env\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T22:28:16Z"}
{"number": 1263, "title": "Add Deepseek R1 Distill Llama 8B/70B configs", "files": ["configs/recipes/deepseek_r1/README.md", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_70b/gcp_job.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/eval.yaml", "configs/recipes/deepseek_r1/evaluation/distill_llama_8b/gcp_job.yaml", "configs/recipes/deepseek_r1/inference/distill_llama_70b_infer.yaml", "configs/recipes/deepseek_r1/inference/distill_llama_8b_infer.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_70b/qlora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/full_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/lora_train.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_gcp_job.yaml", "configs/recipes/deepseek_r1/sft/distill_llama_8b/qlora_train.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml"], "area": "configs", "area_votes": {"docs": 1, "configs": 19}, "body": "# Description\r\n\r\nFor train/eval/infer. Qwen configs to follow. Ran all configs to verify they work.\r\n\r\n## Related issues\r\n\r\nTowards OPE-936\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-27T23:22:50Z"}
{"number": 1262, "title": "Update workflow names", "files": [".github/workflows/doctests.yaml", ".github/workflows/pretest.yaml", ".github/workflows/release_gcp.yaml", ".github/workflows/release_pypi.yaml"], "area": "infra", "area_votes": {"infra": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T21:56:19Z"}
{"number": 1261, "title": "Documentation: Updates on Evaluation/Judge (based on Manos' feedback)", "files": ["docs/_static/judge/judge_figure.svg", "docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/generative_benchmarks.md", "docs/user_guides/evaluate/standardized_benchmarks.md", "docs/user_guides/judge/judge.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T23:22:24Z"}
{"number": 1259, "title": "Add pypi release workflow using testpypi", "files": [".github/workflows/release_pypi.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add pypi release workflow\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T21:30:22Z"}
{"number": 1258, "title": "Add multi-modal (vlm) notebook with Llama 11B", "files": ["notebooks/Oumi - Vision Language Models.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n**Adds a working notebook to explore Oumi's VLM capabilities.**\r\nSpecifically,\r\n - We use LoRA and SFT to train a `Llama-3.2-11B-Vision-Instruct` to produce **succinct** responses on visually grounded questions. We achieve this using the in-house provided `Vqav2SmallDataset` dataset.\r\n - With a single A100 (40GB) GPU, it takes about _25 minutes_ to train a single epoch of 1,000 subsampled examples - which is enough for the model to pick up the nature of the training dataset.\r\n - The maximum GPU memory required during training is approx. 30GB.\r\n - Last, the notebook includes snippets for:\r\n    1. Training with multi-GPU, if available.\r\n    2. Use the CLI to launch the inference instead of the Python API.\r\n    3. Construct one's own question-image input pairs to make inferences on.\r\n\r\n__Example of the effect of training__:\r\nGiven [this](https://huggingface.co/datasets/merve/vqav2-small?image-viewer=9234F8B064BAAA9746575919E2C9D634EC8B58D8) image and the question: _\"Is this boy a good pitcher?\"_\r\nThe default Model responds:\r\n\"The boy in the image is wearing a baseball uniform and appears to be pitching, but it's difficult to determine if he's a good pitcher based on this image alone.\"\r\nThe ground-truth training data is:\r\n\"Yes\"\r\nThe post-tuning model responds:\r\n\"Yes\"\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # OPE-911\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-28T03:41:46Z"}
{"number": 1255, "title": "[docs] more docs feedback", "files": ["README.md", "docs/api/oumi.models.rst", "docs/api/oumi.models.utils.rst", "docs/api/oumi.utils.rst", "docs/development/dev_setup.md", "docs/get_started/quickstart.md", "docs/resources/models/supported_models.md", "docs/resources/recipes.md", "docs/user_guides/infer/configuration.md", "docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_cli.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/train/configuration.md"], "area": "docs", "area_votes": {"docs": 13}, "body": "# Description\r\n\r\n\r\n\r\n- Misc doc feedback fixes\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T20:00:49Z"}
{"number": 1254, "title": "Update Oumi - A Tour.ipynb", "files": ["notebooks/Oumi - A Tour.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "Very small verbiage change to center text around what the user can do with our platform, rather than what we can do for them.\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T18:10:19Z"}
{"number": 1253, "title": "Update quickstart.md", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Emoji Update\r\nUpdated Oumi CLI\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T16:58:02Z"}
{"number": 1252, "title": "Update quickstart.md", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Emoji update\r\nUpdated Prerequisites and removed hyphen\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T16:58:56Z"}
{"number": 1251, "title": "Update quickstart.md", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Emoji updates:\r\n\r\nUpdated cloud emoji\r\nRemoved second alarm from 2 warning headers\r\nAdded compass to What's next\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T16:23:21Z"}
{"number": 1250, "title": "Update quickstart.md", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Updated emoji for Eval  to \ud83d\udcca\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T16:59:41Z"}
{"number": 1249, "title": "Add more details to troubleshooting FAQ", "files": ["docs/development/dev_setup.md", "docs/faq/oom.md", "docs/faq/troubleshooting.md", "docs/user_guides/train/environments/environments.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n## Related issues\r\n\r\nFixes OPE-280\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2025-01-27T17:01:53Z"}
{"number": 1248, "title": "[docs] misc docs feedback", "files": ["docs/api/oumi.models.rst", "docs/api/oumi.models.utils.rst", "docs/get_started/quickstart.md", "docs/resources/models/models.md", "docs/resources/recipes.md", "docs/user_guides/infer/cli_reference.md", "docs/user_guides/infer/configuration.md", "docs/user_guides/infer/infer.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 9}, "body": "# Description\r\n\r\n\r\n\r\n-  misc docs feedback\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T17:16:31Z"}
{"number": 1247, "title": "[tiny] Qwen2-VL activate experimental datapipes", "files": ["configs/recipes/vision/qwen2_vl_2b/sft/train.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\nActivates the `experimental_use_torch_datapipes` in our default training configuration. It turns out that using this, the training becomes significantly faster. E.g., as tested with 4xA100:40GB GPUs it about triple the `train_tokens_per_second`. \r\n\r\n\"Screenshot\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-811\r\nFixes OPE-673\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T17:32:14Z"}
{"number": 1245, "title": "Update training_methods.md - Change compute requirement suggestions", "files": ["docs/user_guides/train/training_methods.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Change compute requirement suggestions to make it more accurate (bigger gap of SFT with pretraining) and less daunting\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T17:02:32Z"}
{"number": 1244, "title": "Update train.md - nit description change", "files": ["docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "nit\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T17:02:44Z"}
{"number": 1243, "title": "Update README.md title to \"Join the Community!\"", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Nit title change\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T00:34:56Z"}
{"number": 1242, "title": "Update index.md title for \"Join the Community!\"", "files": ["docs/index.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Nit\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T00:19:23Z"}
{"number": 1241, "title": "Update quickstart.md - nit for Oumi support request", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-27T00:19:49Z"}
{"number": 1239, "title": "Update our oumi launch documentation.", "files": ["docs/user_guides/launch/custom_cluster.md", "docs/user_guides/launch/deploy.md", "docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n- Added a quickstart section to the main launch.md\r\n- Removed TOCs from both subdocs\r\n- Removed install instructions from both subdocs\r\n- Moved the section for launching jobs higher\r\n- Removed the \"advanced\" header from one section\r\n\r\n![Screenshot 2025-01-26 at 9 29 48\u202fAM](https://github.com/user-attachments/assets/7633d486-acfd-435f-9c53-db4b3e5a22dd)\r\n\r\n![Screenshot 2025-01-26 at 9 29 58\u202fAM](https://github.com/user-attachments/assets/403a9904-cfd4-4f9d-9e87-80de3a292c8c)\r\n\r\n![Screenshot 2025-01-26 at 9 30 04\u202fAM](https://github.com/user-attachments/assets/005478d5-fb7d-4afd-927b-a5c6a93f008c)\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-910\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-26T17:42:04Z"}
{"number": 1237, "title": "Add more e2e evaluation tests", "files": ["configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "tests/e2e/test_eval_e2e.py", "tests/e2e/test_train_e2e.py", "tests/scripts/gcp_e2e_tests_job.yaml", "tests/scripts/launch_tests.sh"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add more e2e evaluation test cases\r\n-- Minor updates to launcher script and test names\r\n-- Update checkpoints validations for LoRA adapters\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-926, OPE-909\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-26T17:20:57Z"}
{"number": 1235, "title": "Re-enable parallel evaluation for VLM-s", "files": ["configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "src/oumi/core/configs/params/model_params.py"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Using LM-harness's `parallelize=True` ( `shard_for_eval=True`) works OK now on GCP  (couldn't get the previous data parallel recipe work)\r\n-- Removing the outdated `ModelParams` check and comment that `shard_for_eval` is not compatible with `accelerate`. The current documentation suggests using `accelerate` for this: https://github.com/EleutherAI/lm-evaluation-harness\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-926\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-25T02:22:06Z"}
{"number": 1233, "title": "[tiny][docs] Update PEFT/LoRA content", "files": ["docs/user_guides/train/configuration.md", "docs/user_guides/train/train.md", "docs/user_guides/train/training_methods.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n- Adds a configuration example for LoRA.\r\n- Lists the `init_lora_weights ` in the PEFT configuration parameters. \r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes #OPE-878\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-24T23:50:41Z"}
{"number": 1232, "title": "[BugFix] GGUF does not work with VLLM", "files": ["src/oumi/builders/models.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\nProblem: GGFU does not work with VLLM\r\nCause: The tokenizer's code tries to lookup the GGFU model name in HF and throws an OS Error since it does NOT exist. It should look up the tokenizer's name instead.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-25T00:12:11Z"}
{"number": 1231, "title": "Add a script to pre-download models for `gpu_tests`", "files": [".github/workflows/gpu_tests.yaml", "tests/scripts/predownload_for_github_gpu_tests.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add a script to download few models used in GPU tests\r\n-- Update Github Workflow to invoke the script\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-935\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-24T23:31:53Z"}
{"number": 1228, "title": "Add example distillation notebook", "files": ["notebooks/Oumi - Distill a Large Model.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nCreate example notebook to show how to collect inferences on a larger model, finetune a smaller model, and show measurable improvements on a particular eval.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-912\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-24T19:19:36Z"}
{"number": 1227, "title": "[docs] Update inference engines doc page", "files": ["docs/index.md", "docs/user_guides/infer/inference_engines.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Update inference engines doc page\r\n- Minor bug fixes on index.md\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-24T17:17:13Z"}
{"number": 1226, "title": "Clean-up inference engine builder", "files": ["src/oumi/builders/inference_engines.py", "tests/unit/inference/test_inference_engine_init.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Cleanup inference engines builder class\r\n- Add test to make sure all engines are covered \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-24T18:13:45Z"}
{"number": 1225, "title": "Scaffolding and the first testcase for e2e evaluation tests", "files": ["src/oumi/core/configs/base_config.py", "tests/e2e/__init__.py", "tests/e2e/test_eval_e2e.py", "tests/e2e/test_train_e2e.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Scaffolding and the first testcase for e2e evaluation tests\r\n-- Refactor a couple of common helper utils used in e2e tests\r\n-- Improve logging for OmegaConf merge errors\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-926, OPE-934\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-24T03:27:47Z"}
{"number": 1223, "title": "[VLLM Engine] Enabling BitsAndBytes quantization", "files": ["src/oumi/inference/vllm_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nVLLM Engine: Enabling BitsAndBytes quantization\r\n\r\n2 different scenarios tested:\r\n(original model weights for `meta-llama/Llama-3.2-1B-Instruct`: **_2.3185 GB_**; expectation was 1B --> 1 GB* 2 bytes = 2GB)\r\n\r\n1) Loading a **_pre-quantized checkpoint_** (`unsloth/Llama-3.2-1B-Instruct-bnb-4bit`)\r\n- [requires](https://docs.vllm.ai/en/latest/features/quantization/bnb.html):  quantization = \"bitsandbytes\", load_format\" = \"bitsandbytes\"\r\n- quantization (4bit or 8bit) depends on the model itself; it is 4bit by default in this case. \r\n- most bitsandbytes models [discussed in HF](https://huggingface.co/models?other=bitsandbytes) do NOT work as expected, but I validated that this one does.\r\n- model weights reduction 2.37x (**_2.3185 GB --> 0.9786 GB_**), expectation was ~4x (going from 16 bits to 4 bits)\r\n```\r\nWARNING 01-24 00:39:48 config.py:321] bitsandbytes quantization is not fully optimized yet. The speed can be slower than non-quantized models.\r\nINFO 01-24 00:39:50 loader.py:1051] Loading weights with BitsAndBytes quantization.  May take a while ...\r\nINFO 01-24 00:39:51 model_runner.py:1067] Loading model weights took 0.9786 GB\r\n```\r\n\r\n2) Leveraging **_Inflight quantization_** (`meta-llama/Llama-3.2-1B-Instruct`)\r\n- [requires](https://docs.vllm.ai/en/latest/features/quantization/bnb.html):  quantization = \"bitsandbytes\", load_format\" = \"bitsandbytes\"\r\n- quantization can ONLY be 4bit (no option for 8bit, so Oumi's flag `load_in_8bit` will still load in 4 bit)\r\n- model weights reduction 2.37 (**_2.3185 GB --> 0.9785 GB_**), expectation was ~4x (going from 16 bits to 4 bits)\r\n```\r\nWARNING 01-24 00:46:16 config.py:321] bitsandbytes quantization is not fully optimized yet. The speed can be slower than non-quantized models.\r\nINFO 01-24 00:46:18 loader.py:1051] Loading weights with BitsAndBytes quantization.  May take a while ...\r\nINFO 01-24 00:46:19 model_runner.py:1067] Loading model weights took 0.9785 GB\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-24T18:16:48Z"}
{"number": 1222, "title": "Set `use_spot` to False in our JobConfigs", "files": ["configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/projects/chatqa/gcp_job.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/gpt2/pretraining/gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_3/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/phi3/dpo/gcp_job.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/sft/135m/gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "docs/user_guides/launch/launch.md", "src/experimental/configs/projects/zephyr/sft/full_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_gcp_job.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "tests/scripts/gcp_e2e_tests_job.yaml"], "area": "configs", "area_votes": {"configs": 49, "docs": 1, "other": 3}, "body": "# Description\r\n\r\nThis is because the VM disappears without clear messaging in cases of preemption, which will be a source of friction. Non-spot instances work as expected from a user's perspective.\r\n\r\nThe downside is there's less on-demand quota, and the user's job may not schedule, but the error message in this case will be clear. In addition, I add comments for jobs likely to run into quota issues (requesting 8 A100s or 4/8 A100-80GBs) to mention enabling spot VMs.\r\n\r\n## Related issues\r\n\r\nFixes OPE-932\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-24T02:03:13Z"}
{"number": 1221, "title": "Update ConsoleLogger to write to STDOUT", "files": ["src/oumi/utils/logging.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n--  Since there are concerns about splitting STDOUT/STDERR logs (https://github.com/oumi-ai/oumi/pull/1219 ), let's write to STDOUT for now \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-859\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-24T00:01:00Z"}
{"number": 1220, "title": "[docs] Update index page", "files": ["docs/index.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-23T21:38:46Z"}
{"number": 1218, "title": "[Notebook] Evaluation with Oumi", "files": ["notebooks/Oumi - Evaluation with Oumi.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nNotebook: Evaluate with Oumi (LM Harness & AlpacaEval 2.0) for 3 models (LLaMA 1B, 3B, 8B)\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # OPE-913\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-23T20:54:44Z"}
{"number": 1217, "title": "gpt2: move `include_performance_metrics` param from script to yaml", "files": ["configs/recipes/gpt2/pretraining/gcp_job.yaml", "configs/recipes/gpt2/pretraining/mac_train.yaml", "configs/recipes/gpt2/pretraining/train.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-23T21:27:25Z"}
{"number": 1215, "title": "Move configs to experimental", "files": ["docs/resources/models/models.md", "docs/resources/recipes.md", "src/experimental/configs/projects/zephyr/README.md", "src/experimental/configs/projects/zephyr/evaluation/eval.yaml", "src/experimental/configs/projects/zephyr/sft/full_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/full_train.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_gcp_job.yaml", "src/experimental/configs/projects/zephyr/sft/qlora_train.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "src/experimental/configs/recipes/phi3/dpo/fsdp_nvidia_24g_train.yaml"], "area": "other", "area_votes": {"docs": 3, "other": 7}, "body": "# Description\r\n\r\nAfter we fix them, we can move them back to their original locations.\r\n\r\n## Related issues\r\n\r\nTowards OPE-929\r\nTowards OPE-818\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2025-01-23T21:34:46Z"}
{"number": 1214, "title": "Simplify inference engine API", "files": ["src/oumi/builders/inference_engines.py", "src/oumi/cli/judge.py", "src/oumi/core/configs/params/remote_params.py", "src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/deepseek_inference_engine.py", "src/oumi/inference/gemini_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/openai_inference_engine.py", "src/oumi/inference/parasail_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/remote_vllm_inference_engine.py", "src/oumi/inference/sglang_inference_engine.py", "src/oumi/inference/together_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/unit/inference/test_deepseek_inference_engine.py", "tests/unit/inference/test_generation_params.py", "tests/unit/inference/test_inference_engine_init.py", "tests/unit/inference/test_openai_inference_engine.py", "tests/unit/inference/test_parasail_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py", "tests/unit/inference/test_together_inference_engine.py"], "area": "inference", "area_votes": {"other": 1, "cli": 1, "configs": 1, "inference": 13}, "body": "# Description\r\n\r\n\r\n\r\n- The goal of this PR is to simplify the inference engine API. \r\n  - Users can now set their generation, remote and model params in the engine constructor\r\n  - Users can optionally override `engine.infer(inputs, config=config)`, but by default sending a config is now optional\r\n- Fix a regression in the unit tests that was making unnecessary network calls to check if the model is a vlm. This reduced the runtime by 50%\r\n- Simplify remote engines, which now only have to set the `base_url` instead of adding a constructor. \r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-23T21:32:40Z"}
{"number": 1213, "title": "Fix most job configs", "files": ["configs/examples/bulk_inference/README.md", "configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/bulk_inference/mistral_small_infer.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/examples/misc/hello_world_gcp_job.yaml", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/chatqa/chatqa_stage2_train.yaml", "configs/projects/zephyr/sft/full_train.yaml", "configs/projects/zephyr/sft/qlora_train.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/gpt2/pretraining/train.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "configs/recipes/phi3/dpo/fsdp_nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/gcp_job.yaml", "configs/recipes/phi3/dpo/mac_train.yaml", "configs/recipes/phi3/dpo/nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/nvidia_80g_train.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "docs/resources/recipes.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "src/oumi/datasets/sft/chatqa.py"], "area": "configs", "area_votes": {"docs": 4, "configs": 28, "data": 1}, "body": "# Description\r\n\r\nThis PR fixes almost all Oumi job configs to be able to run. Those that require non-trivial fixes will be moved to experimental in a follow-up PR, and will be resolved at a later time.\r\n\r\nFixes made to config files (not comprehensive):\r\n- Delete Mistral config. It was OOMing, and there was no specific reason at the time of creation to use Mistral over another model we support like Llama\r\n- Add tokenizer pad tokens\r\n- Comment out GCS bucket mounting. This way the config runs out of the box, and users can opt into mounting a bucket if desired.\r\n- Switch some eval jobs from 4 GPUs to 1 GPU, given the bug with data-parallel LM Harness mentioned in OPE-931\r\n\r\nFixes made to platform (requires a version update to fix):\r\n- Delete unneded parameter in `src/oumi/datasets/sft/chatqa.py`\r\n\r\nI've manually tested all job configs to ensure they work (with the shortcut of re-using a VM with the same accelerator setup to test multiple configs). The only broken configs are Zephyr and FSDP DPO Phi3.\r\n\r\n## Related issues\r\n\r\nTowards OPE-818\r\nTowards OPE-931\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2025-01-23T07:13:37Z"}
{"number": 1212, "title": "[BugFix] Throw a runtime error for quantized models & inference=VLLM", "files": ["src/oumi/inference/vllm_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\nThrow a runtime error when instantiating a {judge, evaluation, inference} config that includes a \"BitsAndBytes\" quantized model and a VLLM inference engine. I am aware this does NOT cover all possible use cases (users may use an inference engine directly without a config) but my hope is that it covers most cases. \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-23T04:45:59Z"}
{"number": 1210, "title": "Exclude `multi_gpu` tests from GitHub GPU tests", "files": [".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-698, OPE-909\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-23T01:00:06Z"}
{"number": 1209, "title": "Add wrappers for remote inference engines", "files": ["src/oumi/builders/inference_engines.py", "src/oumi/core/configs/inference_config.py", "src/oumi/inference/__init__.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/deepseek_inference_engine.py", "src/oumi/inference/gemini_inference_engine.py", "src/oumi/inference/openai_inference_engine.py", "src/oumi/inference/parasail_inference_engine.py", "src/oumi/inference/together_inference_engine.py", "tests/unit/inference/test_deepseek_inference_engine.py", "tests/unit/inference/test_openai_inference_engine.py", "tests/unit/inference/test_parasail_inference_engine.py", "tests/unit/inference/test_together_inference_engine.py"], "area": "inference", "area_votes": {"other": 1, "configs": 1, "inference": 6}, "body": "# Description\r\n\r\n\r\n\r\n\r\n- Add default remote params for engines that have one (e.g. openai, anthropic, parasail, etc)\r\n- Add new inference engine classes for OpenAI, Together, DeepSeek. \r\n- Update InferenceEngineType and builder function to include all supported engines\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-486\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-23T02:25:14Z"}
{"number": 1208, "title": "[docs] Add inference snippet for together.ai and DeepSeek APIs", "files": ["docs/user_guides/infer/inference_engines.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add inference snippet for together.ai and DeepSeek APIs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-774, OPE-458\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-22T23:43:20Z"}
{"number": 1207, "title": "[Evaluation] Bug: serialization", "files": ["src/oumi/utils/serialization_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\nDetails at https://linear.app/oumi/issue/OPE-924\r\n\r\n**_Current experience_**\r\n\r\nDisplay message:\r\n```\r\n[2025-01-22 14:15:40,787][oumi][rank0][pid:10457][MainThread][WARNING]][serialization_utils.py:33] Non-serializable value `functools.partial(, including_answer=True)` of type ``.\r\n```\r\n\r\nSaved file:\r\n```\r\n      \"fewshot_config\": {\r\n        \"sampler\": \"first_n\",\r\n        \"doc_to_text\": \"functools.partial(, including_answer=True)\",\r\n        \"doc_to_target\": \"\"\r\n      },\r\n```\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-22T23:15:06Z"}
{"number": 1205, "title": "GCP tests launcher script changes", "files": ["pyproject.toml", "tests/e2e/test_train_e2e.py", "tests/scripts/gcp_e2e_tests_job.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Define `multi_gpu` test marker, and update Launcher script to  use it\r\n-- Switch to editable install\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-698, OPE-909\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-22T19:24:44Z"}
{"number": 1204, "title": "Add newline before `pformat(train_config)`", "files": ["src/oumi/train.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- This results in slightly-better formatting in logs, and often done for other `pformat()`-s in oumi codabse\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-22T18:21:59Z"}
{"number": 1203, "title": "Update tests to make them runnable on GCP", "files": ["src/oumi/builders/models.py", "tests/__init__.py", "tests/conftest.py", "tests/e2e/test_notebooks.py", "tests/e2e/test_train_e2e.py", "tests/integration/infer/test_infer.py", "tests/integration/infer/test_native_text_inference_engine.py", "tests/integration/models/test_integration_cnn_classifier.py", "tests/scripts/gcp_e2e_tests_job.yaml", "tests/unit/inference/test_remote_inference_engine.py", "tests/unit/utils/test_conversation_utils.py", "tests/unit/utils/test_device_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add GCP Launcher script for running tests.\r\n-- Minimal changes to making tests pass on GCP multi-GPU machine.\r\n-- New utils to get locations of `testdata`, `notebooks`  (`get_oumi_root_directory().parent...`  leads to `miniconda3` location for `oumi` package `~/miniconda3/lib/python3.10/site-packages/oumi` vs `/home/gcpuser/sky_workdir/`)\r\n-- Tested on GCP `A100:4`, `A100:1`, locally, and on Github\r\n-- Minor improvements.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-698, OPE-909\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-22T04:45:48Z"}
{"number": 1202, "title": "Native inference: Don't set `min_p`, `temperature` in `GenerationConfig` if sampling is disabled", "files": ["src/oumi/inference/native_text_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- This leads to verbose warnings \r\n-- There is a chance it may contribute to observed non-determinism in native inference integration tests\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-921\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-22T04:38:40Z"}
{"number": 1201, "title": "Define `single_gpu` test marker", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Define `single_gpu` test marker  to allow skipping such tests on multi-GPU machines if needed.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-698, OPE-909\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-22T04:38:18Z"}
{"number": 1200, "title": "Configs Train/Infer/Eval and Llama 3.3v (70b)", "files": ["README.md", "configs/recipes/llama3_3/README.md", "configs/recipes/llama3_3/evaluation/70b_eval.yaml", "configs/recipes/llama3_3/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_3/inference/70b_infer.yaml", "configs/recipes/llama3_3/inference/70b_vllm_infer.yaml", "configs/recipes/llama3_3/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_full/train.yaml", "configs/recipes/llama3_3/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_lora/train.yaml", "configs/recipes/llama3_3/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_3/sft/70b_qlora/train.yaml"], "area": "configs", "area_votes": {"docs": 2, "configs": 10}, "body": "# Description\r\nAdds configurations to train/infer/eval 3.3. Llama 70b (text only) models.\r\n\r\n- We mirror the hyper-parameters we used in the 3.1 version of Llama 70b model.\r\n\r\n - Noticeable exception which should not affect the underlying compute demands: `max_grad_norm` was set to null (instead of defaulting to \"1\") to better much, torchtune recipes. Also, the default `sharding_strategy` was explicitly set to \"FULL_SHARD\" to increase readability/transparency.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-851\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-21T20:14:36Z"}
{"number": 1198, "title": "[docs] Add more details and cross-references related to customization", "files": ["docs/resources/datasets/other_datasets.md", "docs/resources/models/custom_models.md", "docs/user_guides/customization.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add cross-references for \"CNN Classifier\" notebook, Numpy dataset, and misc customization documentation\r\n \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-639\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-21T21:20:04Z"}
{"number": 1197, "title": "[Cherrypick for launch] Evaluate: return dict of results", "files": ["src/oumi/__init__.py", "src/oumi/evaluate.py", "src/oumi/evaluation/alpaca_eval.py", "src/oumi/evaluation/lm_harness.py"], "area": "evaluation", "area_votes": {"evaluation": 3}, "body": "# Description\r\n\r\n\r\n\r\nEvaluate: return dict of results\r\n(!) Not supported for evaluate async @taenin \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-21T05:12:21Z"}
{"number": 1195, "title": "Documentation: Judge | Custom Model page", "files": ["docs/user_guides/judge/custom_infer.md", "docs/user_guides/judge/custom_prompt.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Judge | Custom Model page\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-21T00:24:59Z"}
{"number": 1194, "title": "[docs] Resolve PyRight issues in NpzDataset in custom dataset example", "files": ["docs/resources/datasets/other_datasets.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-639\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T21:56:51Z"}
{"number": 1191, "title": "Bugfix in CnnClassifier example", "files": ["src/oumi/models/cnn_classifier.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Fixed few cases when `kernel_size` param is ignored \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards \r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T19:56:57Z"}
{"number": 1190, "title": "[tiny] Add error message to bare exceptions", "files": ["src/oumi/builders/models.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n-  Add error message to bare exceptions\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T19:36:41Z"}
{"number": 1189, "title": "[WIP] Define `default_dataset` property for all pretraining datasets", "files": ["src/oumi/datasets/pretraining/c4.py", "src/oumi/datasets/pretraining/dolma.py", "src/oumi/datasets/pretraining/falcon_refinedweb.py", "src/oumi/datasets/pretraining/fineweb_edu.py", "src/oumi/datasets/pretraining/pile.py", "src/oumi/datasets/pretraining/red_pajama_v1.py", "src/oumi/datasets/pretraining/red_pajama_v2.py", "src/oumi/datasets/pretraining/slim_pajama.py", "src/oumi/datasets/pretraining/starcoder.py", "src/oumi/datasets/pretraining/the_stack.py", "src/oumi/datasets/pretraining/tiny_stories.py", "src/oumi/datasets/pretraining/tiny_textbooks.py", "src/oumi/datasets/pretraining/wikipedia.py", "src/oumi/datasets/pretraining/wikitext.py", "src/oumi/datasets/pretraining/youtube_commons.py"], "area": "data", "area_votes": {"data": 15}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-897\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T19:21:00Z"}
{"number": 1188, "title": "Support `dataset_name_override`  dataset_kwarg", "files": ["src/oumi/builders/data.py", "src/oumi/builders/oumi_data.py", "tests/unit/builders/test_build_data.py", "tests/unit/builders/test_oumi_data.py"], "area": "other", "area_votes": {"other": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update `oumi_data.py` to be consistent with `data.py` for `dataset_name`\r\n-- If `dataset_name_override` is defined in `dataset_kwargs` then use that value over `dataset_params.dataset_name`. This also removes the potential `duplicate kwarg` error for `dataset_name` key\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-897, OPE-916\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-02-01T01:16:17Z"}
{"number": 1187, "title": "[tiny] Raise explicit error for models not supporting infer", "files": ["src/oumi/inference/native_text_inference_engine.py", "tests/integration/infer/test_native_text_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Currently we get an obscure error, this PR raises a more descriptive exception for models not supporting infer\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T19:34:56Z"}
{"number": 1186, "title": "[docs] fix misc broken links", "files": ["docs/_docsummaries.sh", "docs/development/contributing.md", "docs/development/dev_setup.md", "docs/get_started/quickstart.md", "docs/user_guides/train/environments/local.md", "docs/user_guides/train/environments/vscode.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 7}, "body": "# Description\r\n\r\n\r\n\r\n-  Fix misc broken links\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T04:00:29Z"}
{"number": 1183, "title": "Documentation: Judge | Custom Prompts page", "files": ["docs/user_guides/judge/custom_prompt.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Judge | Custom Prompts page\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T03:18:54Z"}
{"number": 1182, "title": "[docs] Add initial page how to define datasets for custom data types", "files": ["docs/resources/datasets/datasets.md", "docs/resources/datasets/other_datasets.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add a stub page, and an example how to define custom dataset to load from `.npz` files\r\n-- More changes will follow\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-639\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T19:02:03Z"}
{"number": 1180, "title": "Add `CnnClassifier` custom model", "files": ["src/oumi/models/__init__.py", "src/oumi/models/cnn_classifier.py", "tests/unit/builders/test_models.py", "tests/unit/models/test_cnn_classifier.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Adding CNN example (in addition to `MlpEncoder`) to show Oumi can be used for non-text models\r\n-- The example may be referenced from docs and/or notebooks later.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-639\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T19:20:34Z"}
{"number": 1179, "title": "Training vlm llama 90b", "files": ["configs/recipes/vision/llama3_2_vision/sft/90b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/90b_full/train.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\nAdds a basic configuration tested to be trainable with Oumi and Llama-3.2-90B-Vision-Instruct. \r\n\r\n- Many parameters were chosen to reduce the memory footprint almost maximally. (E.g., using an 8-bit adam, batch_size=1, etc.).\r\n\r\n**Note:**\r\n1. Even setting `gradient_accumulation_steps` to `2` will result in OOM. Activating `cpu_offload` or deactivating prefetches (`forward_prefetch`, `backward_prefetch`) will not enable to increase the _effective batch size_.\r\n2. Surprisingly,  activating `enable_gradient_checkpointing` results in faster training (logged in OPE-906).\r\n\r\nTowards: OPE-688\r\n\r\nRelates and can also be the basis for investigations per OPE-906 and OPE-905.\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T03:01:47Z"}
{"number": 1178, "title": "OUMI Trainer: Improve handling of `num_training_epochs` and update `_get_total_training_steps()`", "files": ["src/oumi/core/trainers/oumi_trainer.py", "tests/unit/core/trainers/test_oumi_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update stopping condition to use `params.max_steps` exclusively  if specified \r\n-- Update `_get_total_training_steps()` to cover more cases. Previously it could return a negative number if `max_steps` isn't set, leading to completely wrong LR schedule which doesn't converge.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-904, OPE-811\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-20T04:46:59Z"}
{"number": 1177, "title": "Update readme", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Update readme\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-19T21:32:26Z"}
{"number": 1175, "title": "OUMI Trainer: Support non-text models", "files": ["src/oumi/core/trainers/oumi_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Mark `tokenizer` as `Optional`\r\n-- Only count tokens if tokenizer is not None, and `input_ids` are present\r\n\r\nThe changes are needed to support non-textual models\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-639\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-19T18:07:06Z"}
{"number": 1173, "title": "Documentation: Judge |  Built-In page", "files": ["docs/user_guides/judge/built_in_judge.md", "docs/user_guides/judge/judge.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Judge |  Built-In page\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-19T00:12:32Z"}
{"number": 1172, "title": "[Documentation] Evaluation pages: fixing broken links", "files": ["docs/user_guides/evaluate/leaderboards.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n[Documentation] Evaluation pages: fixing broken links\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-18T04:34:11Z"}
{"number": 1171, "title": "Update GitHub actions markers expression", "files": [".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Use `-m`  instead of `-k`, to only match on markers (vs test names, etc)\r\n-- Exclude `e2e_eternal`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-18T01:49:40Z"}
{"number": 1169, "title": "Define `e2e_eternal` test marker", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- One level above `e2e` in size and slowness\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T23:20:13Z"}
{"number": 1168, "title": "Removed unnecessary cloud.md which is already covered in depth in another section.", "files": ["docs/user_guides/train/environments/cloud.md", "docs/user_guides/train/environments/environments.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\n\n\n\n\n\n\n## Related issues\n\n\n\n\nFixes # (issue)\n\n\n\n## Before submitting\n\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [ ] Did you link the issue(s) related to this PR in the section above?\n- [ ] Did you add / update tests where needed?\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2025-01-18T01:36:57Z"}
{"number": 1167, "title": "Update train.yaml", "files": ["configs/recipes/vision/llava_7b/sft/train.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "Add comment and spacing clarifying comments\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T22:59:47Z"}
{"number": 1165, "title": "Documentation: Judge | Judge (main) page", "files": ["docs/_static/judge/judge_figure.svg", "docs/user_guides/judge/custom_infer.md", "docs/user_guides/judge/custom_prompt.md", "docs/user_guides/judge/judge.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Description\r\n\r\n\r\n\r\nFinal (i.e., 95% ready) version for Judge's main documentation page\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-18T01:55:24Z"}
{"number": 1164, "title": "Exclude unit/integration tests explicitly marked as `e2e` from GitHub runs", "files": [".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Gives us a quick way to exclude tests (vs `skip()`)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T22:07:08Z"}
{"number": 1162, "title": "Update style_guide.md", "files": ["docs/development/style_guide.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Removed numbering, as the heading sizes make this clear for such a short document. Added link.\r\nMinor rewrites\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T21:19:14Z"}
{"number": 1161, "title": "Update contributing.md", "files": ["docs/development/contributing.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Added emoji\r\nAdded links\r\nMinor verbiage edits\r\nUpdated copyright date to 2025\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T23:21:34Z"}
{"number": 1160, "title": "Update quickstart.md Removed H1 emoji", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Until we determine / add emoji to all H1 (single \"#\") at the top of each doc, better to leave this one out. It sticks out in the left nav as being the only one with a lead emoji for the moment.\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T20:56:58Z"}
{"number": 1159, "title": "Update training_methods.md", "files": ["docs/user_guides/train/training_methods.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Re-order to have JSON before Python for consistency\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T19:56:25Z"}
{"number": 1158, "title": "Update train.md", "files": ["docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Made some minor corrections\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T19:56:56Z"}
{"number": 1157, "title": "[docs] Improve docs for custom multimodal SFT datasets", "files": ["docs/resources/datasets/vl_sft_datasets.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Make the example more realistic by providing an example how to specify raw HF dataset, and use image bytes instead of image path (more common)\r\n-- Change custom dataset name for consistency with other examples on the page\r\n-- Minor descriptions update. \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-639\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T19:59:57Z"}
{"number": 1156, "title": "[docs] Update the list of commands on Core Concepts page", "files": ["docs/get_started/core_concepts.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add `env` and `distributed` and add descriptions\r\n-- Format transformers library as `transformers`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T18:36:39Z"}
{"number": 1155, "title": "[docs] Add a description to the oumi env command.", "files": ["docs/cli/commands.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdd a short description to our `oumi env` command.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T18:26:50Z"}
{"number": 1154, "title": "[docs] Put `-m oumi train` on single line", "files": ["docs/get_started/quickstart.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T18:06:38Z"}
{"number": 1151, "title": "updated quickstart with clearer instructions about setting up cloud", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nRewrote the Launching Jobs section of the Quickstart to make it clearer.\r\n\r\n* Specified that this is for launching cloud jobs\r\n* Added links to GCP and Skypilot documentation and some getting started commands\r\n* Added clearly marked warnings about the risks of running cloud jobs (e.g., cost, data loss) \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T17:21:52Z"}
{"number": 1150, "title": "add max_memory_per_gpu and device_map params to lm eval harness", "files": ["src/oumi/core/configs/params/model_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nLM evaluation harness accepts device_map and max_memory_per_gpu kwargs, and these are very helpful when you're trying to evaluate large models. This PR passes through those kwargs to LM evaluation harness.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T17:22:05Z"}
{"number": 1149, "title": "Add a comment to Omegaconf yaml files with config info", "files": ["configs/README.md", "configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/bulk_inference/mistral_small_infer.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/examples/misc/hello_world_gcp_job.yaml", "configs/examples/misc/hello_world_polaris_job.yaml", "configs/examples/misc/vllm_polaris_job.yaml", "configs/projects/aya/evaluation/eval.yaml", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/chatqa/chatqa_stage1_train.yaml", "configs/projects/chatqa/chatqa_stage2_train.yaml", "configs/projects/chatqa/gcp_job.yaml", "configs/projects/zephyr/evaluation/eval.yaml", "configs/projects/zephyr/sft/full_gcp_job.yaml", "configs/projects/zephyr/sft/full_train.yaml", "configs/projects/zephyr/sft/qlora_gcp_job.yaml", "configs/projects/zephyr/sft/qlora_train.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/gpt2/inference/infer.yaml", "configs/recipes/gpt2/pretraining/gcp_job.yaml", "configs/recipes/gpt2/pretraining/mac_train.yaml", "configs/recipes/gpt2/pretraining/train.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/inference/70b_infer.yaml", "configs/recipes/llama3_1/inference/8b_infer.yaml", "configs/recipes/llama3_1/inference/8b_rvllm_infer.yaml", "configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/polaris_job.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/405b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/llama3_2/inference/1b_infer.yaml", "configs/recipes/llama3_2/inference/1b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/1b_vllm_infer.yaml", "configs/recipes/llama3_2/inference/3b_infer.yaml", "configs/recipes/llama3_2/inference/3b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/3b_vllm_infer.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "configs/recipes/phi3/dpo/fsdp_nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/gcp_job.yaml", "configs/recipes/phi3/dpo/mac_train.yaml", "configs/recipes/phi3/dpo/nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/nvidia_80g_train.yaml", "configs/recipes/phi3/dpo/train.yaml", "configs/recipes/phi3/evaluation/eval.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/phi3/sft/lora_train.yaml", "configs/recipes/phi3/sft/mac_lora_train.yaml", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/smollm/evaluation/135m/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_alpaca_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/inference/135m_infer.yaml", "configs/recipes/smollm/sft/135m/gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_train.yaml", "configs/recipes/smollm/sft/135m/train.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_rvllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/train.yaml", "configs/recipes/vision/llava_7b/inference/infer.yaml", "configs/recipes/vision/llava_7b/inference/vllm_infer.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/inference/vllm_infer.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/sglang_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml", "docs/user_guides/evaluate/leaderboards.md"], "area": "configs", "area_votes": {"docs": 2, "configs": 147}, "body": "# Description\r\n\r\nThis comment identifies the config class that will load the YAML file, for self-documentation purposes. It also renames two files from `*_gcp_job_eval.yaml` to `*_gcp_job.yaml` for consistency, and updates some pip installs to use uv.\r\n\r\n## Related issues\r\n\r\nFixes OPE-894\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-17T19:34:46Z"}
{"number": 1148, "title": "Documentation: Judge | 1st revision (of many)", "files": ["docs/user_guides/judge/built_in_judge.md", "docs/user_guides/judge/custom_infer.md", "docs/user_guides/judge/custom_judges.md", "docs/user_guides/judge/custom_prompt.md", "docs/user_guides/judge/judge.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\n1) Re-written most of the main page\r\n- Intro: Done\r\n- Overview: Done\r\n- When to Use? Done\r\n- Oumi Offerrings: Done\r\n- Built-In Judge: _NEEDS REVISION_\r\n- Custom Judges: _NEEDS REVISION_\r\n\r\n2) Refactored the structure \r\n- Judges Court page --> Renamed to `built_in_judge`; now **_only_** focusing on our built-in judge\r\n- Custom Judges --> refactored and broke down...\r\n   Intro: Adapted and moved to main page (`judge.md`)\r\n   Level 1: Moved to `custom_prompt.md`\r\n   Level 2: Moved to `custom_infer.md`\r\n   Level 3: Deleted\r\n\r\nWhat's next?\r\n- Revise: `judge.md` / Built-In Judge\r\n- Revise: `judge.md` / Custom Judges\r\n- Revise: `custom_prompt.md`\r\n- Revise: `custom_infer.md`\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T18:01:17Z"}
{"number": 1147, "title": "Update quickstart.md with emoji", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T05:28:49Z"}
{"number": 1145, "title": "Remove some notebook E2E tests", "files": ["notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "tests/e2e/test_notebooks.py"], "area": "docs", "area_votes": {"docs": 7}, "body": "# Description\r\n\r\nThis PR removes notebooks which are too expensive/slow to run for E2E tests (ex. they call GPT-4, they start a remote instance). In addition, it restores cell outputs for many of our notebooks.\r\n\r\n## Related issues\r\n\r\nTowards OPE-713\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2025-01-17T06:18:33Z"}
{"number": 1142, "title": "Add a Customizing Oumi page to our docs", "files": ["docs/index.md", "docs/resources/datasets/preference_datasets.md", "docs/resources/datasets/pretraining_datasets.md", "docs/resources/datasets/sft_datasets.md", "docs/resources/datasets/vl_sft_datasets.md", "docs/resources/models/custom_models.md", "docs/user_guides/customization.md"], "area": "docs", "area_votes": {"docs": 7}, "body": "# Description\r\n\r\n\r\n\r\nI've added a `Customizing Oumi` page in our `User Guides` section:\r\n\"image\"\r\n\r\n\r\nThis page covers which classes you can register in the Oumi Registry, as well as how to set our environment variables to use them via the CLI.\r\n\r\nAdditionally, I updated most `Custom ...` pages to point to this tutorial for using the CLI.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-865\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T21:12:50Z"}
{"number": 1141, "title": "[tiny] Looser pandas version requirements", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- The pandas API is very stable, the current constraints is a little bit too strict. Updating to avoid env conflicts\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T20:52:46Z"}
{"number": 1139, "title": "Get all notebooks to run", "files": ["configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "configs/recipes/gpt2/pretraining/mac_train.yaml", "configs/recipes/gpt2/pretraining/train.yaml", "docs/user_guides/infer/infer.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "scripts/polaris/notebooks/Oumi - Multinode Inference on Polaris.ipynb"], "area": "docs", "area_votes": {"configs": 4, "docs": 13}, "body": "# Description\r\n\r\nNotebooks are now bug-free, but additional work is still required to let them run in an automated fashion, and to make them run in under 10 min. For example:\r\n- Some notebooks have runnable cells that spawn GCP nodes, with the expectation that the rest of the notebook is run on GCP\r\n- vLLM inference loads Llama 70B by default, which takes awhile\r\n- For me, the native inference engine on SmolLM 135M with one inference example is taking 3-6 minutes on my Mac. This is way too slow\r\n\r\n## Related issues\r\n\r\nTowards OPE-713\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-16T22:11:06Z"}
{"number": 1138, "title": "VLM e2e integration tests update", "files": ["src/oumi/core/trainers/oumi_trainer.py", "tests/e2e/test_train_e2e.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add Qwen2 VL e2e test for OUMI trainer\r\n-- Start using `pytest.skip()` for skipped tests\r\n-- Disable wandb by default for tests\r\n-- Add new params to test spec\r\n-- Update OUMI trainer to empty CUDA cache before and after model/state saving (shouldn't hurt)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-698\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T20:25:20Z"}
{"number": 1136, "title": "Reduce batch size in Qwen2 VL eval config to reduce chances of OOM", "files": ["configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T20:25:36Z"}
{"number": 1135, "title": "Update readme, first iteration", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Update readme, first iteration\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-833\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T20:25:54Z"}
{"number": 1134, "title": "Updated notebooks and quickstart documentation", "files": ["docs/get_started/quickstart.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\nThis update contains three changes.\r\n\r\n1. Updated quickstart documentation to make the custom dataset code runnable, and to provide more options for launching distributed training.\r\n2. Changed a line of text in the Oumi Finetuning Tutorial Notebook to more accurately reflect the state of the code and avoid potential confusion\r\n3. Refactored YAML generation in the Oumi - A Tour notebook to allow for `tutorial_dir` to be set only once, at the beginning of the notebook.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-17T16:31:31Z"}
{"number": 1132, "title": "[docs] Add an entry for \"Fine-tuning a Vision-Language Model\"", "files": ["docs/resources/datasets/vl_sft_datasets.md", "docs/user_guides/evaluate/standardized_benchmarks.md", "docs/user_guides/infer/infer.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add a top-level entry for \"Fine-tuning a Vision-Language Model\", and cross-link related docs for discoverability.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-733\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T19:00:06Z"}
{"number": 1131, "title": "[Documentation] Evaluate | Standardized Benchmarks page (refactor)", "files": ["docs/user_guides/evaluate/standardized_benchmarks.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Evaluate | Standardized Benchmarks page (refactor)\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T02:18:56Z"}
{"number": 1130, "title": "[docs] Add doc section for MMMU benchmark", "files": ["docs/user_guides/evaluate/standardized_benchmarks.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add basic info about MMMU in docs/user_guides/evaluate/standardized_benchmarks.md\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-733\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T03:13:17Z"}
{"number": 1129, "title": "Update sample VLM eval config", "files": ["README.md", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/eval.yaml", "configs/recipes/vision/qwen2_vl_2b/evaluation/gcp_job.yaml"], "area": "configs", "area_votes": {"docs": 1, "configs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Define eval config for Qwen2 VL w/ `MMMU`\r\n-- Improve Llama 3.2 11B Vision Instruct eval config \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-733\r\nFixes OPE-649\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T01:24:26Z"}
{"number": 1128, "title": "Update our documentation for custom clusters", "files": ["CONTRIBUTING.md", "docs/_doclinks.config", "docs/development/contributing.md", "docs/user_guides/launch/custom_cluster.md", "docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\nI removed one duplicate tutorial.\r\nThe custom cluster tutorial is almost a direct port of the previous notebook. The only addition is a few CLI commands and instructions on how to register the class for use in the CLI.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-860\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T00:25:32Z"}
{"number": 1126, "title": "Remove `target_col: ` from VLM configs", "files": ["configs/recipes/vision/blip2_opt_2.7b/sft/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_full/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_lora/train.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml"], "area": "configs", "area_votes": {"configs": 7}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- No longer required after https://github.com/oumi-ai/oumi/pull/1080\r\n-- Tested OK with several models\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-812\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T23:12:53Z"}
{"number": 1124, "title": "Review default values", "files": ["docs/user_guides/train/configuration.md", "src/oumi/core/configs/params/fsdp_params.py", "src/oumi/core/configs/params/peft_params.py", "src/oumi/core/configs/params/training_params.py", "tests/unit/core/test_distributed.py"], "area": "configs", "area_votes": {"docs": 1, "configs": 3}, "body": "# Description\r\n\r\nReview our config's default values, and update some to a more reasonable value (or one that's equivalent to HF's defaults if we don't have a strong opinion).\r\n\r\n## Related issues\r\n\r\nFixes OPE-819\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2025-01-17T17:29:25Z"}
{"number": 1123, "title": "[docs] Multimodal SFT datasets: minor updates", "files": ["docs/resources/datasets/vl_sft_datasets.md", "src/oumi/datasets/vision_language/coco_captions.py", "src/oumi/datasets/vision_language/flickr30k.py", "src/oumi/datasets/vision_language/llava_instruct_mix_vsft.py", "src/oumi/datasets/vision_language/vqav2_small.py"], "area": "data", "area_votes": {"docs": 1, "data": 4}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Call `@register_dataset` in custom dataset example\r\n-- Add basic class docstrings for custom MM datasets.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-733\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T20:56:08Z"}
{"number": 1121, "title": "[docs] Add badges, smaller logos, and nit fix to notebook", "files": ["docs/_static/logo/oumi_logo_dark.png", "docs/_static/logo/oumi_logo_light.png", "docs/index.md", "notebooks/Oumi - Evaluation with MT Bench.ipynb"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Description\r\n\r\n\r\n\r\n- Add badges, smaller logos, and nit fix to notebook\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-838, Towards OPE-828\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T20:28:52Z"}
{"number": 1119, "title": "Fix a circular dependency when invoking the oumi src directory via Python.", "files": ["src/oumi/__main__.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description\r\n\r\n\r\n\r\nYou will get a circular dependency error if you run the simple command:\r\n```\r\npython ./src/oumi\r\n```\r\n\r\nThe reason here is a nuance of the import system. When running the above command, `/src/oumi` becomes the default directory from which packages are imported. Therefore running `import datasets` imports our `oumi.datasets` package instead of the global `datasets` 3rd party package we have installed. This leads to a circular dependency.\r\n\r\nThe simple solution is to move the first system path to the end of the system list when we detect oumi is being called in such a fashion.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-881\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T20:48:15Z"}
{"number": 1118, "title": "Reorganize oumi distributed run logic", "files": ["src/oumi/cli/distributed_run.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Reorganize oumi distributed run cli commands without any logic change\r\n- Better isolate each environments logic, and clarify which variables are used for execution vs validation\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-879\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T19:57:15Z"}
{"number": 1117, "title": "Update `oumi train` path to set `TOKENIZERS_PARALLELISM` variable", "files": ["src/oumi/cli/cli_utils.py", "src/oumi/cli/distributed_run.py", "src/oumi/cli/train.py", "tests/unit/cli/test_cli_utils.py"], "area": "cli", "area_votes": {"cli": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Set `TOKENIZERS_PARALLELISM` and `ACCELERATE_LOG_LEVEL`  if not already configured to suppress scary-looking warning about forked process.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-880\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T19:08:00Z"}
{"number": 1115, "title": "Update quickstart.md", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nModified the quickstart output directory to reflect the current default -- added instructions that changing output directory might be necessary.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T17:20:04Z"}
{"number": 1114, "title": "[docs] address mic feedback from the team", "files": ["docs/conf.py", "docs/get_started/core_concepts.md", "docs/get_started/installation.md", "docs/get_started/quickstart.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\n- Address mic feedback from the team\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T05:51:35Z"}
{"number": 1113, "title": "[docs] Remove placeholder pages", "files": ["docs/get_started/core_concepts.md", "docs/resources/datasets/custom_datasets.md", "docs/resources/datasets/datasets.md", "docs/user_guides/infer/infer.md", "docs/user_guides/train/monitoring.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 6}, "body": "# Description\r\n\r\n\r\n\r\n- Remove placeholder pages and sections, until they are ready to be added\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T05:51:20Z"}
{"number": 1112, "title": "[docs] Update dataset formats page", "files": ["docs/resources/datasets/data_formats.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- First pass at cleaning up the dataset format page\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-809\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T05:51:00Z"}
{"number": 1109, "title": "Set `rank/world_size/device_id=` params in `init_process_group()` to resolve warnings", "files": ["src/oumi/core/distributed.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nSet `torch.distributed.init_process_group(rank=..., world_size=..., device_id=...)` params, which resolved logs warnings like below:\r\n```\r\n [rank2]:[W114 23:19:56.289295099 ProcessGroupNCCL.cpp:4115] [PG ID 0 PG GUID 0 Rank 2]  using GPU 2 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device,or call init_process_group() with a device_id.\r\n```\r\n\r\nStarted to see these warnings recently. The change eliminates them,  no other difference observed. \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-738\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T01:00:27Z"}
{"number": 1108, "title": "Evaluation yaml configs: standardize task name", "files": ["configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml"], "area": "configs", "area_votes": {"configs": 7}, "body": "# Description\r\n\r\n\r\n\r\nWhy am I doing this?\r\n\r\nGoals:\r\n- Standardize to a default `task_name` for all sample evaluation configs \r\n- Choose a task name that executes in O(sec) on a MAC (vs. ~5mins ATM, for mmlu)\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T05:16:05Z"}
{"number": 1106, "title": "Update the Deploy page to use CLI and python instructions.", "files": ["docs/_doclinks.config", "docs/user_guides/launch/deploy.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nI've rewritten this tutorial to be strictly markdown. We no longer copy the notebook for deployment.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nTowards OPE-860\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-15T00:05:02Z"}
{"number": 1104, "title": "Update training logging", "files": ["src/oumi/core/trainers/hf_trainer.py", "src/oumi/train.py"], "area": "training", "area_votes": {"training": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Log trainer type and `transformers` version for all trainers\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T21:14:28Z"}
{"number": 1101, "title": "[docs] Refresh datasets resource page", "files": ["docs/resources/datasets/datasets.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Refresh datasets resource page\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-809\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T19:37:59Z"}
{"number": 1099, "title": "[tiny] Upgrade minimum numpy version to unblock python3.12 installation", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Upgrade minimum numpy version to unblock python3.12 installation\r\n- More info [here](https://numpy.org/doc/2.1/reference/distutils_guide.html)\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-870\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T17:35:27Z"}
{"number": 1098, "title": "Update our Readme with a new header image.", "files": ["README.md", "docs/_static/logo/header_logo.png"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nAdded a header image to the README.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T17:41:59Z"}
{"number": 1097, "title": "[docs] Minor refresh to dataset resource pages", "files": ["docs/resources/datasets/preference_datasets.md", "docs/resources/datasets/pretraining_datasets.md", "docs/resources/datasets/sft_datasets.md", "docs/resources/datasets/vl_sft_datasets.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Description\r\n\r\n\r\n\r\n- Minor refresh to dataset resource pages\r\n- Add description in the intro and fix renamed base classes\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-809\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T17:47:50Z"}
{"number": 1096, "title": "[docs] Add docs guide page", "files": ["docs/development/docs_guide.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n- Add docs guide page with instructions to edit and build the docs locally\r\n- Copied @taenin & @wizeng23 's guide\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T17:51:29Z"}
{"number": 1095, "title": "[Documentation] Evaluate | Main Page (small refactor)", "files": ["docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/generative_benchmarks.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nEvaluate | Main Page (small refactor)\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T06:01:18Z"}
{"number": 1094, "title": "[docs] remove unused page, fix links", "files": ["docs/development/git_workflow.md", "docs/index.md", "docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/leaderboards.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Description\r\n\r\n\r\n\r\n- Fix include links in evaluation pages\r\n- Remove git workflow page, moved to wiki \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T05:49:34Z"}
{"number": 1093, "title": "Add GitHub Actions workflow for doctests", "files": [".github/workflows/doctests.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add GitHub Actions workflow for doctests\r\n- For now, it just includes running pre-commit checks, as they may cause the build to break\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T05:26:45Z"}
{"number": 1092, "title": "[tiny] Fix precommit", "files": ["docs/user_guides/infer/cli_reference.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Fix pre-commit\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T04:48:49Z"}
{"number": 1091, "title": "Add timeout for unit & integration tests", "files": [".github/workflows/gpu_tests.yaml", ".github/workflows/pretest.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 3}, "body": "# Description\r\n\r\n\r\n\r\n- Add timeout for unit & integration tests at 5min (the most time-consuming test takes less than 2min right now)\r\n- The goal is prevent excessive CI resources usage (especially GPU checks). One recent PR contained a bug that let the tests run for multiple hours\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T05:26:18Z"}
{"number": 1089, "title": "[Documentation] Evaluate | Main Page (revision)", "files": ["docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/evaluation_config.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\nDocumentation: Evaluate | Main Page (major revision)\r\n\r\n**Core updates:**\r\n1. Revised overview and converted it to a section of 5 bullet points motivating why users should use Oumi for their evaluations\r\n2. Mentioned underlying platforms we are integrating with on the very top\r\n3. Only kept the very basic yaml config and moved it to QuickStart. Everything else config-related went to a (new) dedicated page \"Evaluation config\"\r\n4. Results and Logging: Broke it down to 3 subsections. \r\n- Evaluation Results: Added more details regarding how our results look like (schema)\r\n- Reproducibility Metadata: Highlighted & motivated more that Oumi evals are reproducible\r\n- Weights & Biases: Added more context regarding what Wandb is and its importance. \r\n5. (new) \"Evaluation config\" (=yaml) page\r\n- Motivated a bit why we use yaml \r\n- Added more configuration options (not all, that's impossible), including inference_engine and inference_remote_params.\r\n\r\n(i) Addressed 90-95% of [Manos feedback](https://docs.google.com/document/d/13WKnBJG2y2SGPGlvl9QyYoG-WKS_pCOBi2XzD7SRpZk) \r\n\r\n**Not done:**\r\n- Updating the term \"Standardized Benchmarks\"\r\n- Moving LM Harness generative benchmarks into Generative page (not relevant to this page though); Needs further scoping (from Kostas) and discussion with Oussama. Likely to postpone after launch. \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T04:05:54Z"}
{"number": 1088, "title": "Remove `pytest.mark.skip()` for basic e2e tests", "files": ["pyproject.toml", "tests/e2e/test_train_e2e.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n-- I have no context on this  `@pytest.mark.skip ` but removing it seems like the right thing to do\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-698\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T16:48:31Z"}
{"number": 1085, "title": "[Minor] Notebook typo", "files": ["notebooks/Oumi - Running Jobs Remotely.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-13T22:04:20Z"}
{"number": 1083, "title": "[doc] Oumi Models (part-1)", "files": ["docs/resources/models/models.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\nSolidify exposure or Oumi models in Docs.\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T16:14:40Z"}
{"number": 1082, "title": "[Super Nit Doc Update] environments.md", "files": ["docs/user_guides/train/environments/environments.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nRemoving these dots. Dont look good.\r\nhttps://oumi.ai/docs/latest/user_guides/train/environments/environments.html\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-13T18:00:06Z"}
{"number": 1079, "title": "Add e2e tests for running tutorial notebooks", "files": ["pyproject.toml", "tests/e2e/test_notebooks.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add e2e tests for running tutorial notebooks\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-781\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-13T17:36:57Z"}
{"number": 1076, "title": "[Doc > Quickstart] Should we add links to guides for better discoverability?", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "\u2026 discoverability?\r\n\r\n# Description\r\n\r\n\r\n\r\nI was looking for a section on how to launch jobs and (surprisingly - no pan intended) I missed it. \r\nShould we add one sentence at the end of Inference, Evaluation, Launching Jobs quickstart and link to the relevant user guide?\r\nI know there are trade-offs here and this will increase the quickstart page's real-estate so... no strong opinion, just a suggestion. \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-11T23:58:21Z"}
{"number": 1075, "title": "Specify `engine: NATIVE` is inference configs", "files": ["configs/examples/bulk_inference/mistral_small_infer.yaml", "configs/recipes/gpt2/inference/infer.yaml", "configs/recipes/llama3_1/inference/70b_infer.yaml", "configs/recipes/llama3_1/inference/8b_infer.yaml", "configs/recipes/llama3_2/inference/1b_infer.yaml", "configs/recipes/llama3_2/inference/3b_infer.yaml", "configs/recipes/smollm/inference/135m_infer.yaml"], "area": "configs", "area_votes": {"configs": 7}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- For clarity, let's have explicit engine specified .\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-11T23:56:34Z"}
{"number": 1072, "title": "[tiny][docs] Update recipes page", "files": ["docs/api/oumi.utils.rst", "docs/resources/recipes.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\nQuestion: thoughts on deleting the inference section? It mostly repeats lines from the section above.\r\n\r\n## Related issues\r\n\r\nTowards OPE-810\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-14T01:18:06Z"}
{"number": 1071, "title": "[Evaluation] HF Leaderboards yaml files", "files": ["configs/projects/aya/evaluation/gcp_job.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v1_gcp_job_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_eval.yaml", "configs/recipes/smollm/evaluation/135m/leaderboards/huggingface_leaderboard_v2_gcp_job_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml", "docs/user_guides/evaluate/leaderboards.md"], "area": "configs", "area_votes": {"configs": 11, "docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdding HF Leaderboards yaml files\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-11T21:00:44Z"}
{"number": 1070, "title": "[tiny] [docs] troubleshooting improvements", "files": ["docs/faq/troubleshooting.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n- Tiny changes in the example:\r\nset `max_grad_norm` to 0.5 since the default is 1\r\nuse `warmup_ratio` instead of `warmup_steps` since the former is agnostic to number of steps \r\n\r\n- Added links to tokenizer, input data, and inference engines\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-16T19:03:23Z"}
{"number": 1069, "title": "Define inference configs for more models", "files": ["README.md", "configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/llama3_1/inference/8b_vllm_infer.yaml", "configs/recipes/llama3_2/inference/1b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/3b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml", "configs/recipes/vision/phi3/inference/vllm_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/sglang_infer.yaml", "configs/recipes/vision/qwen2_vl_2b/inference/vllm_infer.yaml", "docs/user_guides/infer/infer.md"], "area": "configs", "area_votes": {"docs": 2, "configs": 10}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Define inference configs for more VLM models, and add to README\r\n-- Add missing entry for Llama 1B in README\r\n-- Update docs to show how to use `oumi infer` for multimodal\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-733, OPE-862\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-11T20:59:50Z"}
{"number": 1068, "title": "Replace assert in `NativeInferenceEngine` with `RuntimeError`", "files": ["src/oumi/inference/native_text_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- For robustness. Shouldn't assert on model responses. \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-11T02:18:18Z"}
{"number": 1067, "title": "Update dev set up instructions to use a Fork.", "files": ["docs/development/dev_setup.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Removed instructions for installing the git CLI. We shouldn't need to tell users how to do this.\r\n- Updated our dev instructions to use a fork + clone as this is what public contributors will need to do.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-11T16:27:25Z"}
{"number": 1066, "title": "Fix various typos in contributing.md", "files": ["CONTRIBUTING.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nSmall typo fixes / rewording.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-11T00:08:31Z"}
{"number": 1065, "title": "SGLang inference documentation", "files": ["configs/recipes/llama3_1/inference/8b_sglang_infer.yaml", "configs/recipes/llama3_1/inference/8b_vllm_infer.yaml", "configs/recipes/llama3_2/inference/1b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/1b_vllm_infer.yaml", "configs/recipes/llama3_2/inference/3b_sglang_infer.yaml", "configs/recipes/llama3_2/inference/3b_vllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml", "docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_engines.md"], "area": "configs", "area_votes": {"configs": 8, "docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add SGLang inference documentation\r\n-- Change port number used in docs from `30000` to `6864` (phone digits corresponding to \"OUMI\") \r\n-- Create more inference configs for text-only models\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-816\r\nTowards OPE-370, OPE-733\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-11T00:41:45Z"}
{"number": 1064, "title": "Add a Github Issues selector for questions and have it redirect to Discord.", "files": [".github/ISSUE_TEMPLATE/config.yml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdd a Github Issues selector for questions and have it redirect to Discord\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T23:23:34Z"}
{"number": 1063, "title": "Nits for cli_reference.md", "files": ["docs/user_guides/infer/cli_reference.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-14T03:55:04Z"}
{"number": 1062, "title": "Add sample trouble shooting for remote jobs.", "files": ["docs/faq/troubleshooting.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nAdd a sample for handling the file mount issue discussed offline.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T23:12:04Z"}
{"number": 1061, "title": "Nits for common_workflows.md", "files": ["docs/user_guides/infer/common_workflows.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Nits\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T23:06:58Z"}
{"number": 1059, "title": "Minor updates to Launch.md", "files": ["docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nGive a better summary of a typical workflow.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T21:46:22Z"}
{"number": 1058, "title": "[docs] Update docs/user_guides/infer/infer.md", "files": ["docs/user_guides/infer/infer.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Switch from `meta-llama/Meta-Llama-3.2-1B-Instruct` to `SmolLM2`. The `SmolLM2` model is faster to load and doesn't require Hub allowlisting (prone to causing issues)\r\n-- Provide Multimodal inference example\r\n-- Format examples\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-733\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T22:31:35Z"}
{"number": 1057, "title": "Nit fixes for acknowledgements.md", "files": ["docs/about/acknowledgements.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Fixing nits. No major changes\r\n\r\n# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T23:07:17Z"}
{"number": 1056, "title": "[tiny] Update tutorials page", "files": ["docs/get_started/tutorials.md", "notebooks/Oumi - Custom Judge.ipynb"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\nFixes Colab links. However, the links won't work until the repo is public\r\n\r\n## Related issues\r\n\r\nTowards OPE-810, OPE-856\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2025-01-10T21:13:32Z"}
{"number": 1055, "title": "[evaluations/generative_benchmark] Removing notebook link", "files": ["docs/user_guides/evaluate/generative_benchmarks.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nThis notebook was developed before we properly integrated with AlpacaEval. ATM, we do NOT need a notebook to run Alpaca, all we need is running a yaml file (as shown right above). \r\n\r\nStill, if users do want to run AlpacaEval outside Oumi, they can use this notebook, but this is also advertised right above (\"If you prefer to use AlpacaEval outside Oumi, we refer you to our example notebook\"), so we do not need to re-link it in our resources IMO.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T19:58:19Z"}
{"number": 1054, "title": "[evaluations/generative_benchmark] Broken link", "files": ["docs/user_guides/evaluate/generative_benchmarks.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T19:50:02Z"}
{"number": 1053, "title": "Remove dangling reference to `jupyter`  in Makefile help", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Missed in https://github.com/oumi-ai/oumi/pull/620\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T19:51:01Z"}
{"number": 1052, "title": "Define WSL in our vscode docs.", "files": ["docs/user_guides/train/environments/vscode.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nWindows Subsystem for Linux = WSL\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T19:16:16Z"}
{"number": 1051, "title": "[tiny] disable unit tests on safe paths", "files": [".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Disable unit tests on safe paths\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T19:48:28Z"}
{"number": 1050, "title": "[docs] Fix contributing and open issue links", "files": ["README.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T19:49:51Z"}
{"number": 1049, "title": "Fix a typo in the VS Code environment page.", "files": ["docs/user_guides/train/environments/vscode.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T19:08:13Z"}
{"number": 1048, "title": "[docs] Add Gemini to the list of supported inference API-s, and sort them", "files": ["docs/user_guides/infer/infer.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T18:44:36Z"}
{"number": 1047, "title": "Fix issues in most notebooks", "files": ["notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "src/experimental/notebooks/Oumi - Datasets Tutorial.ipynb", "src/oumi/datasets/__init__.py", "src/oumi/datasets/evaluation/alpaca.py"], "area": "docs", "area_votes": {"docs": 9, "data": 1}, "body": "# Description\r\n\r\n- Update configs to they output saved models to the appropriate directory\r\n- Switch to using SmolLM in more places\r\n- Switched to mmlu_college_computer_science as the eval benchmark. While it's not as useful of a metric, mmlu/hellaswag take nearly 10 minutes to run which may be a bit slow for users\r\n\r\n## Related issues\r\n\r\nTowards OPE-713\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-10T18:55:51Z"}
{"number": 1046, "title": "Fix the second python snippet in the train page.", "files": ["docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Update to use a real config\r\n- Fix import to actually work.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T18:39:30Z"}
{"number": 1045, "title": "Fix the broken python text snippet on the train page.", "files": ["docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nTrain is top level, not in core.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-666\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T18:25:16Z"}
{"number": 1044, "title": "Define more terms in our training docs.", "files": ["docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nDefine DDP and FSDP in the training page. Also use an emdash for flavor in the intro.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T18:24:45Z"}
{"number": 1043, "title": "Skip running GPU tests on low-risk code paths", "files": [".github/workflows/gpu_tests.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Skip running GPU tests on low-risk code paths\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T18:23:33Z"}
{"number": 1042, "title": "Define CLI in our quickstart.", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nWrite out the CLI acronym completely the first time it's used.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T18:05:02Z"}
{"number": 1041, "title": "[docs][tiny] remove termynal from sphinx conf", "files": ["docs/conf.py"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- It's not used anymore, disabling for now to simplify install instructions \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T19:01:21Z"}
{"number": 1038, "title": "[docs][tiny] fix examples in inference guide", "files": ["docs/user_guides/infer/common_workflows.md", "docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_engines.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description\r\n\r\n\r\n\r\n- fix examples in inference guide\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T15:58:31Z"}
{"number": 1037, "title": "[docs] Fix feedback on training and inference user guides", "files": ["docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/judge/judge.md", "docs/user_guides/launch/launch.md", "docs/user_guides/train/configuration.md", "docs/user_guides/train/environments/environments.md", "docs/user_guides/train/environments/notebooks.md", "docs/user_guides/train/train.md", "docs/user_guides/train/training_methods.md", "notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb", "notebooks/Oumi - Evaluation with MT Bench.ipynb"], "area": "docs", "area_votes": {"docs": 11}, "body": "# Description\r\n\r\n\r\n\r\n-  Fix feedback on training and inference user guides\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T08:02:52Z"}
{"number": 1035, "title": "[tiny] Cleanup judge engine builder & fix circular dep", "files": ["src/oumi/judges/base_judge.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Cleanup judge engine builder & fix circular dependency\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-836\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T05:06:20Z"}
{"number": 1034, "title": "[tiny] Update GitHub workflows", "files": [".github/ISSUE_TEMPLATE/bug-report.yaml", ".github/ISSUE_TEMPLATE/feature-request.yaml", ".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 3}, "body": "# Description\r\n\r\n\r\n\r\n- Log test duration for debugging\r\n- Change the default issue tags to match the ones we have on linear\r\n- Update the bug report form to ask for `oumi env` output\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T04:07:39Z"}
{"number": 1032, "title": "Support constrained decoding in SGLang inference engine", "files": ["src/oumi/inference/sglang_inference_engine.py", "tests/unit/inference/test_sglang_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Propagate `GuidedDecodingParams` to SGLang sampling params \r\n-- Run manual sanity checks with local Llama Vision model\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-370\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T21:07:23Z"}
{"number": 1026, "title": "Update `BaseConfig.from_yaml` to also support Path", "files": ["src/oumi/core/configs/base_config.py", "tests/e2e/test_train_e2e.py", "tests/unit/core/configs/test_config.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- for consistency with to_yaml()\r\n \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-698\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T04:12:14Z"}
{"number": 1025, "title": "VLM docs update", "files": ["README.md", "configs/recipes/vision/phi3/README.md", "configs/recipes/vision/qwen2_vl_2b/README.md", "configs/recipes/vision/smolvlm/README.md", "docs/resources/models/supported_models.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Add BLIP2 to the list of supported models\r\n-- Add README files for Qwen2-VL, Phi3, and SmolVLM models\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-733\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T23:04:07Z"}
{"number": 1024, "title": "Remove our docs password from the readme.", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T21:01:32Z"}
{"number": 1022, "title": "Add structured outputs support to gemini/vertex engines", "files": ["src/oumi/inference/gcp_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Add supported for structured decoding using GoogleVertexInferenceEngine\r\n- The API requires special handling by removing all the `$refs` from the json schema. Otherwise the implementation is similar to OpenAI's \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-10T07:03:05Z"}
{"number": 1020, "title": "[tiny] Update debug datasets", "files": ["src/oumi/datasets/debug.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Update the pretraining debug dataset to use the BasePretrainingClass\r\n- Update all datasets to return unique samples to help with debugging \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T18:15:47Z"}
{"number": 1019, "title": "[docs] Address misc docs feedback", "files": ["README.md", "configs/recipes/llama3_1/README.md", "docs/get_started/core_concepts.md", "docs/get_started/quickstart.md", "docs/get_started/tutorials.md", "docs/index.md", "docs/user_guides/train/monitoring.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "src/oumi/__init__.py"], "area": "docs", "area_votes": {"docs": 9}, "body": "# Description\r\n\r\n\r\n\r\n- Address misc docs feedback from the team\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-833\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T18:41:05Z"}
{"number": 1018, "title": "Update default installation instructions to pypi", "files": ["docs/get_started/installation.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nMoved PyPi to be the default install method. Added instructions to use a venv when installing to prevent issues. Opted to ignore conda for now as that set up is more involved.\r\n\r\nAdded instructions for Linux / MacOs and Windows:\r\n\r\n\"image\"\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-737\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T18:13:43Z"}
{"number": 1017, "title": "Update the registry to load registered core values upon use.", "files": ["src/oumi/builders/data.py", "src/oumi/core/registry/registry.py", "tests/unit/core/test_registry.py"], "area": "other", "area_votes": {"other": 2}, "body": "# Description\r\n\r\n\r\n\r\nCurrently the registry is empty if it's the only module that's imported. Registering our default datasets, clouds, models, and judge configs requires importing the appropriate packages after importing the registry.\r\n\r\nThis change adds a decorator to initialize these values once upon using a method in the registry.\r\n\r\nThis also fixes breaking tests, such as https://github.com/oumi-ai/oumi/blame/main/tests/integration/cli/test_judge_e2e.py \r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-836\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T18:03:45Z"}
{"number": 1016, "title": "Add docstrings how to start vLLM and SGLang servers for `Llama-3.2-11B-Vision-Instruct`", "files": ["configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards  OPE-816, OPE-733\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T16:42:06Z"}
{"number": 1014, "title": "Update lists of supported VLM-s in README and docs", "files": ["README.md", "docs/resources/models/supported_models.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-733\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T02:34:18Z"}
{"number": 1013, "title": "Configure `asyncio_default_fixture_loop_scope` to reduce pytest warnings", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Using the future default value: https://pytest-asyncio.readthedocs.io/en/latest/reference/configuration.html\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T17:39:01Z"}
{"number": 1012, "title": "Cleanup experimental folder", "files": ["scripts/polaris/notebooks/Oumi - Multinode Inference on Polaris.ipynb", "scripts/polaris/notebooks/Oumi - Tuning Llama.ipynb", "scripts/pretokenize/README.md", "scripts/pretokenize/process_dataset.py", "scripts/pretokenize/sky.yaml", "src/experimental/Oumi - Build Zephyr 7B.ipynb", "src/experimental/Oumi - Colab Setup Example.ipynb", "src/experimental/Run-ChatRAG-Bench.ipynb", "src/experimental/llama_cpp_inference.py", "src/experimental/notebooks/Oumi - Datasets Tutorial.ipynb"], "area": "docs", "area_votes": {"docs": 7, "infra": 2, "other": 1}, "body": "# Description\r\n\r\n\r\n\r\n- Remove experimental notebooks, move to scratch\r\n- Move pre tokenize tool to script (and add readme)\r\n- Move Polaris notebooks to Polaris folder\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T00:38:28Z"}
{"number": 1010, "title": "[docs] Update readme", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n- \r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-08T22:31:14Z"}
{"number": 1009, "title": "Fix readme.", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nPrecommits are failing with trailing whitespace.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-08T21:59:40Z"}
{"number": 1008, "title": "Updated our ascii logo", "files": ["src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\nUpdate our ascii logo.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-736\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-08T22:15:32Z"}
{"number": 1007, "title": "Freeze Python package versions", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update package deps to the following format `foo>=CURR_VER,=1.25.2,<1.26` to freeze `MAJOR.MINOR` but allow patches\r\n-- Remove Azure from `cloud` dependencies. It's incompatible with GCP and others (`0.6.1` vs `0.7.0`)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-786\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-09T16:23:47Z"}
{"number": 1002, "title": "[docs][tiny] Fix broken links, update sft dataset examples", "files": ["README.md", "docs/get_started/core_concepts.md", "docs/get_started/quickstart.md", "docs/user_guides/train/configuration.md", "docs/user_guides/train/train.md", "docs/user_guides/train/training_methods.md"], "area": "docs", "area_votes": {"docs": 6}, "body": "# Description\r\n\r\n\r\n\r\n- Fix a couple broken links in the readme\r\n- Default to show `text_sft` in the docs instead of `text_sft_jsonl` (both are equivalent)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-07T17:34:54Z"}
{"number": 998, "title": "Create alias for json/jsonl datasets", "files": ["src/oumi/datasets/sft/sft_jsonlines.py", "src/oumi/datasets/vision_language/vision_jsonlines.py", "tests/integration/datasets/test_sft_vision_datasets_load_datasets.py", "tests/unit/builders/test_build_data.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Description\r\n\r\n\r\n\r\n- The dataset classes support both json and jsonl. This PR adds a generic `text_sft` and `vl_sft` aliases for `text_sft_jsonl` and `vision_language_jsonl` datasets\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nCloses OPE-805\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-07T01:31:00Z"}
{"number": 997, "title": "Fix regression in CLI speed", "files": ["src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nRemoved a dependency on `oumi.core` to speed up the CLI. This was introduced in #946 \r\nDisabled our ASCII art for the distributed command.\r\n\r\nBefore:\r\n```\r\ntime oumi       \r\noumi  2.12s user 2.45s system 304% cpu 1.500 total\r\n```\r\n\r\nAfter:\r\n```\r\ntime oumi\r\noumi  0.09s user 0.03s system 97% cpu 0.129 total\r\n```\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-06T23:33:05Z"}
{"number": 996, "title": "Update README.md - Describe Oumi's most common capabilities", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\nDescribe Oumi's most common capabilities to inspire the reader\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-08T21:03:21Z"}
{"number": 995, "title": "Update README.md - Better highlight features & nits", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\nSeveral nits to improve README.md and ensure that features are highlighted better.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-08T14:53:02Z"}
{"number": 993, "title": "[Evaluation] Updating Documentation (1st pass)", "files": ["docs/user_guides/evaluate/custom_lm_harness.md", "docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/generative_benchmarks.md", "docs/user_guides/evaluate/leaderboards.md", "docs/user_guides/evaluate/mcqa.md", "docs/user_guides/evaluate/standardized_benchmarks.md"], "area": "docs", "area_votes": {"docs": 6}, "body": "# Description\r\n\r\n\r\n\r\nUpdating the documentation of our evaluation section. \r\n\r\n*** Outline Before *** \r\n\r\n`Evaluation`\r\n- `mcqa`\r\n- `generative`\r\n- `custom_lm_harness`\r\n\r\n*** Outline Now *** \r\n\r\n`Evaluation` _<-- updated_\r\n- `mcqa` _<-- renamed to `standardized_benchmarks` and added content_ \r\n- `generative` _<-- renamed to `generative_benchmarks`; NOT updated yet_\r\n- `custom_lm_harness` _<-- (X) DELETED_\r\n- `leaderboards` _<-- (+) ADDED; NO content yet_\r\n\r\n*** Next PR *** \r\n\r\n1) `generative_benchmarks`: update\r\n2) `leaderboards`: add content\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # N/A\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-04T01:10:16Z"}
{"number": 991, "title": "`data.train.text_col` cleanup for VLM-s", "files": ["configs/recipes/vision/blip2_opt_2.7b/sft/train.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_train.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml", "src/oumi/core/datasets/vision_language_dataset.py", "tests/unit/core/datasets/test_vision_language_dataset.py"], "area": "configs", "area_votes": {"configs": 6, "data": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Update `src/oumi/core/datasets/vision_language_dataset.py` to start populating `target_col`. At minimum, it can be useful for debugging.\r\n-- Remove warning about `useless col, but required by the trainer`\r\n-- Replace `text` with `prompt` as the default value of `target_col` for all VLM-s. The name `prompt` is more commonly used and more clear.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards  OPE-697, OPE-699, OPE-673, OPE-688\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-04T00:32:46Z"}
{"number": 990, "title": "Define `configs/recipes/vision/phi3/sft/trl_gcp_job.yaml` for Phi3 Vision", "files": ["configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/phi3/sft/trl_gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- Define TRL_SFT config for Phi3 Vision\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-699\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-03T19:29:54Z"}
{"number": 989, "title": "Add `torchvision` dependency", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description\r\n\r\n\r\n\r\n\r\n-- it's required by some VLM-s e.g., Phi3 Vision\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-699\r\n\r\n\r\n\r\n## Before submitting\r\n\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-03T19:05:36Z"}
{"number": 988, "title": "[docs] Add training monitoring documentation page", "files": ["docs/user_guides/train/monitoring.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n-  Add training monitoring documentation page\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-03T05:37:51Z"}
{"number": 987, "title": "Update vision/smollm config names", "files": [".github/pull_request_template.md", ".vscode/launch.json", "README.md", "configs/recipes/smollm/evaluation/135m/eval.yaml", "configs/recipes/smollm/evaluation/135m/gcp_job.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml", "configs/recipes/smollm/evaluation/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/sft/135m/gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml", "configs/recipes/smollm/sft/135m/quickstart_train.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_gcp_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/train.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_sglang_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_vllm_infer.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_train.yaml", "configs/recipes/vision/llava_7b/sft/7b_trl_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/train.yaml", "configs/recipes/vision/llava_7b/sft/trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/train.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/train.yaml", "configs/recipes/vision/smolvlm/sft/gcp_job.yaml", "configs/recipes/vision/smolvlm/sft/train.yaml", "docs/get_started/quickstart.md", "docs/resources/recipes.md", "docs/user_guides/launch/launch.md", "docs/user_guides/train/configuration.md", "docs/user_guides/train/train.md"], "area": "configs", "area_votes": {"docs": 7, "other": 1, "configs": 25}, "body": "# Description:\r\n\r\n- Updates vision/smollm config names to be consistent with the others. This includes having the job configs in the same dirs as their corresponding train/eval configs, and having the suffix of all files be their class (ex. ..._train.yaml, ..._gcp_job.yaml). I made sure to ctrl+f replace all old paths with new paths.\r\n- Simplifies redundant parts of config names (ex. `configs/recipes/vision/llava_7b/sft/7b_sft_train.yaml` -> `configs/recipes/vision/llava_7b/sft/train.yaml`).\r\n- Deleted duplicate job configs\r\n\r\n## Related issues\r\n\r\nFixes OPE-776\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-03T00:20:29Z"}
{"number": 984, "title": "[docs] Minor updates to datasets intro page", "files": ["docs/resources/datasets/datasets.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Minor updates to datasets intro page\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-02T22:43:55Z"}
{"number": 982, "title": "Remove internal GCS mentions", "files": ["configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/misc/dev_gcp_job.yaml", "configs/projects/zephyr/sft/full_gcp_job.yaml", "configs/projects/zephyr/sft/qlora_gcp_job.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "docs/get_started/core_concepts.md", "docs/user_guides/launch/launch.md", "docs/user_guides/train/environments/notebooks.md", "docs/user_guides/train/environments/vscode.md"], "area": "configs", "area_votes": {"configs": 17, "docs": 4}, "body": "# Description:\r\n\r\nI removed our `storage_mounts` from all jobs except GCP bulk inference, where it's required. I also moved our instructions about our internal GCS buckets to: https://github.com/oumi-ai/oumi/wiki/Oumi-Core-Team-Resources, and instructions about mounting cloud storage to the launch docs. Keeping a copy of 405B model weights in a GCS bucket for faster copying may be too niche and specific to our team, so I'm not sure if it's worth putting it in the official documentation.\r\n\r\nAlso addresses leftover comments from #975 and makes other misc. cleanup fixes.\r\n\r\n## Related issues\r\n\r\nFixes OPE-754\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2025-01-02T23:13:33Z"}
{"number": 980, "title": "[docs] update inference engine user guide", "files": ["docs/user_guides/infer/cli_reference.md", "docs/user_guides/infer/common_workflows.md", "docs/user_guides/infer/configuration.md", "docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/infer/local_inference.md", "docs/user_guides/infer/remote_inference.md"], "area": "docs", "area_votes": {"docs": 7}, "body": "# Description:\r\n\r\n\r\n\r\n- Merge local and remote inference engines into a single page\r\n- Refresh inference engines page\r\n- Update default models and other minor nits\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-02T00:11:50Z"}
{"number": 979, "title": "[docs] Add supported models page", "files": ["docs/resources/models/supported_models.md", "docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/mcqa.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description:\r\n\r\n\r\n\r\n- Add supported models page\r\n- Add placeholder supported mcqa evals page\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-01T22:47:04Z"}
{"number": 978, "title": "[docs] Add infer common workflows page", "files": ["docs/user_guides/infer/common_workflows.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Add infer common workflows page\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-01T19:55:24Z"}
{"number": 977, "title": "[docs][tiny] Remove train/common_workflows page", "files": ["docs/user_guides/train/common_workflows.md", "docs/user_guides/train/environments/environments.md", "docs/user_guides/train/environments/local.md", "docs/user_guides/train/environments/notebooks.md", "docs/user_guides/train/environments/vscode.md", "docs/user_guides/train/training_methods.md"], "area": "docs", "area_votes": {"docs": 6}, "body": "# Description:\r\n\r\n\r\n\r\n- Remove train/common_workflows page as it overlaps too much in content with the main train user_guide\r\n- Replace/remove references in other pages\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-01T18:58:31Z"}
{"number": 976, "title": "[docs] misc clarifications in train, judge, dataset pages", "files": ["docs/resources/datasets/datasets.md", "docs/user_guides/judge/judges_court.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description:\r\n\r\n\r\n\r\n- Misc clarifications in train, judge, dataset pages\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2025-01-01T18:52:44Z"}
{"number": 974, "title": "[docs] Add doc page for generative evals", "files": ["docs/user_guides/evaluate/evaluate.md", "docs/user_guides/evaluate/generative.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description:\r\n\r\n\r\n\r\n- Add doc page for generative evals\r\n- Fix broken links\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-31T20:27:33Z"}
{"number": 973, "title": "[docs] Update tutorials page", "files": ["docs/_doclinks.config", "docs/get_started/tutorials.md", "docs/user_guides/infer/infer.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description:\r\n\r\n\r\n\r\n- Update tutorials page\r\n- Add links to download, view on GitHub for each config, or open in collab\r\n- Config links are checked for availability at docs build time to prevent errors\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-31T19:19:25Z"}
{"number": 972, "title": "[docs] Update recipes page", "files": ["docs/_docsummaries.sh", "docs/conf.py", "docs/resources/recipes.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description:\r\n\r\n\r\n\r\n- Update recipes page\r\n- Recipes are no longer auto-generated. The page is manually constructed to allow more flexibility in organizing the models and configs\r\n- Add links to download and view on GitHub for each config\r\n- Config links are checked for availability at docs build time to prevent errors\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-31T06:09:04Z"}
{"number": 971, "title": "[docs] Cleanup evaluation guide page", "files": ["docs/user_guides/evaluate/custom_lm_harness.md", "docs/user_guides/evaluate/evaluate.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description:\r\n\r\n\r\n\r\n- Cleanup evaluation page\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-31T01:51:05Z"}
{"number": 970, "title": "[docs] Add vscode environment doc page", "files": ["docs/user_guides/train/environments/vscode.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Add vscode environment doc page\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-31T01:50:33Z"}
{"number": 969, "title": "[docs] Add documentation page for notebook environment", "files": ["docs/user_guides/train/environments/notebooks.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Add documentation page for notebook environment\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-31T01:02:00Z"}
{"number": 967, "title": "[docs] Update documentation index page", "files": ["docs/index.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Update documentation index page\r\n  - Add more quickstart links\r\n  - Break-up the big text paragraph into more readable sections\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T22:48:16Z"}
{"number": 966, "title": "[docs] Add documentation page for local training", "files": ["docs/user_guides/train/environments/cloud.md", "docs/user_guides/train/environments/environments.md", "docs/user_guides/train/environments/local.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description:\r\n\r\n\r\n\r\n- Add documentation page for local training\r\n- Add placeholder page for cloud training\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T22:26:15Z"}
{"number": 965, "title": "[docs] Refresh judge user guide", "files": ["docs/user_guides/judge/custom_judges.md", "docs/user_guides/judge/judge.md", "docs/user_guides/judge/judges_court.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description:\r\n\r\n\r\n\r\n- Refresh judge user guide pages\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T21:07:26Z"}
{"number": 963, "title": "[docs] Update doc generation config", "files": ["docs/conf.py"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Enable auto anchors generations for links\r\n- Ignore experimental models section\r\n- Enable nitpicky mode for more build-time warnings (broken links, etc)\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T18:56:25Z"}
{"number": 962, "title": "[docs][tiny] Update links", "files": ["docs/_doclinks.config", "docs/get_started/core_concepts.md", "docs/get_started/quickstart.md", "docs/index.md", "docs/user_guides/launch/launch.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description:\r\n\r\n\r\n\r\n- Update misc broken links caused by refactor\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T18:34:16Z"}
{"number": 961, "title": "[docs] Refresh evaluate user guide", "files": ["docs/user_guides/evaluate/custom_lm_harness.md", "docs/user_guides/evaluate/generative.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description:\r\n\r\n\r\n\r\n- Refresh evaluate user guide\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T18:20:48Z"}
{"number": 960, "title": "[docs] Refresh datasets resource section", "files": ["docs/datasets/custom_vl_datasets.md", "docs/datasets/preference_tuning.md", "docs/resources/datasets/custom_datasets.md", "docs/resources/datasets/data_formats.md", "docs/resources/datasets/datasets.md", "docs/resources/datasets/preference_datasets.md", "docs/resources/datasets/pretraining_datasets.md", "docs/resources/datasets/sft_datasets.md", "docs/resources/datasets/vl_sft_datasets.md", "docs/user_guides/evaluate/evaluate.md"], "area": "docs", "area_votes": {"docs": 10}, "body": "# Description:\r\n\r\n\r\n\r\n- Refresh datasets resource section\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T18:21:00Z"}
{"number": 959, "title": "[docs] Refresh models resource section", "files": ["docs/models/custom_models.md", "docs/resources/models/custom_models.md", "docs/resources/models/models.md", "docs/resources/models/supported_models.md", "docs/resources/recipes.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "# Description:\r\n\r\n\r\n\r\n- Refresh models resource section\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T18:21:16Z"}
{"number": 958, "title": "[docs] Refresh training user guide", "files": ["docs/user_guides/train/common_workflows.md", "docs/user_guides/train/configuration.md", "docs/user_guides/train/distributed_training.md", "docs/user_guides/train/environments/environments.md", "docs/user_guides/train/environments/local.md", "docs/user_guides/train/environments/notebooks.md", "docs/user_guides/train/environments/vscode.md", "docs/user_guides/train/monitoring.md", "docs/user_guides/train/performance_optimization.md", "docs/user_guides/train/quantization.md", "docs/user_guides/train/train.md", "docs/user_guides/train/trainers.md", "docs/user_guides/train/training_config.rst", "docs/user_guides/train/training_methods.md"], "area": "docs", "area_votes": {"docs": 14}, "body": "# Description:\r\n\r\n\r\n\r\n- Refresh training user guide\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T18:21:47Z"}
{"number": 957, "title": "[docs] Refresh infer user guide", "files": ["docs/user_guides/infer/cli_reference.md", "docs/user_guides/infer/common_workflows.md", "docs/user_guides/infer/configuration.md", "docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/infer/local_inference.md", "docs/user_guides/infer/remote_inference.md"], "area": "docs", "area_votes": {"docs": 7}, "body": "# Description:\r\n\r\n\r\n\r\n- Refresh infer user guide\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T18:21:30Z"}
{"number": 956, "title": "[docs][bug] Fix rendering issue in installation page", "files": ["docs/get_started/installation.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Fix rendering issue in installation page\r\n- Avoid relative paths for linking to docs, use absolute myst links\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T18:20:34Z"}
{"number": 955, "title": "[docs][tiny] remove unused pages", "files": ["docs/faq/gpu_sizing.md", "docs/faq/troubleshooting.md", "docs/index.md", "docs/user_guides/train/custom_loss.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Description:\r\n\r\n\r\n\r\n- Remove unused pages\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T04:22:39Z"}
{"number": 954, "title": "[docs][tiny] Update acknowledgement page", "files": ["docs/about/acknowledgements.md", "docs/api/oumi.utils.rst"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description:\r\n\r\n\r\n\r\n- Update acknowledgement page with recently added packages\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-763\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-30T04:17:41Z"}
{"number": 952, "title": "[docs] Add core concepts page", "files": ["docs/conf.py", "docs/get_started/core_concepts.md", "docs/index.md", "pyproject.toml"], "area": "docs", "area_votes": {"docs": 3, "infra": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Add core concepts page\r\n- Enable rendering mermaid diagrams in the docs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-740\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-29T23:04:27Z"}
{"number": 950, "title": "Print telemetry stats in more compact format", "files": ["src/oumi/performance/telemetry.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-27T23:52:33Z"}
{"number": 949, "title": "[Evaluation] Refactor repro info & add repro info in AlpacaEval", "files": ["src/oumi/evaluate.py", "src/oumi/evaluation/alpaca_eval.py", "src/oumi/evaluation/lm_harness.py", "src/oumi/evaluation/save_utils.py", "tests/integration/evaluate/test_evaluate.py"], "area": "evaluation", "area_votes": {"evaluation": 4}, "body": "# Description:\r\n\r\n\r\n\r\nEvaluation: \r\n- Adding repro info in AlpacaEval\r\n- Refactoring repro info into a common (platform-agnostic) function \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-702\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-31T00:10:53Z"}
{"number": 947, "title": "[bug] Fix issue with registered dataset loading", "files": [".vscode/launch.json", "src/oumi/builders/data.py"], "area": "other", "area_votes": {"other": 2}, "body": "# Description:\r\n\r\n\r\n\r\n- Explicitly import `oumi.dataset` in the `oumi.builders.data` module so that registered datasets are discovered / registered at runtime\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nFixes OPE-761\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-27T21:28:13Z"}
{"number": 946, "title": "Only print ASCII log on main process", "files": ["src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Reducing logs noise in multi-node jobs.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-736\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-27T21:01:32Z"}
{"number": 940, "title": "Update fineweb config", "files": ["configs/examples/fineweb_ablation_pretraining/ddp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "scripts/polaris/jobs/fineweb_pt_worker.sh"], "area": "configs", "area_votes": {"configs": 4, "infra": 1}, "body": "# Description:\r\n\r\n- Add a config for DDP\r\n- Copy scheduler params from the paper: https://arxiv.org/pdf/2406.17557\r\n\r\nThis is in preparation for our experiment to pretrain FineWeb on Polaris. Config changes specific to that experiment won't be included in this PR.\r\n\r\n## Related issues\r\n\r\nTowards OPE-40\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2024-12-24T05:51:51Z"}
{"number": 938, "title": "Migrate Sky configs to Oumi job configs", "files": [".vscode/launch.json", "configs/projects/aya/evaluation/gcp_job.yaml", "configs/projects/aya/sft/gcp_job.yaml", "configs/projects/chatqa/chatqa_stage2_train.yaml", "configs/projects/chatqa/gcp_job.yaml", "configs/projects/zephyr/sft/full_gcp_job.yaml", "configs/projects/zephyr/sft/qlora_gcp_job.yaml", "configs/projects/zephyr/sft/qlora_sky_job.yaml", "configs/recipes/gpt2/evaluation/async_gcp_job.yaml", "configs/recipes/gpt2/pretraining/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/phi3/dpo/fsdp_gcp_job.yaml", "configs/recipes/phi3/dpo/gcp_job.yaml", "configs/recipes/phi3/evaluation/gcp_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_gcp_job.yaml", "docs/user_guides/train/distributed_training.md", "tests/unit/core/configs/test_parse_configs.py"], "area": "configs", "area_votes": {"other": 1, "configs": 15, "docs": 1}, "body": "# Description:\r\n\r\nThis requires the following changes:\r\n- Add `resources.cloud: gcp`\r\n- `\"A100:4\"` instead of `{\"A100\": 4}` for resources.accelerators\r\n- Set `resources.use_spot: true` instead of `any_of`\r\n- `working_dir` instead of `work_dir`\r\n- Mount HF token instead of requiring user to set HF env var (null value doesn't work)\r\n- Move GCS mounts from `file_mounts` to `storage_mounts`\r\n- Rename files from `*sky_job.yaml` to `*gcp_job.yaml`\r\n\r\nI also reorganized the `launch.json` items. I tried maintaining file history on `configs/projects/zephyr/sft/qlora_gcp_job.yaml`, but GitHub was acting up for it.\r\n\r\nThis is unit tested in `tests/unit/core/configs/test_parse_configs.py` to ensure the configs can be successfully loaded. I also manually tested some configs. Some of them are broken, but they were already broken and it was not due to this PR. I will fix them in a follow-up PR.\r\n\r\n## Related issues\r\n\r\nFixes OPE-634\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2024-12-23T22:41:08Z"}
{"number": 935, "title": "[docs] Reorganize doc pages", "files": ["README.md", "docs/_doclinks.config", "docs/advanced/customization.md", "docs/api/oumi.cli.rst", "docs/api/oumi.core.cli.rst", "docs/api/oumi.core.rst", "docs/api/oumi.rst", "docs/datasets/custom_vl_datasets.md", "docs/faq/troubleshooting.md", "docs/get_started/configuration.md", "docs/index.md", "docs/models/custom_models.md", "docs/user_guides/evaluate/custom_evaluation.md", "docs/user_guides/evaluate/evaluate.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/train/custom_loss.md", "docs/user_guides/train/distributed_training.md", "docs/user_guides/train/performance_optimization.md", "docs/user_guides/train/quantization.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 20}, "body": "# Description:\r\n\r\n\r\n\r\n- Remove `Advanced Topics`, and move pages to the relevant sections (models, datasets, etc)\r\n- Remove redundant pages\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-23T19:16:50Z"}
{"number": 934, "title": "Add ascii art to all oumi commands in the CLI.", "files": ["src/oumi/cli/main.py"], "area": "cli", "area_votes": {"cli": 1}, "body": "# Description:\r\n\r\n\r\n\r\nDisplay an ascii version of the Oumi logo when running commands in the CLI:\r\n\r\n\"Screenshot\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-736\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-23T16:43:12Z"}
{"number": 933, "title": "Clean up configs", "files": ["configs/examples/fineweb_ablation_pretraining/deepspeed/accelerate.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/ds_config.json", "configs/examples/fineweb_ablation_pretraining/deepspeed/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/train.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "notebooks/Oumi - Tuning Llama.ipynb", "scripts/polaris/jobs/fineweb_pt_worker.sh"], "area": "configs", "area_votes": {"configs": 9, "docs": 1, "infra": 1}, "body": "# Description:\r\n\r\n- Delete FineWeb deepspeed configs, as we haven't touched it in months, and it's one of the only configs that can't be migrated off the Accelerate launcher.\r\n- Modify Llama eval configs to only use `model.model_name`.\r\n\r\n## Related issues\r\n\r\nFixes OPE-76\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2024-12-23T04:08:01Z"}
{"number": 932, "title": "Extend dev setup guide", "files": ["CONTRIBUTING.md", "Makefile", "docs/development/dev_setup.md", "docs/development/style_guide.md", "docs/get_started/installation.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 5, "infra": 1}, "body": "# Description:\r\n\r\nExtends our dev setup guide to include instructions for setting up HF, WandB, VSCode, etc.\r\n\r\nOther minor edits:\r\n- Update Discord links to newest version\r\n- Add more details to installation section, including other pip optional dependencies\r\n\r\n## Related issues\r\n\r\nTowards OPE-641\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2024-12-22T21:45:53Z"}
{"number": 931, "title": "Remove remaining references to `MessageContentItem`", "files": ["src/oumi/core/datasets/vision_language_dataset.py", "src/oumi/core/types/conversation.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-575\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-21T01:16:32Z"}
{"number": 929, "title": "90% speed up for the Oumi CLI", "files": ["docs/cli/commands.md", "pyproject.toml", "src/oumi/__init__.py", "src/oumi/__main__.py", "src/oumi/cli/cli_utils.py", "src/oumi/cli/distributed_run.py", "src/oumi/cli/env.py", "src/oumi/cli/evaluate.py", "src/oumi/cli/infer.py", "src/oumi/cli/judge.py", "src/oumi/cli/launch.py", "src/oumi/cli/main.py", "src/oumi/cli/train.py", "tests/integration/cli/test_judge_e2e.py", "tests/unit/cli/test_cli_distributed_run.py", "tests/unit/cli/test_cli_env.py", "tests/unit/cli/test_cli_evaluate.py", "tests/unit/cli/test_cli_infer.py", "tests/unit/cli/test_cli_judge.py", "tests/unit/cli/test_cli_launch.py", "tests/unit/cli/test_cli_main.py", "tests/unit/cli/test_cli_train.py", "tests/unit/cli/test_cli_utils.py"], "area": "cli", "area_votes": {"docs": 1, "infra": 1, "other": 1, "cli": 9}, "body": "# Description:\r\n\r\nCurrently the Oumi CLI has ~2s of overhead as it loads almost all dependent modules when entering the CLI.\r\nWe can trim this down by smartly waiting to import modules until we've verified that the user's command is valid.\r\n\r\n**NOTE for Reviewers:** Most of the changes are in `__init__.py`. The CLI was moved out of `core` to `/oumi/cli`, and moved all necessary imports into the CLI methods themselves.\r\n\r\n\r\nNew benchmarks:\r\n```\r\ntime oumi\r\noumi  0.278 total\r\n```\r\n\r\n```\r\ntime oumi env\r\noumi env   0.754 total\r\n```\r\n\r\nOld benchmarks:\r\n```\r\ntime oumi\r\noumi 2.558 total\r\n```\r\n\r\n```\r\ntime oumi env\r\noumi env 2.700 total\r\n```\r\n\r\nThe benefit:\r\n- oumi --help now takes ~.25s !\r\n- Users will get feedback much faster from any submitted command (meaning we can show our ascii art asap :) )\r\n\r\nThe cost:\r\n- The CLI has all import buried in functions\r\n- All top-level exports of `oumi` like `train`, `evaluate`, etc are now wrapped in methods in our __init__.py\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-594\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-21T00:33:53Z"}
{"number": 928, "title": "Update `docs/datasets/vl_sft.md` and `docs/datasets/local_datasets.md` reflecting the latest changes in Conversation format and VL-dataset setup", "files": ["docs/datasets/local_datasets.md", "docs/datasets/vl_sft.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Update `docs/datasets/vl_sft.md`  and `docs/datasets/local_datasets.md` to reflect the latest changes in `Conversation` format and VL-dataset setup\r\n-- Auto-formatted some JSON examples (+ fixed a format issue) \r\n-- Updated timestamps in examples from 2024 to 2025 for fresher look XD\r\n-- Served updated docs locally: LGTM\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-575\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-21T00:39:01Z"}
{"number": 927, "title": "Fix broken pip targets", "files": ["README.md", "configs/README.md", "notebooks/Oumi - Evaluation with MT Bench.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "scripts/polaris/jobs/vllm_worker.sh", "src/experimental/Oumi - Colab Setup Example.ipynb", "src/oumi/core/configs/params/peft_params.py"], "area": "docs", "area_votes": {"docs": 7, "infra": 1, "configs": 1}, "body": "# Description:\r\n\r\nThis also updates configs and documentation. I switched the Finetuning notebook from half to bfloat16 precision to be in line with other notebooks using bfloat16.\r\n\r\n## Related issues\r\n\r\nTowards OPE-635\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2024-12-23T20:01:43Z"}
{"number": 926, "title": "Update OOM FAQ with instructions to tune CUDA alloc", "files": ["docs/faq/oom.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n--Built oumi docs and verified the change looks OK (serving docs locally)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-671\r\n\r\n\r\n\r\n## Before submitting\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-20T18:51:12Z"}
{"number": 923, "title": "Mark `Message` as frozen", "files": ["src/oumi/core/types/conversation.py", "tests/unit/core/types/test_conversation.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Extra degree of immutability shouldn't hurt: We don't have any code that mutates `Message`-s, only `Conversation`-s.\r\n-- Caveat: Pydantic frozen ensures that fields can't be reset, which is fine for primitive fields (`role`, `id`), but for complex types like `content` (`list[]`) you can still mutate them inplace (e.g., append) . \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes  OPE-575\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-20T16:57:30Z"}
{"number": 922, "title": "Delete `configs/examples/llama3_1`", "files": ["configs/examples/llama3_1/README.md", "configs/examples/llama3_1/sft/405b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/405b_qlora/train.yaml", "configs/examples/llama3_1/sft/70b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_lora/train.yaml", "configs/examples/llama3_1/sft/70b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_qlora/train.yaml", "configs/examples/llama3_1/sft/8b_full/sky_job.yaml", "configs/examples/llama3_1/sft/8b_full/train.yaml", "configs/examples/llama3_1/sft/8b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_lora/train.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_train.yaml", "configs/examples/llama3_1/sft/8b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/train.yaml"], "area": "configs", "area_votes": {"docs": 1, "configs": 14}, "body": "# Description:\r\n\r\nThese are Panos' configs for reproducing Llama 3.1 trainings. It has very high overlap with `configs/recipes/llama3_1`, which has led to some confusion with users. I've moved it to our scratch repo, and will be deleting this one for cleanliness: https://github.com/oumi-ai/scratch/commit/d1046d4078fcb7a6ad132aee5fb057462e14f400.\r\n\r\n## Related issues\r\n\r\nTowards OPE-671\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2024-12-20T18:46:08Z"}
{"number": 921, "title": "Refactor `is_custom_model()` check into a function + Minor tests updates", "files": ["src/oumi/builders/models.py", "tests/integration/train/test_custom_models.py", "tests/unit/builders/test_models.py", "tests/unit/core/types/test_conversation.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Refactor `is_custom_model()` check into a separate function \r\n-- Minor tests updates: Add more calls to is_image_text_llm, is_custom_model, remove remainign references to \"compound\" type (now deleted), add `HuggingFaceTB/SmolVLM-Instruct` to is_vlm tests\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-697\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-20T01:08:48Z"}
{"number": 919, "title": "Exclude custom models from Vision-Language model checks.", "files": ["src/oumi/builders/models.py", "tests/unit/builders/test_models.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nVision models use a HF check that throws exceptions for custom models. For now we exclude custom models from this check.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n#917 \r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-19T23:58:42Z"}
{"number": 918, "title": "Catch exceptions for missing LoRA adapters", "files": ["src/oumi/core/configs/params/model_params.py", "tests/unit/core/configs/params/test_model_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nfind_adapter_config_file(...) will throw an OSError if it cannot find a valid HF repo for a given model. Catch this error and return None for the adapter in this case.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nRelated #917 \r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-19T23:44:04Z"}
{"number": 914, "title": "[docs] clean-up docs pages", "files": ["docs/_doclinks.config", "docs/advanced/custom_loss.md", "docs/advanced/customization.md", "docs/index.md", "docs/user_guides/infer/infer.md", "docs/user_guides/launch/launch.md", "docs/user_guides/launch/skypilot.md", "docs/user_guides/train/train.md", "docs/user_guides/train/training_config.md", "docs/user_guides/train/training_config.rst"], "area": "docs", "area_votes": {"docs": 10}, "body": "# Description:\r\n\r\n\r\n\r\n- Remove deprecated / duplicate docs pages\r\n\r\n\r\n\r\nCloses OPE-743\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-19T18:00:35Z"}
{"number": 913, "title": "[tiny] minor docs updates", "files": ["docs/about/acknowledgements.md", "docs/about/license.md", "docs/api/oumi.utils.rst", "docs/get_started/installation.md", "docs/get_started/quickstart.md", "docs/index.md"], "area": "docs", "area_votes": {"docs": 6}, "body": "# Description:\r\n\r\n\r\n\r\n- Tiny doc updates\r\n\r\n\r\n\r\nTowards OPE-739\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-19T17:36:48Z"}
{"number": 908, "title": "Set `max_pixels`/`min_pixels`  params for Qwen2 VL model", "files": ["configs/recipes/vision/phi3/sft/sft_train.yaml", "src/oumi/core/configs/internal/supported_models.py"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- The model processor generates variable number of image tokens. \r\n-- For large images it can generate too many causing OOM. For example it happens for `HuggingFaceH4/llava-instruct-mix-vsft` dataset but not for `merve/vqav2-small`\r\n-- Configure `max_pixels` using the values quoted on https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct , which solves OOM for  `HuggingFaceH4/llava-instruct-mix-vsft` \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-673\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-18T20:17:32Z"}
{"number": 906, "title": "Update chat template/conversation format - Phase 4", "files": ["src/oumi/core/types/conversation.py", "src/oumi/datasets/debug.py", "src/oumi/datasets/sft/aya.py", "src/oumi/datasets/sft/magpie.py", "tests/unit/core/types/test_conversation.py"], "area": "data", "area_votes": {"data": 4}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Make `Message.content` non-optional\r\n-- Misc minor changes to make pyright pass\r\n--  Update some stale `Message` docstrings to reflect the latest design\r\n-- Add  `Type.__str__` implementation for consistency with `Role.__str__` (human-readable value for display purposes)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-575\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-18T20:32:01Z"}
{"number": 905, "title": "Update configs to use `oumi distributed torchrun`", "files": [".vscode/launch.json", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/examples/llama3_1/sft/405b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_full/sky_job.yaml", "configs/examples/llama3_1/sft/8b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/sky_job.yaml", "configs/projects/aya/sft/sky_job.yaml", "configs/projects/chatqa/sky_job.yaml", "configs/projects/zephyr/sft/full_sky_job.yaml", "configs/projects/zephyr/sft/qlora_sky_job.yaml", "configs/recipes/gpt2/pretraining/sky_job.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/phi3/dpo/fsdp_sky_job.yaml", "configs/recipes/phi3/dpo/sky_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_sky_job.yaml", "docs/get_started/configuration.md", "docs/get_started/quickstart.md", "notebooks/Oumi - Running Jobs Remotely.ipynb", "scripts/benchmarks/benchmark_trainers.sh", "scripts/benchmarks/minimal_fsdp_training.py", "scripts/benchmarks/minimal_multimodal_training.py", "scripts/polaris/jobs/example_job.sh", "scripts/polaris/jobs/fineweb_pt_worker.sh", "scripts/polaris/jobs/llama_tune.sh", "src/oumi/core/cli/distributed_run.py"], "area": "configs", "area_votes": {"other": 2, "configs": 34, "docs": 3, "infra": 6}, "body": "# Description:\r\n\r\nThis builds on #888 by migrating all our configs to use the new `oumi distributed` wrapper.\r\n\r\nThese commands:\r\n```\r\n  torchrun \\\r\n      --nnodes=${OUMI_NUM_NODES} \\\r\n      --node-rank=${SKYPILOT_NODE_RANK} \\\r\n      --nproc-per-node=${SKYPILOT_NUM_GPUS_PER_NODE} \\\r\n      --master-addr=${OUMI_MASTER_ADDR} \\\r\n      --master-port=8007 \\\r\n      ...\r\n\r\ntorchrun \\\r\n    --nnodes=${OUMI_NUM_NODES} \\\r\n    --node-rank=${POLARIS_NODE_RANK} \\\r\n    --nproc-per-node=${OUMI_POLARIS_NUM_GPUS_PER_NODE} \\\r\n    --master-addr=${OUMI_MASTER_ADDR} \\\r\n    --master-port=8007 \\\r\n      ...\r\n```\r\nbecomes `  oumi distributed torchrun \\ ...`\r\n\r\nThese commands:\r\n```\r\n  accelerate launch \\\r\n      --num_machines ${OUMI_NUM_NODES} \\\r\n      --machine_rank ${SKYPILOT_NODE_RANK} \\\r\n      --num_processes ${OUMI_TOTAL_NUM_GPUS} \\\r\n      --main_process_ip ${OUMI_MASTER_ADDR} \\\r\n      --main_process_port 8007 \\\r\n      ...\r\n\r\n    accelerate launch \\\r\n        --num_machines ${OUMI_NUM_NODES} \\\r\n        --machine_rank ${POLARIS_NODE_RANK} \\\r\n        --num_processes ${OUMI_TOTAL_NUM_GPUS} \\\r\n        --main_process_ip ${OUMI_MASTER_ADDR} \\\r\n        --main_process_port 8007 \\\r\n      ...\r\n```\r\nbecomes `  oumi distributed accelerate launch \\ ...`\r\n\r\nThis PR also splits up the try-except blocks in `distributed_run.py` to separately catch errors with detecting the backend run info. Finally, I've also renamed `--nproc_per_node ` to `--nproc-per-node` for consistency, as we had both in our code. Both are [valid](https://github.com/pytorch/pytorch/blob/efe21ee59dfdd6642cc693e69e07aa9d8be13eb9/torch/distributed/run.py#L433).\r\n\r\nI've confirmed that I updated all relevant torchrun and accelerate commands. I've tested a representative subset of these configs to verify functionality.\r\n\r\n## Related issues\r\n\r\nFixes OPE-691\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2024-12-18T21:04:44Z"}
{"number": 904, "title": "Update Inference links in oumi README", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Remove broken (non-existing yet) links to vLLM  and SG-Lang configs for Llama 8B and 70B text models\r\n-- Add `Native` inference config link to Llama 3.2 Vision 11B\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-401\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-17T22:51:31Z"}
{"number": 902, "title": "Update configs/notebooks to use typer CLI instead of `oumi.train/eval/infer` modules", "files": ["configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/examples/llama3_1/sft/405b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_full/sky_job.yaml", "configs/examples/llama3_1/sft/8b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/sky_job.yaml", "configs/projects/aya/evaluation/sky_job.yaml", "configs/projects/aya/sft/sky_job.yaml", "configs/projects/chatqa/sky_job.yaml", "configs/projects/zephyr/sft/full_sky_job.yaml", "configs/projects/zephyr/sft/qlora_sky_job.yaml", "configs/recipes/gpt2/evaluation/async_sky_job.yaml", "configs/recipes/gpt2/pretraining/sky_job.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/phi3/dpo/fsdp_sky_job.yaml", "configs/recipes/phi3/dpo/sky_job.yaml", "configs/recipes/phi3/evaluation/sky_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_oumi_gcp_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_sky_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_trl_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_sky_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "docs/get_started/configuration.md", "notebooks/Oumi - Running Jobs Remotely.ipynb", "scripts/benchmarks/benchmark_trainers.sh", "scripts/polaris/jobs/example_job.sh", "scripts/polaris/jobs/fineweb_pt_worker.sh", "scripts/polaris/jobs/llama_tune.sh", "tests/unit/core/cli/test_cli_distributed_run.py", "tests/unit/core/cli/test_cli_main.py"], "area": "configs", "area_votes": {"configs": 51, "docs": 2, "infra": 4}, "body": "# Description:\r\n\r\nSee bug below for further details. In short:\r\n- We want to delete the `main()` function in train/eval/infer.py. It is deprecated since we deleted the `oumi-train` CLI command, and have the `oumi` CLI.\r\n- We still invoke the `main()` functions in most of our configs, ex. `oumi.train`. To prevent confusing users, we should consistently use the typer CLI everywhere.\r\n- `torchrun oumi train ...` does not work, as it seems to be interpeting `oumi` as `./oumi`, instead of searching for it in PATH. Thus we have to use `torchrun -m oumi train ...`.\r\n\r\nThis PR migrates all existing usages of `oumi.train/eval/infer` to the CLI, and this is the only change the PR makes. The only exception is `evalute_async`, which hasn't been integrated into the typer CLI. The main() functions will be deleted in a follow-up PR. Note that the way we specify flags with the typer CLI is different (`\"training.logging_steps=10\"` -> `--training.logging_steps 10`).\r\n\r\nI've manually double checked each change in this PR. I've also tested a subset of the configs to confirm that they're still working.\r\n\r\n## Related issues\r\n\r\nTowards OPE-731\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2024-12-17T01:20:50Z"}
{"number": 901, "title": "Enable `Message.content` with sub-items - phase 2", "files": ["src/oumi/core/types/conversation.py", "src/oumi/datasets/chat_templates/llama3-instruct.jinja", "src/oumi/datasets/chat_templates/llava.jinja", "src/oumi/datasets/chat_templates/phi3-instruct.jinja", "src/oumi/datasets/vision_language/coco_captions.py", "src/oumi/datasets/vision_language/flickr30k.py", "src/oumi/datasets/vision_language/llava_instruct_mix_vsft.py", "src/oumi/datasets/vision_language/vqav2_small.py", "src/oumi/infer.py", "tests/integration/infer/test_infer.py", "tests/integration/infer/test_native_text_inference_engine.py", "tests/unit/builders/test_processors.py", "tests/unit/core/datasets/test_vision_language_dataset.py", "tests/unit/core/types/test_conversation.py", "tests/unit/datasets/test_chat_templates.py", "tests/unit/datasets/test_vision_language_jsonlines_dataset.py", "tests/unit/inference/test_gcp_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py", "tests/unit/inference/test_sglang_inference_engine.py"], "area": "data", "area_votes": {"data": 8, "inference": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Disallow images as top-level messages: Images must go as sub-items in `Message.content`. Now `Message.content` can only be `TEXT` or a list of `MessageContentItems` for multimodal data.\r\n-- Delete the `Message.binary`  field (Reducing the number of options to keep complexity under control).\r\n-- Update multimodal chat templates to handle nested `message.content` + Reformatting for clarity.\r\n-- Update vision datasets to switch to emitting multimodal data in nested `content` format (1 complex message per each role's turn)\r\n-- Update unit tests to reflect these changes, and add new tests\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-575\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-16T21:33:39Z"}
{"number": 899, "title": "Evaluation Notebooks: MT Bench", "files": ["notebooks/Oumi - Evaluation with MT Bench.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\nEvaluation Notebooks: Add support for MT Bench\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-730\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-17T03:43:02Z"}
{"number": 898, "title": "Fix missing import precommits when you create a fresh install of Oumi", "files": ["src/oumi/inference/gcp_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nWhen running a vanilla install of oumi, I get a precommit error if I don't install GCP bits.\r\n\r\nMy initial install was:\r\n```\r\npip install -e \".[dev]\"\r\n```\r\n\r\nWe simply need to ignore these deps if they're not present as they're in the optional GCP package.\r\n\r\nComplete repro steps:\r\n\r\n```\r\nconda create -n oumi_clean_test python=3.11 -y\r\nconda activate oumi_clean_test\r\npip install -e \".[dev]\"\r\n(MAKE A CHANGE TO A FILE)\r\ngit add -A\r\ngit commit -m \"Some change.\"\r\n\r\n/Users/matthewpersons/Repos/oumi/src/oumi/inference/gcp_inference_engine.py\r\n  /Users/matthewpersons/Repos/oumi/src/oumi/inference/gcp_inference_engine.py:20:18 - error: Import \"google.auth\" could not be resolved (reportMissingImports)\r\n  /Users/matthewpersons/Repos/oumi/src/oumi/inference/gcp_inference_engine.py:21:18 - error: Import \"google.auth.transport.requests\" could not be resolved (reportMissingImports)\r\n  /Users/matthewpersons/Repos/oumi/src/oumi/inference/gcp_inference_engine.py:22:18 - error: Import \"google.oauth2\" could not be resolved (reportMissingImports)\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-12T22:52:12Z"}
{"number": 893, "title": "Update inference documentation", "files": [".gitignore", "docs/_doclinks.config", "docs/api/oumi.core.cli.rst", "docs/api/oumi.core.configs.internal.rst", "docs/api/oumi.core.configs.rst", "docs/api/oumi.datasets.evaluation.rst", "docs/api/oumi.datasets.rst", "docs/user_guides/infer/infer.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Multinode Inference on Polaris.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "src/oumi/evaluate.py"], "area": "docs", "area_votes": {"infra": 1, "docs": 13, "evaluation": 1}, "body": "# Description:\r\n\r\n- Update inference documentation\r\n- Fix and update the two inference notebooks and the tour notebook\r\n- Use SmolLM2 instead of GPT2 in our tour notebook\r\n- Rename config created by notebook so it'll be covered by our gitignore, and update gitignore to also include files with `tutorial` in the name.\r\n\r\nTested that the three notebooks mentioned above work E2E.\r\n\r\n## Related issues\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n", "merged_at": "2024-12-13T00:29:51Z"}
{"number": 891, "title": "Update VLM configs to use `oumi distributed torchrun`", "files": ["configs/recipes/vision/blip2_opt_2.7b/sft/2b_oumi_gcp_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_sky_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_trl_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_trl_gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "src/oumi/core/cli/distributed_run.py", "tests/unit/core/cli/test_cli_distributed_run.py"], "area": "configs", "area_votes": {"configs": 8, "other": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Update VLM configs to start using`oumi distributed torchrun`\r\n-- Fix handling of  `accelerate launch` sub-command.\r\n-- Minor changes in  `distributed_run.py`: Update command docstring, log rank, add docstring for ports.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-691\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-11T18:40:23Z"}
{"number": 890, "title": "Freeze torchdata>=0.8.0,<0.10.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nApparently, `torchdata.datapipes` is removed in `0.10.0` , which causes test breaks.\r\n\r\n```\r\n/home/runner/work/oumi/oumi/src/oumi/builders/oumi_data.py\r\n  /home/runner/work/oumi/oumi/src/oumi/builders/oumi_data.py:5:6 - error: Import \"torchdata.datapipes.iter\" could not be resolved (reportMissingImports)\r\n  /home/runner/work/oumi/oumi/src/oumi/builders/oumi_data.py:10:6 - error: Import \"torchdata.datapipes.map.util.converter\" could not be resolved (reportMissingImports)\r\n2 errors, 0 warnings, 0 informations\r\n\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-10T22:26:39Z"}
{"number": 887, "title": "Alpaca Eval 2.0 notebook", "files": ["notebooks/Oumi - Evaluation with AlpacaEval 2.0.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\nAlpaca Eval 2.0 notebook\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-424\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-10T20:43:56Z"}
{"number": 886, "title": "Move multimodal configs from `configs/examples/vision` to `configs/recipes/vision`", "files": [".vscode/launch.json", "README.md", "configs/recipes/vision/README.md", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_oumi_gcp_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_sft_train.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_sky_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/2b_trl_gcp_job.yaml", "configs/recipes/vision/blip2_opt_2.7b/sft/README.md", "configs/recipes/vision/llama3_2_vision/README.md", "configs/recipes/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/recipes/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_infer.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_infer_sglang.yaml", "configs/recipes/vision/llama3_2_vision/inference/11b_infer_vllm.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_gcp_job.yaml", "configs/recipes/vision/llama3_2_vision/sft/11b_sft_train.yaml", "configs/recipes/vision/llava_7b/README.md", "configs/recipes/vision/llava_7b/inference/infer.yaml", "configs/recipes/vision/llava_7b/sft/7b_oumi_gcp_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_sft_train.yaml", "configs/recipes/vision/llava_7b/sft/7b_sky_job.yaml", "configs/recipes/vision/llava_7b/sft/7b_trl_gcp_job.yaml", "configs/recipes/vision/phi3/sft/oumi_gcp_job.yaml", "configs/recipes/vision/phi3/sft/sft_train.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/gcp_job.yaml", "configs/recipes/vision/qwen2_vl_2b/sft/sft_train.yaml"], "area": "configs", "area_votes": {"other": 1, "docs": 5, "configs": 20}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Also update paths in README.md\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-716\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-07T17:12:46Z"}
{"number": 884, "title": "Improve error messaging for the remote inference engine.", "files": ["src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nPass the latest HTTP error as part of our exception messaging.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-719\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-06T22:58:42Z"}
{"number": 883, "title": "[tiny] fix remote inference bug for messages with image urls", "files": ["src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Fix issue converting image turns to openai format\r\n\r\n**Before**\r\nTurn `image_url` key is rendered as a `set`, which is not json serializable, and not what the openai api expects\r\n\r\n```json\r\n{'type': 'image_url', 'image_url': {'https://upload.wikimedia.org/wikipedia/commons/1/15/Cat_August_2010-4.jpg'}}]\r\n```\r\n\r\n**After**\r\nTurn content is rendered as a `dict`\r\n\r\n```json\r\n[{'type': 'image_url', 'image_url': {'url': 'https://upload.wikimedia.org/wikipedia/commons/1/15/Cat_August_2010-4.jpg'}}]\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-06T21:06:44Z"}
{"number": 882, "title": "Cap TRL library version to <0.12.2", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "In case anyone runs into this error:\r\n\r\nTypeError: Trainer.__init__() got an unexpected keyword argument 'processing_class'\r\n\r\nTRL's latest update breaks us:\r\nhttps://github.com/huggingface/trl/releases\r\n\r\nThe fix is to prevent TRL from upgrading:\r\n\"trl>=0.9.0,<0.12.2\",\r\n\r\n# Description:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-06T18:37:40Z"}
{"number": 881, "title": "Update documentation landing page", "files": ["README.md", "docs/conf.py", "docs/index.md", "docs/index.rst", "src/oumi/utils/torch_utils.py"], "area": "docs", "area_votes": {"docs": 4, "other": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Update documentation landing page to match the updated GitHub readme changes\r\n- Convert RST sphinx index to markdown for easier transfer from the GitHub readme\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-708\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-06T16:48:03Z"}
{"number": 880, "title": "Adding support for Alpaca Eval", "files": ["src/oumi/core/datasets/base_sft_dataset.py", "src/oumi/datasets/evaluation/__init__.py", "src/oumi/datasets/evaluation/alpaca.py", "src/oumi/datasets/evaluation/utils.py", "src/oumi/datasets/sft/alpaca.py"], "area": "data", "area_votes": {"data": 4}, "body": "# Description:\r\nAdding support for Alpaca Eval \r\n\r\n## Related issues\r\nFixes # N/A\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n## Reviewers\r\nWill, Oussama\r\n", "merged_at": "2024-12-06T22:26:09Z"}
{"number": 879, "title": "Update the list of supported VLM-s in README", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Move Llama3.2 Vision to the top\r\n-- Replace BLIP with Phi3 (there is some regression affecting BLIP, so removed it temporarily)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-06T00:52:54Z"}
{"number": 878, "title": "Update recipes list in readme", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n- Update readme recipes list\r\n- Many of these resources need to be updated / refresh, and we're missing a few others. For now, this PR just updates the existing list of examples / recipes / notebooks\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nTowards OPE-708\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-06T00:13:56Z"}
{"number": 877, "title": "Refresh documentation pages", "files": [".markdownlint.json", "CONTRIBUTING.md", "docs/.gitignore", "docs/_doclinks.config", "docs/api/oumi.core.cli.rst", "docs/api/oumi.core.configs.params.rst", "docs/api/oumi.models.layers.rst", "docs/api/oumi.models.rst", "docs/development/contributing.md", "docs/development/dev_setup.md", "docs/get_started/installation.md", "docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"other": 1, "docs": 11}, "body": "# Description:\r\n\r\n\r\n\r\n- Misc updates / fixes in our documentation\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-708\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-05T22:59:19Z"}
{"number": 875, "title": "Update readme sections (part 2)", "files": ["README.md", "docs/about/citations.md", "docs/faq/troubleshooting.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description:\r\n\r\n\r\n\r\n- Update remaining readme sections, and remaining feedback from part 1 PR.\r\n- Still WIP: table with supported models & notebook (in a follow-up)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-708\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-05T21:12:50Z"}
{"number": 874, "title": "Update main readme (part 1)", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Update & refresh main readme\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-708\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-05T19:12:03Z"}
{"number": 870, "title": "Tweak writer batch size computation in `BaseMapDataset`", "files": ["src/oumi/core/datasets/base_map_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- It's a follow-up to https://github.com/oumi-ai/oumi/pull/868 with some improvements.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-569\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-03T23:28:34Z"}
{"number": 867, "title": "Rewrite CONTRIBUTING.md with new content, fixes and nits", "files": ["CONTRIBUTING.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-05T18:04:14Z"}
{"number": 866, "title": "Update dev_setup.md with minor clarification for use of SSH in GitHub", "files": ["docs/development/dev_setup.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-12-05T18:02:56Z"}
{"number": 865, "title": "Update base config to_yaml signature", "files": ["src/oumi/core/configs/base_config.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n- Small change to allow using `Path` and `StringIO` as inputs to the `BaseConfig.to_yaml`. This is just to avoid linting errors as the underlying function supports it\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-26T22:59:24Z"}
{"number": 862, "title": "Update inference engine docs.", "files": ["docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_engines.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nSmall changes to give a better description of our inference engines.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-25T23:17:15Z"}
{"number": 861, "title": "Initial port of zigzag attention", "files": ["src/oumi/core/trainers/oumi_trainer.py", "src/oumi/models/layers/ring_attention.py", "src/oumi/models/layers/zigzag.py", "src/oumi/models/layers/zigzag_utils.py"], "area": "other", "area_votes": {"training": 1, "other": 3}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Based on `oelachqar/dist_attention`,  which is based on  https://github.com/jzhang38/EasyContext, https://github.com/zhuzilin/ring-flash-attention/\r\n-- Includes minor updates to resolve pre-submit and `PyRight` errors.\r\n-- The code is not enabled yet.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-333, OPE-334\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-26T17:00:54Z"}
{"number": 859, "title": "Change config dir structure  for LLAVA and blip2 models", "files": ["configs/examples/vision/blip2_opt_2.7b/sft/2b_oumi_gcp_job.yaml", "configs/examples/vision/blip2_opt_2.7b/sft/2b_sft_train.yaml", "configs/examples/vision/blip2_opt_2.7b/sft/2b_sky_job.yaml", "configs/examples/vision/blip2_opt_2.7b/sft/2b_trl_gcp_job.yaml", "configs/examples/vision/blip2_opt_2.7b/sft/README.md", "configs/examples/vision/llava_7b/inference/infer.yaml", "configs/examples/vision/llava_7b/sft/7b_oumi_gcp_job.yaml", "configs/examples/vision/llava_7b/sft/7b_sft_train.yaml", "configs/examples/vision/llava_7b/sft/7b_sky_job.yaml", "configs/examples/vision/llava_7b/sft/7b_trl_gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 9, "docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- For consistency with other models, and to make it easier to add larger model sizes \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-688\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-21T16:58:19Z"}
{"number": 858, "title": "Enable LM-harness eval for multimodal, and add eval config for Llama 3.2 Vision 11B", "files": [".vscode/launch.json", "configs/examples/vision/llama3_2_vision/evaluation/11b_eval.yaml", "configs/examples/vision/llama3_2_vision/evaluation/11b_gcp_job.yaml", "configs/examples/vision/llama3_2_vision/inference/11b_infer.yaml", "configs/examples/vision/llama3_2_vision/inference/11b_infer_sglang.yaml", "configs/examples/vision/llama3_2_vision/inference/11b_infer_vllm.yaml", "configs/examples/vision/llama3_2_vision/sft/11b_gcp_job.yaml", "configs/examples/vision/llama3_2_vision/sft/11b_sft_train.yaml", "pyproject.toml", "src/oumi/evaluate.py"], "area": "configs", "area_votes": {"other": 1, "configs": 7, "infra": 1, "evaluation": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Update `src/oumi/evaluate.py` to invoke `lm-eval` with multimodal specific params\r\n-- Eval for Llama 3.2 11B model, using MMMU benchmark\r\n-- GCP Launcher configs for 1 node with 4 A100 GPUs.\r\n-- Re-organized directory structure for Llama 3.2 11B model for train/eval/infer configs\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-649\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-21T17:06:12Z"}
{"number": 856, "title": "Update  Llama 405B qlora config to use `adamw_torch_fused` optimizer", "files": ["configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Replace `sgd`  with `adamw_torch_fused`, as training loss doesn't seem to decrease with `sgd`\r\n-- `sgd` was introduced in https://github.com/oumi-ai/oumi/pull/838 . Reverting the change.\r\n-- Tested on GCP: 1 node with 8 `A100-80GB`\r\n-- Also, reverting max sequence length from `1028` back to `2048`.  It works OK e.g.,  https://wandb.ai/lema-train-test/oumi-train/runs/7tzsaddn\r\n\r\nwith  `adamw_torch_fused`. Loss decreases:\r\nhttps://wandb.ai/lema-train-test/oumi-train/runs/5w46zd2l\r\nhttps://wandb.ai/lema-train-test/oumi-train/runs/wwd2e3kg\r\n\r\nBefore (`sgd`):\r\nhttps://wandb.ai/lema-train-test/oumi-train/runs/pwz7gasu\r\nhttps://wandb.ai/lema-train-test/oumi-train/runs/1pne1jch\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-671\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-19T18:33:10Z"}
{"number": 855, "title": "Fix our config parsing test", "files": ["configs/recipes/phi3/evaluation/eval.yaml", "tests/unit/core/configs/test_parse_configs.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Description:\r\n\r\n\r\n\r\nAfter our recent restructuring of our test folder, our config parsing test was silently running on 0 configs.\r\nUpdated the test to fail in that case. Updated the test to properly point to the repo root again.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-18T19:01:37Z"}
{"number": 851, "title": "Fixes for quickstart and launch commands.", "files": ["configs/recipes/smollm/evaluation/135m_eval_quickstart.yaml", "configs/recipes/smollm/inference/135m_infer.yaml", "configs/recipes/smollm/launch/135m_gcp_evaluate.yaml", "configs/recipes/smollm/launch/135m_gcp_evaluate_quickstart.yaml", "configs/recipes/smollm/launch/135m_gcp_train.yaml", "configs/recipes/smollm/launch/135m_gcp_train_quickstart.yaml", "configs/recipes/smollm/sft/135m/train_quickstart.yaml", "docs/get_started/quickstart.md", "src/oumi/core/cli/launch.py"], "area": "configs", "area_votes": {"configs": 7, "docs": 1, "other": 1}, "body": "# Description:\r\n\r\n\r\n\r\nThis PR comes in two parts:\r\n\r\n1) Update the quickstart to take less than 10 minutes e2e. Training and eval are not really useful for this demo, but a user can see the jobs start, run, and generate results.\r\n\r\nWe also now show configs inline on the quickstart page\r\n![image](https://github.com/user-attachments/assets/45a09319-0d48-4434-a25d-595665dcebe9)\r\n\r\n![image](https://github.com/user-attachments/assets/26574cfe-e00c-423d-87be-3defa52a17f0)\r\n\r\n\r\n2) Update several CLI commands to clear the previous line when printing. This cleans up miscellaneous issues when polling jobs.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-15T17:15:07Z"}
{"number": 849, "title": "Update RemoteInferenceEngine to support guided decoding", "files": ["src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Update RemoteInferenceEngine to support guided decoding\r\n- This leverages the OpenAI structured output api `https://platform.openai.com/docs/guides/structured-outputs`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nCloses OPE-685\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-14T19:34:44Z"}
{"number": 848, "title": "[tiny] fix openai remote engine auth bug", "files": ["src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Bug generating the auth headers for remote inference engine when using `api_key_env_varname`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-14T17:28:06Z"}
{"number": 846, "title": "Add `@override` annotations to misc inference engines methods where missing", "files": ["src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/gcp_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py"], "area": "inference", "area_votes": {"inference": 6}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n\r\n-- also add `pyright: ignore[reportMissingImports]` to `vlm.GuidedDecodingParams`\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-370, OPE-627\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-13T18:24:29Z"}
{"number": 845, "title": "Add support for remote vllm inference engine", "files": ["src/oumi/inference/__init__.py", "src/oumi/inference/remote_vllm_inference_engine.py", "tests/unit/core/configs/test_guided_params.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- This PR adds a new `RemoteVLLMInferenceEngine` class that enables running inference against remote VLLM servers. \r\n- Compared to `RemoteInferenceEngine`, it supports the additional parameters that can be used by `vllm`, e.g. structured decoding\r\n\r\n#### Usage\r\n```python\r\nfrom oumi.inference import RemoteVLLMInferenceEngine\r\nfrom oumi.core.configs import ModelParams, GenerationParams\r\n\r\nengine = RemoteVLLMInferenceEngine(\r\n    ModelParams(\r\n        model_name=\"your-model\",\r\n        remote_params={\r\n            \"api_base\": \"http://your-vllm-server\",\r\n        }\r\n    )\r\n)\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-13T18:00:46Z"}
{"number": 843, "title": "[tiny] fix typo in docs/quickstart", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- fix typo in docs/quickstart\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-13T01:30:44Z"}
{"number": 841, "title": "[tiny] pass dataset_path to constructor", "files": ["src/oumi/builders/data.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Fix a tiny bug constructing datasets: the `dataset_path` argument was not passed to the dataset constructor\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-13T00:05:53Z"}
{"number": 840, "title": "Documentation fix for training.", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nFix a typo in local dataset loading.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-12T23:19:40Z"}
{"number": 839, "title": "Add a debug command for dumping environment information.", "files": ["src/oumi/core/cli/env.py", "src/oumi/core/cli/main.py", "src/oumi/utils/torch_utils.py", "tests/unit/core/cli/test_cli_env.py", "tests/unit/core/cli/test_cli_main.py"], "area": "other", "area_votes": {"other": 3}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nThis PR adds the CLI command `oumi env` which dumps information about the user's environment.\r\n\r\nSample output (from my mac):\r\n```\r\n----------Oumi environment information:----------\r\n\r\nOumi version: 0.1a1.dev252+g84436cbc.d20241112\r\nPython version: 3.11.9\r\nPlatform: macOS-14.7.1-arm64-arm-64bit\r\n\r\nInstalled dependencies:\r\nPACKAGE             VERSION\r\naccelerate          1.0.0\r\naiohttp             3.10.9\r\nbitsandbytes        0.42.0\r\ndatasets            3.0.0\r\ndiffusers           \r\neinops              0.8.0\r\njsonlines           4.0.0\r\nliger-kernel        \r\nlm-eval             0.4.5\r\nnumpy               1.26.4\r\nnvidia-ml-py        12.560.30\r\nomegaconf           2.3.0\r\nopen_clip_torch     \r\npandas              2.2.3\r\npeft                0.12.0\r\npexpect             4.9.0\r\npillow              10.4.0\r\npydantic            2.9.2\r\nresponses           0.25.3\r\nskypilot            0.6.1\r\ntensorboard         2.17.1\r\ntimm                \r\ntorch               2.4.0\r\ntorchdata           0.8.0\r\ntqdm                4.66.5\r\ntransformers        4.45.2\r\ntrl                 0.11.0\r\ntyper               0.12.5\r\nvllm                \r\nwandb               0.17.7\r\n\r\nPyTorch information:\r\nCUDA available: False\r\n```\r\n\r\nSample on GCP:\r\n```\r\n----------Oumi environment information:----------\r\n\r\nOumi version: 0.1a1.dev255+gb8d24207.d20241112\r\nPython version: 3.10.13\r\nPlatform: Linux-5.10.0-26-cloud-amd64-x86_64-with-glibc2.31\r\n\r\nInstalled dependencies:\r\nPACKAGE             VERSION\r\naccelerate          1.1.1\r\naiohttp             3.10.9\r\nbitsandbytes        \r\ndatasets            3.1.0\r\ndiffusers           \r\neinops              0.8.0\r\njsonlines           4.0.0\r\nliger-kernel        0.4.0\r\nlm-eval             0.4.5\r\nnumpy               1.25.2\r\nnvidia-ml-py        11.495.46\r\nomegaconf           2.3.0\r\nopen_clip_torch     \r\npandas              2.0.3\r\npeft                0.13.2\r\npexpect             4.8.0\r\npillow              10.4.0\r\npydantic            2.9.2\r\nresponses           0.25.3\r\nskypilot            0.7.0\r\ntensorboard         2.18.0\r\ntimm                \r\ntorch               2.4.0\r\ntorchdata           0.9.0\r\ntqdm                4.67.0\r\ntransformers        4.45.2\r\ntrl                 0.11.4\r\ntyper               0.9.0\r\nvllm                0.6.3.post1\r\nwandb               0.18.6\r\n\r\nPyTorch information:\r\nCUDA available: True\r\nCUDA version: 12.1\r\nNumber of GPUs: 1\r\nGPU type: NVIDIA A100-SXM4-40GB\r\nCUDNN version: 90.1.0\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-615\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-", "merged_at": "2024-11-13T18:27:53Z"}
{"number": 838, "title": "Additional improvements to 405B", "files": ["configs/examples/llama3_1/sft/405b_qlora/train.yaml", "configs/examples/vision/llama3_2_vision/gcp_job.yaml", "configs/recipes/llama3_1/README.md", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "pyproject.toml", "scripts/polaris/jobs/llama_tune.sh"], "area": "configs", "area_votes": {"configs": 24, "docs": 1, "infra": 2}, "body": "# Description:\r\n\r\n- For Polaris training, download the model weights from Eagle to worker disk, reducing the total time needed to load the model from 3 hours to ~20 minutes.\r\n- Update 405B training params to reduce memory consumption/increase nodes\r\n- Move `bnb_4bit_quant_storage` to base config\r\n- Refactor `llama_tune.sh` since we only need to swap out the config file\r\n- Remove model copying from `llama_tune.sh` for 8B/70B. These aren't really necessary since the models are relatively small, and the model only has to be downloaded once\r\n- Exclude downloading the `original/*` subdir for HF Llama models, since they're redundant copies of the weights. Excluding this reduces the amount of downloaded files by 2-3x and speeds up download time.\r\n\r\n## Related issues\r\n\r\nTowards OPE-671\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2024-11-13T08:56:35Z"}
{"number": 837, "title": "Request user feedback after training.", "files": ["src/oumi/train.py"], "area": "training", "area_votes": {"training": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nAfter a training run, we log a call to action for https://oumi.ai/feedback\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-680\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-12T19:21:07Z"}
{"number": 836, "title": "Add options for specifying log levels in the CLI.", "files": ["src/oumi/core/cli/cli_utils.py", "src/oumi/core/cli/evaluate.py", "src/oumi/core/cli/infer.py", "src/oumi/core/cli/judge.py", "src/oumi/core/cli/launch.py", "src/oumi/core/cli/train.py", "tests/conftest.py", "tests/unit/core/cli/test_cli_evaluate.py", "tests/unit/core/cli/test_cli_infer.py", "tests/unit/core/cli/test_cli_judge.py", "tests/unit/core/cli/test_cli_launch.py", "tests/unit/core/cli/test_cli_train.py", "tests/unit/core/cli/test_cli_utils.py"], "area": "other", "area_votes": {"other": 6}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nThis PR adds the `--log-level` / `-log` argument to all CLI commands.\r\n\r\nUnfortunately there is no good solution in Typer currently for sharing options across all commands: https://github.com/fastapi/typer/issues/153\r\n\r\nI took this approach as it feels the most natural for the user.\r\n\r\nSample help command: \r\n![image](https://github.com/user-attachments/assets/4f0989dd-8d22-429d-ab19-cb3cae8c03d8)\r\n\r\nSample invalid value:\r\n![image](https://github.com/user-attachments/assets/ad0ce387-d7ea-4015-a983-25ca0fedd1aa)\r\n\r\nSample valid value:\r\n![image](https://github.com/user-attachments/assets/d5a54002-ee59-436b-a4d6-3e795d996f1c)\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-669\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-12T17:22:10Z"}
{"number": 835, "title": "Switch from 8-bit AdamW to AdamW for FSDP Lora/QLora", "files": ["configs/examples/llama3_1/sft/405b_qlora/train.yaml", "configs/examples/llama3_1/sft/70b_qlora/train.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "pyproject.toml", "src/oumi/utils/torch_utils.py", "tests/unit/utils/test_torch_utils.py"], "area": "configs", "area_votes": {"configs": 6, "infra": 1, "other": 1}, "body": "# Description:\r\n\r\nWe get an error with saving the model with 8-bit AdamW when doing FSDP Lora/QLora training:\r\n\r\n```\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/transformers/trainer.py\", line 2918, in _maybe_log_save_evaluate\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     self._save_checkpoint(model, trial, metrics=metrics)\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/transformers/trainer.py\", line 3012, in _save_checkpoint\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     self._save_optimizer_and_scheduler(output_dir)\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/transformers/trainer.py\", line 3150, in _save_optimizer_and_scheduler\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     save_fsdp_optimizer(\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/accelerate/utils/fsdp_utils.py\", line 186, in save_fsdp_optimizer\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     optim_state = FSDP.optim_state_dict(model, optimizer)\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py\", line 1873, in optim_state_dict\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     return FullyShardedDataParallel._optim_state_dict_impl(\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py\", line 1284, in _optim_state_dict_impl\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     return _optim_state_dict(\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py\", line 116, in decorate_context\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     return func(*args, **kwargs)\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/torch/distributed/fsdp/_optim_utils.py\", line 1975, in _optim_state_dict\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     fsdp_osd_state = convert_fn(\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/torch/distributed/fsdp/_optim_utils.py\", line 1798, in _convert_state_with_orig_params\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     _gather_all_orig_param_state(\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/torch/distributed/fsdp/_optim_utils.py\", line 1692, in _gather_all_orig_param_state\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     output_states = _allgather_orig_param_states(\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/torch/distributed/fsdp/_optim_utils.py\", line 1522, in _allgather_orig_param_states\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     dtype, state_buffers = _convert_all_state_info(\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:   File \"/opt/conda/lib/python3.10/site-packages/torch/distributed/fsdp/_optim_utils.py\", line 1381, in _convert_all_state_info\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]:     assert dtype == info.dtype\r\n(llama405b-qlora-fsdp, pid=17187) [rank0]: AssertionError\r\n```\r\n\r\nThis PR also adds `responses` to our pyproject since it's required by `tests/unit/utils/test_image_utils.py`.\r\n\r\n## Related issues\r\n\r\nTowards OPE-671\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2024-11-11T23:00:11Z"}
{"number": 833, "title": "Add Llama405B Qlora/Lora/FFT configs", "files": ["configs/examples/misc/sky_ssh_job.yaml", "configs/recipes/llama3_1/sft/405b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_full/train.yaml", "configs/recipes/llama3_1/sft/405b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_lora/train.yaml", "configs/recipes/llama3_1/sft/405b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/405b_qlora/train.yaml", "configs/recipes/llama3_1/sft/70b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "scripts/polaris/jobs/llama_tune.sh", "src/oumi/launcher/clusters/polaris_cluster.py", "tests/unit/launcher/clusters/test_polaris_cluster.py"], "area": "configs", "area_votes": {"configs": 21, "infra": 1, "launcher": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Add initial Llama405B Qlora/Lora/FFT configs. Lora/FFT are still untested due to difficulty scheduling on Polaris. In Polaris, we update an env variable to point to Eagle as the HF cache dir, since the model is too big to fit in the user's `/home` dir.\r\n- Update params for Llama 8B/70B\r\n- Rename SFT to FFT in more places\r\n- Update Polaris cluster to use YYYYMMDD instead of DDMMYYYY timestamp\r\n\r\nTested 405B Qlora training on GCP and Polaris. Everything runs slow given model size\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-671\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-09T00:34:03Z"}
{"number": 830, "title": "Update our documentation to include modules for directories that aren't packages.", "files": ["README.md", "docs/_templates/apidoc/package.rst.jinja", "docs/api/oumi.core.cli.rst", "docs/api/oumi.core.collators.rst", "docs/api/oumi.core.configs.params.rst", "docs/api/oumi.core.processors.rst", "docs/api/oumi.core.registry.rst", "docs/api/oumi.core.rst", "docs/api/oumi.launcher.clients.rst", "docs/api/oumi.launcher.clusters.rst", "docs/api/oumi.models.experimental.cambrian.model.multimodal_projector.rst", "docs/api/oumi.performance.rst", "docs/api/oumi.utils.rst", "notebooks/Oumi - A Tour.ipynb", "pyproject.toml", "src/oumi/core/registry/__init__.py", "src/oumi/core/registry/registry.py"], "area": "docs", "area_votes": {"docs": 14, "infra": 1, "other": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nThis PR comes in several parts.\r\n\r\n1. Updated the jinja template for apidoc to generate documentation for directories that are namespaces but not packages if they do not contain any subpackages. This is suitable for leaf directories in our code base.\r\n\r\n2. Rerun `make docs-rebuild`. We should consider not tracking any generated docs under `/docs`. I'll consider addressing this later.\r\n\r\n3. Fix several broken links in docs / notebooks.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-08T18:57:00Z"}
{"number": 828, "title": "Misc documentation updates for installation, quickstart, and tutorials.", "files": ["docs/get_started/installation.md", "docs/get_started/quickstart.md", "docs/get_started/tutorials.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nThis is an initial first pass on our installation / quickstart documentation. I've run most commands and fixed those with typos.\r\n\r\nI also updated our suggested install path to `local` as we don't have pip integration with our latest bits yet.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-07T22:05:24Z"}
{"number": 820, "title": "Update quickstart to use the new CLI arg format.", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nUpdate quickstart to use the new `-arg value` format for overriding values.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-06T23:03:58Z"}
{"number": 819, "title": "Correct names of few env vars in `llama_tune.sh`", "files": ["scripts/polaris/jobs/llama_tune.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n--Update  names of few env vars in llama_tune.sh\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-362\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-06T20:10:59Z"}
{"number": 818, "title": "Fix a small typo in quickstart", "files": ["docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nUpdate an old config reference.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-06T19:18:54Z"}
{"number": 817, "title": "Update pretest.yaml", "files": [".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nOnly run unit tests during presubmit runs.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-06T18:46:06Z"}
{"number": 814, "title": "Update installation.md", "files": ["docs/get_started/installation.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nAdd instructions for installing via SSH.\r\n\r\nNew UI: \r\n\"image\"\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [X] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-06T18:48:08Z"}
{"number": 813, "title": "Add extra logging in Polaris client", "files": ["src/oumi/launcher/clients/polaris_client.py", "tests/unit/launcher/clients/test_polaris_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description:\r\n\r\n-- Add extra error logging in Polaris client.\r\n-- Include time durations.\r\n--  Should not affect happy path.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-670\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-06T19:08:48Z"}
{"number": 812, "title": "Disable uv in polaris launcher", "files": ["src/oumi/launcher/clusters/polaris_cluster.py", "tests/unit/launcher/clusters/test_polaris_cluster.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description:\r\n\r\n-- Disable uv in polaris launcher temporarily. `uv pip install` doesn't seem effective leading to `Error while finding module specification for 'oumi.train' (ModuleNotFoundError: No module named 'oumi')`\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-670\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-06T18:33:44Z"}
{"number": 811, "title": "Migrate configs to use Oumi instead of Accelerate configs", "files": ["README.md", "configs/examples/chatqa/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/README.md", "configs/examples/fineweb_ablation_pretraining/ddp/accelerate.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/accelerate_ddp_sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/accelerate.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/trl_gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/trl_train.yaml", "configs/examples/llama3_1/sft/405b_qlora/accelerate.yaml", "configs/examples/llama3_1/sft/405b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/405b_qlora/train.yaml", "configs/examples/llama3_1/sft/70b_lora/accelerate.yaml", "configs/examples/llama3_1/sft/70b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_lora/train.yaml", "configs/examples/llama3_1/sft/70b_qlora/accelerate.yaml", "configs/examples/llama3_1/sft/70b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_qlora/train.yaml", "configs/examples/llama3_1/sft/8b_full/accelerate.yaml", "configs/examples/llama3_1/sft/8b_full/sky_job.yaml", "configs/examples/llama3_1/sft/8b_full/train.yaml", "configs/examples/llama3_1/sft/8b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_lora/train.yaml", "configs/examples/llama3_1/sft/8b_qlora/accelerate.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_train.yaml", "configs/examples/llama3_1/sft/8b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/train.yaml", "configs/examples/vision/blip2_opt_2.7b/oumi_gcp_job.yaml", "configs/examples/vision/blip2_opt_2.7b/sky_job.yaml", "configs/examples/vision/blip2_opt_2.7b/trl_gcp_job.yaml", "configs/examples/vision/llama3_2_vision/accelerate.yaml", "configs/examples/vision/llama3_2_vision/gcp_job.yaml", "configs/examples/vision/llama3_2_vision/sft_train.yaml", "configs/examples/vision/llava_7b/oumi_gcp_job.yaml", "configs/examples/vision/llava_7b/sky_job.yaml", "configs/examples/vision/llava_7b/trl_gcp_job.yaml", "configs/projects/aya/sft/sky_job.yaml", "configs/projects/zephyr/sft/full_sky_job.yaml", "configs/projects/zephyr/sft/qlora_sky_job.yaml", "configs/recipes/gpt2/pretraining/accelerate.yaml", "configs/recipes/gpt2/pretraining/cpu_accelerate.yaml", "configs/recipes/gpt2/pretraining/fsdp_sky_job.yaml", "configs/recipes/gpt2/pretraining/mac_train.yaml", "configs/recipes/gpt2/pretraining/sky_job.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/pretraining/8b/accelerate.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/hsdp_sky_job.yaml", "configs/recipes/llama3_1/pretraining/8b/polaris_job.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "configs/recipes/llama3_1/sft/70b_full/accelerate.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/accelerate.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/accelerate.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_full/sky_job.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/accelerate.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_train.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/sky_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/accelerate.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/accelerate.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_full/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/accelerate.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_lora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/accelerate.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/phi3/dpo/accelerate.yaml", "configs/recipes/phi3/dpo/fsdp_nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/fsdp_sky_job.yaml", "configs/recipes/phi3/dpo/sky_job.yaml", "configs/recipes/phi3/sft/mac_lora_train.yaml", "docs/user_guides/train/trainers.md", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "scripts/polaris/jobs/build_apptainer_from_docker.sh", "scripts/polaris/jobs/download_model_from_hf.sh", "scripts/polaris/jobs/fineweb_pt_job.sh", "scripts/polaris/jobs/fineweb_pt_worker.sh", "scripts/polaris/jobs/llama_tune.sh", "scripts/polaris/jobs/vllm_job.sh", "scripts/polaris/launcher.sh", "src/experimental/pretokenize/sky.yaml", "tests/unit/utils/test_str_utils.py"], "area": "configs", "area_votes": {"docs": 5, "configs": 107, "infra": 7, "other": 1}, "body": "# Description:\r\n\r\n\r\n\r\nFollowing #803, we now migrate our configs to use Oumi `FSDPParams` instead of a separate Accelerate config file, unifying everything under `TrainingConfig`. There should be no functional change from this PR.\r\n\r\nThis PR also does misc. cleanup in our configs.\r\n\r\nI tested the following subset of configs to ensure the total training time is unchanged:\r\n- `configs/examples/fineweb_ablation_pretraining/fsdp/gcp_job.yaml`\r\n- `configs/examples/vision/llama3_2_vision/gcp_job.yaml`\r\n- `configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml`\r\n- `configs/recipes/phi3/dpo/fsdp_sky_job.yaml`\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes OPE-362\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-06T18:37:41Z"}
{"number": 810, "title": "Fixed broken skyssh and skycode commands", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description:\r\n\r\n\r\nCommands were pointing to a file which no longer exists.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-05T22:14:46Z"}
{"number": 809, "title": "[docs] Start adding doctests", "files": ["docs/datasets/local_datasets.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Description:\r\n\r\n\r\n\r\n- Start adding doctests, to prevent documentation from getting stale\r\n- For now adding a single page where all the tests pass\r\n\r\nUsage:\r\n```bash\r\npython -m doctest docs/datasets/local_datasets.md\r\n```\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\nTowards OPE-604\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-05T20:55:28Z"}
{"number": 807, "title": "Create Polaris and GCP launcher configs for Llama 8B pretraining", "files": ["configs/recipes/llama3_1/pretraining/8b/accelerate.yaml", "configs/recipes/llama3_1/pretraining/8b/gcp_job.yaml", "configs/recipes/llama3_1/pretraining/8b/hsdp_sky_job.yaml", "configs/recipes/llama3_1/pretraining/8b/polaris_job.yaml", "configs/recipes/llama3_1/pretraining/8b/train.yaml", "scripts/polaris/jobs/llama_tune.sh"], "area": "configs", "area_votes": {"configs": 5, "infra": 1}, "body": "# Description:\r\n \r\n-- Create Polaris and GCP launcher configs for Llama 8B pre-training\r\n-- Moved configs from `pretraining` to `pretraining/8b/` sub-dir to allow adding 70B model later\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-570, OPE-413\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-05T19:08:02Z"}
{"number": 805, "title": "Fix pyproject.toml merge error", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nFix an issue with the pyproject.toml, where the dependency group for Cambrian was in the wrong section\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-05T02:46:04Z"}
{"number": 804, "title": "Update llama_tune script to support 8B pretraining", "files": ["scripts/polaris/jobs/llama_tune.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\n-- Update lora_tune script to support 8B pretraining\r\n-- This change depends on https://github.com/oumi-ai/oumi/pull/778\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nTowards OPE-570, OPE-413\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-05T02:50:11Z"}
{"number": 801, "title": "Update polaris user regex to handle usernames with dashes", "files": ["src/oumi/launcher/clients/polaris_client.py", "tests/unit/launcher/clients/test_polaris_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "# Description:\r\n\r\n\r\nCurrent polaris regex fails on usernames with dashes in them (it cuts them off early). This PR modifies the regex to add an exclusion of this character to ensure we consider the port number separately.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-01T21:19:36Z"}
{"number": 799, "title": "Small fix for building docs", "files": ["src/oumi/core/cli/main.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Description:\r\n\r\n\r\n\r\n\r\nLook for the sphinx module instead of sphinx-build.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-01T17:03:23Z"}
{"number": 798, "title": "Update pull_request_template.md", "files": [".github/pull_request_template.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Describe your change\r\n\r\n\r\n\r\nMinor PR template update with more arrows.\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-31T21:25:05Z"}
{"number": 797, "title": "Update base SFT dataset to add support for training on assistant turns only", "files": ["src/oumi/core/datasets/base_sft_dataset.py", "src/oumi/core/tokenizers/utils.py", "src/oumi/datasets/chat_templates/default_gen.jinja", "tests/conftest.py", "tests/unit/core/collators/test_text_completions_collator_with_padding.py", "tests/unit/core/datasets/test_base_sft_dataset.py"], "area": "data", "area_votes": {"data": 3}, "body": "# Describe your change\r\n\r\n- Update base SFT dataset to add support for training on assistant turns only\r\n- No behavior change If `assistant_only=False`\r\n- We support two methods to mask non-assistant turns:\r\n  - [recommended] Template annotations. This is faster, much more robust, but requires changing the chat template for the model\r\n  - [not recommend] Using turn prefixes. Very brittle, and only works for simple cases \r\n\r\n## Related issues\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-538\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-11-01T15:48:28Z"}
{"number": 796, "title": "Switch from writing individual conversations to all conversations", "files": ["src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Describe your change\r\nVLLM inference currently waits until inference is complete, then runs a for-loop which repeatedly opens the file and writes one conversation. This is seemingly causing a massive slowdown when using a mounted drive (i.e. GCS bucket), likely due to repeatedly needing to sync?\r\n\r\nThis change switches to opening the file once and writing all conversations at once.\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-31T18:33:26Z"}
{"number": 794, "title": "Fixing 2 broken links in README.md", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2024-10-30T22:56:33Z"}
{"number": 793, "title": "Add a prominent link to our docs.", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Describe your change\r\n\r\n\r\n\r\nAdd a prominent link to our docs.\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-30T22:08:32Z"}
{"number": 792, "title": "Updated default temperature from 1.0 to 0.0 for generation params", "files": ["src/oumi/core/configs/params/generation_params.py", "tests/unit/inference/test_generation_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Describe your change\nDefault temperature of 1.0 is too high for most models and can lead to strange behaviors or bad performance.\n\nGenerally all models are built to support 0.0 in some capacity (as greedy decoding), so proposing that we set this to the default.\n\n\n## Related issues\n\n\n\nFixes # (issue)\n\n\n## Before submitting\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\n- [x] Did you link the issue(s) related to this PR in the section above?\n- [x] Did you add / update tests where needed?\n\n\n## Reviewers\n\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\n\n\n", "merged_at": "2024-10-30T21:48:16Z"}
{"number": 791, "title": "CLI documentation updates.", "files": ["README.md", "docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Describe your change\r\n\r\n\r\n\r\nMinor documentation updates.\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-30T20:11:35Z"}
{"number": 789, "title": "Add `oumi launch stop` subcommand", "files": ["src/oumi/core/cli/launch.py", "src/oumi/core/cli/main.py", "src/oumi/core/launcher/base_cluster.py", "src/oumi/launcher/__init__.py", "src/oumi/launcher/clients/sky_client.py", "src/oumi/launcher/clusters/local_cluster.py", "src/oumi/launcher/clusters/polaris_cluster.py", "src/oumi/launcher/clusters/sky_cluster.py", "src/oumi/launcher/launcher.py", "tests/unit/core/cli/test_cli_launch.py", "tests/unit/core/cli/test_cli_main.py", "tests/unit/launcher/clients/test_sky_client.py", "tests/unit/launcher/clouds/test_sky_cloud.py", "tests/unit/launcher/clusters/test_local_cluster.py", "tests/unit/launcher/clusters/test_polaris_cluster.py", "tests/unit/launcher/clusters/test_sky_cluster.py", "tests/unit/launcher/test_launcher.py"], "area": "launcher", "area_votes": {"other": 2, "launcher": 6}, "body": "# Describe your change\r\n\r\nThis subcommand stops a cluster, shutting down the instance while preserving its disk. This is currently only relevant for the Sky clusters, by calling their `stop` method.\r\n\r\nTested by running `oumi launch up -c configs/examples/misc/hello_world_gcp_job.yaml --cluster test` followed by `oumi launch stop --cluster test`.\r\n\r\n## Related issues\r\n\r\nFixes OPE-650\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n", "merged_at": "2024-10-30T17:02:43Z"}
{"number": 788, "title": "Add sft packed dataset and tests", "files": ["src/oumi/core/datasets/__init__.py", "src/oumi/core/datasets/packed_sft_dataset.py", "tests/unit/core/datasets/test_packed_sft_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "# Describe your change\r\n\r\n- Add `PackedSftDataset` class that allows to pack an existing sft dataset to a constant length\r\n- Similar to TRL's `ConstantLengthDataset`, but for map sft dataset.\r\n- Unlike pre-training, splitting samples in SFT dataset can be problematic. Users can still do that if they want, but it's disabled by default\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes # (issue)\r\nTowards OPE-74\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2024-10-30T19:59:46Z"}
{"number": 787, "title": "Switch to the regular `pip`  temporarily in the old Polaris launcher", "files": ["scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "# Describe your change\r\n\r\n-- Switch from `uv` to regular `pip` \r\n-- `uv` is not found for some reason . It's a temporary quick fix, let's figure it out for real as a follow-up\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-28T21:40:58Z"}
{"number": 785, "title": "Define PAD token for Llama 2B model (fineweb pretrain)", "files": ["configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/oumi_train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/trl_train.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Describe your change\r\n\r\n\r\nPAD token is now required\r\nhttps://huggingface.co/HuggingFaceFW/ablation-model-fineweb-v1/blob/main/special_tokens_map.json\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-28T21:55:21Z"}
{"number": 782, "title": "Correct layer name in Llama vision accelerate config", "files": ["configs/examples/vision/llama3_2_vision/accelerate.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Describe your change\r\n\r\n-- Update the list of transformer layer names (replace a duplicate entry)\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nTowards OPE-552\r\nFixes OPE-468, OPE-632\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-28T16:08:15Z"}
{"number": 781, "title": "Clean up configs", "files": ["configs/examples/llama3_1/sft/405b_qlora/sky_job.yaml", "configs/examples/vision/blip2_opt_2.7b/oumi_gcp_job.yaml", "configs/examples/vision/blip2_opt_2.7b/sky_job.yaml", "configs/examples/vision/blip2_opt_2.7b/trl_gcp_job.yaml", "configs/examples/vision/llava_7b/oumi_gcp_job.yaml", "configs/examples/vision/llava_7b/sky_job.yaml", "configs/examples/vision/llava_7b/trl_gcp_job.yaml", "configs/projects/aya/evaluation/sky_job.yaml", "configs/projects/aya/sft/sky_job.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/zephyr/sft/full_sky_job.yaml", "configs/projects/zephyr/sft/qlora_sky_job.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/inference/70b_infer.yaml", "configs/recipes/llama3_1/inference/8b_infer.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/sky_job.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/sky_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/sky_job.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/sky_job.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/evaluation/3b_eval.yaml", "configs/recipes/llama3_2/inference/1b_infer.yaml", "configs/recipes/llama3_2/inference/3b_infer.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/phi3/sft/lora_train.yaml"], "area": "configs", "area_votes": {"configs": 37}, "body": "# Describe your change\r\n\r\n- Rename incorrectly named inference configs\r\n- Add Llama 3b eval config\r\n- Rename training runs and output dirs from sft to full (fine-tuning)\r\n- Clean up pip installs\r\n\r\n## Related issues\r\n\r\nFixes OPE-548\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2024-10-28T16:56:48Z"}
{"number": 778, "title": "Pre-training HSDP configs for Llama 8B for A100-40GB", "files": ["configs/examples/llama3_1/sft/70b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_full/sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/sky_job.yaml", "configs/recipes/llama3_1/pretraining/accelerate.yaml", "configs/recipes/llama3_1/pretraining/hsdp_sky_job.yaml", "configs/recipes/llama3_1/pretraining/train.yaml"], "area": "configs", "area_votes": {"configs": 7}, "body": "-- Only SkyPilot GCP config is initially included. Launcher configs, including for Polaris will follow shortly\r\n-- Minor cleanups in related configs under the same dir\r\n\r\nTowards OPE-570, OPE-413", "merged_at": "2024-11-05T00:23:38Z"}
{"number": 777, "title": "[WIP] Llama 3.2 Vision 11B configs", "files": ["configs/examples/llama3_1/sft/8b_full/sky_job.yaml", "configs/examples/vision/blip2_opt_2.7b/oumi_gcp_job.yaml", "configs/examples/vision/blip2_opt_2.7b/trl_gcp_job.yaml", "configs/examples/vision/llama3_2_vision/README.md", "configs/examples/vision/llama3_2_vision/accelerate.yaml", "configs/examples/vision/llama3_2_vision/infer.yaml", "configs/examples/vision/llama3_2_vision/sft_train.yaml", "configs/examples/vision/llama3_2_vision/trl_gcp_job.yaml", "configs/examples/vision/llava_7b/oumi_gcp_job.yaml", "scripts/benchmarks/minimal_multimodal_training.py"], "area": "configs", "area_votes": {"configs": 8, "docs": 1, "infra": 1}, "body": "-- Oumi Launcher and Skypilot configs \r\n-- Minor tweaks in `scripts/benchmarks/minimal_multimodal_training.py` and in LLAVA/BLIP2 configs\r\n-- Fix a typo in `configs/examples/llama3_1/sft/8b_full/sky_job.yaml`\r\n\r\nTowards OPE-468, OPE-632, OPE-552", "merged_at": "2024-10-25T17:43:29Z"}
{"number": 770, "title": "Add BaseInferenceEngine supported_params to keep track of inference engine parameters", "files": ["pyproject.toml", "src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/gcp_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/unit/inference/test_generation_params.py"], "area": "inference", "area_votes": {"infra": 1, "inference": 7}, "body": "# Describe your change\r\n\r\n- This PR updates the `BaseInferenceEngine` to add an official list of support parameters\r\n- If a user provides a config with non-supported parameters, a warning will be logged\r\n- This hopefully will help keep track of which engine supports  what parameter.\r\n- Added tests to check that all registered parameters are actually used, and that no non-registered parameter is used\r\n\r\n## Related issues\r\n\r\nFixes # (issue)\r\nCloses OPE-546\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2024-10-24T18:05:14Z"}
{"number": 766, "title": "Suggest using the git root directory when launching jobs via a CLI in dev-build.", "files": ["src/oumi/core/cli/launch.py", "src/oumi/train.py", "src/oumi/utils/git_utils.py", "src/oumi/utils/version_utils.py", "tests/unit/core/cli/test_cli_launch.py", "tests/unit/utils/test_version_utils.py"], "area": "other", "area_votes": {"other": 3, "training": 1}, "body": "# Describe your change\r\n\r\nRunning `oumi launch up` or `oumi launch run` will now check if you're running a dev version of Oumi. If you are, you'll be prompted with a Y/N confirmation to use the root of the current git repo (if one exists) as your working directory.\r\n\r\nIf no github repo exists we skip this confirmation step.\r\n\r\n## Related issues\r\n\r\nN/A\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-24T03:09:26Z"}
{"number": 765, "title": "Create 2 other Launcher configs for LLAVA and blip2 using OUMI training loop", "files": ["configs/examples/vision/blip2_opt_2.7b/oumi_gcp_job.yaml", "configs/examples/vision/llava_7b/oumi_gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "# Describe your change\r\n\r\n-- Adding more configs to simplify experiments\r\n-- Will likely drop some of the extra configs variants later. \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nTowards OPE-632\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-23T23:01:33Z"}
{"number": 763, "title": "Add sample oumi launcher configs for LLAVA and blip2", "files": ["configs/examples/vision/blip2_opt_2.7b/sky_job.yaml", "configs/examples/vision/blip2_opt_2.7b/trl_gcp_job.yaml", "configs/examples/vision/llava_7b/sky_job.yaml", "configs/examples/vision/llava_7b/trl_gcp_job.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Describe your change\r\n\r\n-- Sample GCP launcher configs \r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nTowards OPE-632\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-23T21:46:00Z"}
{"number": 762, "title": "Update vllm engine to allow for continuous batching", "files": ["src/oumi/inference/vllm_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Describe your change\r\n\r\n- Update vLLM engine to use all available GPUs by default\r\n- Update to use continuous batching -- we let vLLM handle the batching\r\n- One downside of this approach is we lose the ability to checkpoint progress. \r\n\r\n## Related issues\r\n\r\nCloses OPE-631, OPE-629\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2024-10-23T20:35:23Z"}
{"number": 761, "title": "Update LLAVA integration tests to use different test images", "files": [".vscode/launch.json", "tests/integration/infer/test_infer.py", "tests/integration/infer/test_native_text_inference_engine.py", "tests/testdata/images/cambrian.png", "tests/testdata/images/math.png", "tests/unit/core/types/test_conversation.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Describe your change\r\n\r\n-- Update tests to use new images\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nTowards OPE-468\r\nFixes OPE-401\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-23T17:21:56Z"}
{"number": 760, "title": "Update remote inference engines to show progress, reduce native engine verbosity", "files": ["src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "tests/unit/inference/test_remote_inference_engine.py"], "area": "inference", "area_votes": {"inference": 2}, "body": "# Describe your change\r\n\r\n- Update remote engines (openai, anthropic, gcp, ...) to show a tqdm progress bar\r\n- Update native text engine to avoid showing a progress bar when running inference on a single sample\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-23T21:41:05Z"}
{"number": 759, "title": "[tests] Add shared mock tokenizer mixture, requires_gpus marker", "files": [".pre-commit-config.yaml", "pyproject.toml", "tests/__init__.py", "tests/integration/evaluate/test_evaluate.py", "tests/integration/infer/test_infer.py", "tests/integration/infer/test_native_text_inference_engine.py", "tests/markers.py", "tests/unit/builders/test_collators.py", "tests/unit/builders/test_models.py", "tests/unit/builders/test_processors.py", "tests/unit/conftest.py", "tests/unit/core/collators/test_text_collator_with_padding.py", "tests/unit/core/collators/test_vision_language_collator_with_padding.py", "tests/unit/core/datasets/test_vision_language_dataset.py", "tests/unit/core/trainers/test_oumi_trainer.py", "tests/unit/datasets/test_vision_language_jsonlines_dataset.py", "tests/unit/performance/test_telemetry.py", "tests/unit/utils/test_device_utils.py"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Describe your change\r\n\r\n- `mock_tokenizer` is defined ~6 times in the unit tests, moved to the top level conftest\r\n- Created a markers module, and updated all tests to use it.\r\n- Updated marker to allow for configurable number of gpus, in preparation of the full integration tests\r\n- `tests` is now a module, so we can import utils etc defined under it\r\n\r\n## Related issues\r\n\r\nTowards OPE-38\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-23T18:47:47Z"}
{"number": 757, "title": "Update base dataset class to use dataset_path", "files": ["src/oumi/core/datasets/base_iterable_dataset.py", "src/oumi/core/datasets/base_map_dataset.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Describe your change\r\n\r\n- Now that we have `dataset_path`, users do not need to overload `dataset_name` to pass a filename to load\r\n- Update BasePretrainingDataset to consume `trust_remote_code` \r\n\r\nFixes # (issue)\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-23T00:12:10Z"}
{"number": 751, "title": "[docs] Add script to summarize configs", "files": ["docs/_docsummaries.sh", "docs/_summarize_module.py", "docs/models/recipes.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "# Describe your change\r\n\r\n- Update the module summarization script to also include configs\r\n- Update doc generation script to include configs\r\n- Add an index file for all the recipes\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T22:17:10Z"}
{"number": 750, "title": "[docs] Enable termynal cli and add example", "files": ["docs/conf.py", "docs/get_started/quickstart.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Describe your change\r\n\r\n[docs] Enable termynal cli and add example\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T22:52:42Z"}
{"number": 749, "title": "[docs] Update apidoc rst files with latest modules", "files": ["docs/api/oumi.core.collators.rst", "docs/api/oumi.core.processors.rst", "docs/api/oumi.core.rst", "docs/api/oumi.datasets.preference_tuning.rst", "docs/api/oumi.datasets.rst", "docs/api/oumi.datasets.sft.rst"], "area": "docs", "area_votes": {"docs": 6}, "body": "# Describe your change\r\n\r\n[docs] Update apidoc rst files with latest modules\r\n## Related issues\r\n\r\nFixes # (issue)\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2024-10-22T22:53:13Z"}
{"number": 748, "title": "[docs] Update about section", "files": ["docs/about/acknowledgements.md", "docs/about/changelog.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Describe your change\r\n\r\n- Add tentative changeling, ack section\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T22:56:04Z"}
{"number": 747, "title": "[docs] Add logo, analytics, and update sphinx config", "files": ["docs/_static/logo/favicon.png", "docs/_static/logo/oumi_logo_dark.png", "docs/_static/logo/oumi_logo_light.png", "docs/conf.py"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Describe your change\r\n\r\n- Add logo, analytics, and update sphinx config\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T22:53:54Z"}
{"number": 746, "title": "[docs] Add page for using skypilot", "files": ["docs/user_guides/launch/launch.md", "docs/user_guides/launch/skypilot.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Describe your change\r\n\r\n- Add page for using skypilot. \r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T23:06:11Z"}
{"number": 745, "title": "[docs] Add evaluate and judge user guides", "files": ["docs/user_guides/evaluate/evaluate.md", "docs/user_guides/judge/judge.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Describe your change\r\n\r\n- Add evaluate and judge user guides\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T16:24:48Z"}
{"number": 744, "title": "[docs] Update inference section", "files": ["docs/user_guides/infer/infer.md", "docs/user_guides/infer/inference_engines.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Describe your change\r\n\r\n- Update inference user guide\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T16:24:27Z"}
{"number": 743, "title": "Reorder IMAGE and TEXT messages in coco and Flick datasets", "files": ["src/oumi/datasets/vision_language/coco_captions.py", "src/oumi/datasets/vision_language/flickr30k.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Describe your change\r\n\r\nBetter if image goes first. It gives model a context for further questions.\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nTowards OPE-467, OPE-468\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-22T16:35:50Z"}
{"number": 741, "title": "[docs] Update training section", "files": ["docs/user_guides/train/train.md", "docs/user_guides/train/trainers.md", "docs/user_guides/train/training_config.md", "docs/user_guides/train/training_config_alt.rst"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Describe your change\r\n\r\n- Add dataset documentation pages for sft, pertaining, local datasets\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T16:27:49Z"}
{"number": 739, "title": "[docs] update faq section", "files": ["docs/faq/oom.md", "docs/faq/troubleshooting.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Describe your change\r\n\r\n- Add FAQ and OOM sections\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T04:02:04Z"}
{"number": 738, "title": "[docs] update datasets section", "files": ["docs/datasets/local_datasets.md", "docs/datasets/pretraining.md", "docs/datasets/sft.md", "docs/datasets/vl_sft.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "# Describe your change\r\n\r\n- Add dataset documentation pages for sft, pertaining, local datasets\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T15:46:28Z"}
{"number": 737, "title": "[docs] update advanced topics section", "files": ["docs/advanced/performance_optimization.md", "docs/advanced/quantization.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "# Describe your change\r\n\r\n- Add performance optimization page, and quantization page\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T04:04:42Z"}
{"number": 736, "title": "[docs] Cleanup placeholder documentation pages", "files": ["docs/about/citations.md", "docs/about/license.md", "docs/advanced/custom_evaluation.md", "docs/advanced/custom_vl_datasets.md", "docs/advanced/customization.md", "docs/advanced/distributed_training.md", "docs/development/dev_setup.md", "docs/get_started/configuration.md", "docs/user_guides/evaluate/lm_harness.md"], "area": "docs", "area_votes": {"docs": 9}, "body": "# Describe your change\r\n\r\n- Clean-up placeholder documentation pages\r\n- This is the first in a series of PRs to bring the documentation in `main` branch up to date with experimental `oelachqar/edge` used to generate the documentation\r\n\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-589\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?", "merged_at": "2024-10-22T01:00:56Z"}
{"number": 735, "title": "Rename base datasets modules", "files": ["docs/get_started/configuration.md", "src/oumi/core/datasets/__init__.py", "src/oumi/core/datasets/base_dpo_dataset.py", "src/oumi/core/datasets/base_iterable_dataset.py", "src/oumi/core/datasets/base_map_dataset.py", "src/oumi/core/datasets/base_pretraining_dataset.py", "src/oumi/core/datasets/base_sft_dataset.py", "src/oumi/core/datasets/vision_language_dataset.py", "src/oumi/datasets/debug.py", "src/oumi/datasets/preference_tuning/orpo_dpo_mix.py", "src/oumi/datasets/pretraining/c4.py", "src/oumi/datasets/pretraining/dolma.py", "src/oumi/datasets/pretraining/falcon_refinedweb.py", "src/oumi/datasets/pretraining/fineweb_edu.py", "src/oumi/datasets/pretraining/pile.py", "src/oumi/datasets/pretraining/red_pajama_v1.py", "src/oumi/datasets/pretraining/red_pajama_v2.py", "src/oumi/datasets/pretraining/slim_pajama.py", "src/oumi/datasets/pretraining/starcoder.py", "src/oumi/datasets/pretraining/the_stack.py", "src/oumi/datasets/pretraining/tiny_stories.py", "src/oumi/datasets/pretraining/tiny_textbooks.py", "src/oumi/datasets/pretraining/wikipedia.py", "src/oumi/datasets/pretraining/wikitext.py", "src/oumi/datasets/pretraining/youtube_commons.py", "src/oumi/datasets/sft/alpaca.py", "src/oumi/datasets/sft/aya.py", "src/oumi/datasets/sft/chatqa.py", "src/oumi/datasets/sft/chatrag_bench.py", "src/oumi/datasets/sft/dolly.py", "src/oumi/datasets/sft/magpie.py", "src/oumi/datasets/sft/sft_jsonlines.py", "src/oumi/datasets/sft/ultrachat.py", "src/oumi/judge.py", "tests/core/datasets/test_pretraining_dataset.py", "tests/integration/datasets/test_sft_datasets_load_datasets.py"], "area": "data", "area_votes": {"docs": 1, "data": 31, "evaluation": 1}, "body": "# Describe your change\r\n\r\n- Move `BaseSftDataset` and `BasePretrainingDataset` into their own modules\r\n- Rename base dataset classes for consistency:\r\n  - `BaseLMSftDataset` -> `BaseSftDataset`\r\n  - `BasePretrainingIterableDataset` -> `BasePretrainingDataset`\r\n  - `BaseExperimentalDpoPreprocessor` -> `BaseExperimentalDpoDataset`\r\n  - `oumi.core.datasets.base_dataset` ->  `oumi.core.datasets.base_map_dataset`\r\n  - `oumi.core.datasets.iterable_dataset` ->  `oumi.core.datasets.base_iterable_dataset`\r\n\r\nFixes # (issue)\r\n\r\nTowards OPE-598\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-22T17:02:23Z"}
{"number": 734, "title": "Add inference engine for vertex GCP", "files": ["src/oumi/inference/__init__.py", "src/oumi/inference/gcp_inference_engine.py", "tests/inference/test_gcp_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "# Describe your change\r\n\r\n- Add inference engine for google vertex using openai api format\r\n\r\nFixes # (issue)\r\nTowards OPE-457\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2024-10-21T23:50:43Z"}
{"number": 730, "title": "Add a helper function `is_image_text_llm()` to detect supported image+text LLMs", "files": ["src/oumi/builders/models.py", "tests/builders/test_models.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Describe your change\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nFixes OPE-619\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-21T22:22:25Z"}
{"number": 729, "title": "Rename LLAVA  sft_sky_job.yaml for consistency with blip2", "files": ["configs/examples/vision/llava_7b/sky_job.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "# Describe your change\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nTowards  OPE-467\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [X] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [X] Did you link the issue(s) related to this PR in the section above?\r\n- [X] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-21T19:10:29Z"}
{"number": 728, "title": "Add DPO base class, initial dataset, integration tests", "files": ["src/oumi/core/datasets/__init__.py", "src/oumi/core/datasets/base_dpo_processor.py", "src/oumi/datasets/__init__.py", "src/oumi/datasets/preference_tuning/__init__.py", "src/oumi/datasets/preference_tuning/orpo_dpo_mix.py", "src/oumi/datasets/preference_tuning/trl_dpo_preprocessor.py", "tests/integration/datasets/test_preference_tuning_datasets_full_epoch.py"], "area": "data", "area_votes": {"data": 3}, "body": "# Describe your change\r\n\r\n- Add experimental base class for preference tuning using DPO\r\n  - This class is experimental for now, as we need to align on a default structure for DPO datasets\r\n  - The current base class matches the pre-existing logic, to convert OrpoDpoMix dataset into the format consumed by TRL's DPO trainer. \r\n- Add integration tests for preference tuning datasets\r\n\r\nFixes # (issue)\r\nCloses OPE-435\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2024-10-21T19:55:54Z"}
{"number": 727, "title": "And WandB integration to eval", "files": ["configs/projects/aya/evaluation/eval.yaml", "configs/projects/aya/evaluation/sky_job.yaml", "configs/projects/zephyr/evaluation/eval.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/gpt2/evaluation/async_sky_job.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/inference/70b_infer.yaml", "configs/recipes/llama3_1/inference/8b_infer.yaml", "configs/recipes/phi3/evaluation/eval.yaml", "configs/recipes/phi3/evaluation/sky_job.yaml", "pyproject.toml", "src/oumi/core/configs/evaluation_config.py", "src/oumi/core/configs/params/training_params.py", "src/oumi/core/trainers/oumi_trainer.py", "src/oumi/evaluate.py", "src/oumi/utils/str_utils.py", "tests/unit/utils/test_str_utils.py"], "area": "configs", "area_votes": {"configs": 17, "infra": 1, "training": 1, "evaluation": 1, "other": 1}, "body": "# Describe your change\r\n\r\nReport eval results to WandB. Similar to training, this requires setting the WandB project env var, along with the `enable_wandb` and `run_name` config params. Also, this PR removes \"default\" as the default value for `run_name`, as the existing comment about it didn't seem to be true when I tested it.\r\n\r\n## Related issues\r\n\r\nFixes OPE-452\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2024-10-23T06:58:18Z"}
{"number": 726, "title": "Remove deprecated preprocessing function", "files": ["configs/examples/llama3_1/sft/405b_qlora/train.yaml", "configs/examples/llama3_1/sft/70b_lora/train.yaml", "configs/examples/llama3_1/sft/70b_qlora/train.yaml", "configs/examples/llama3_1/sft/8b_full/train.yaml", "configs/examples/llama3_1/sft/8b_lora/train.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_train.yaml", "configs/examples/llama3_1/sft/8b_qlora/train.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/zephyr/sft/full_train.yaml", "configs/projects/zephyr/sft/qlora_train.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/phi3/dpo/mac_train.yaml", "configs/recipes/phi3/dpo/nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/nvidia_80g_train.yaml", "configs/recipes/phi3/dpo/train.yaml", "configs/recipes/phi3/sft/lora_train.yaml", "configs/recipes/phi3/sft/mac_lora_train.yaml", "scripts/finetune_sft_phi3.sh", "src/experimental/Oumi - Colab Setup Example.ipynb", "src/oumi/builders/data.py", "src/oumi/core/configs/params/data_params.py", "tests/core/cli/test_cli_launch.py", "tests/core/cli/test_cli_train.py", "tests/integration/train/test_train.py", "tests/test_launch.py"], "area": "configs", "area_votes": {"configs": 27, "infra": 1, "docs": 1, "other": 1}, "body": "# Describe your change\r\n\r\n- Remove deprecated code paths for manually specifying a dataset preprocessing function\r\n- Remove remaining references in code and configs (those were not used in a while)\r\n\r\nFixes # (issue)\r\nTowards OPE-598\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2024-10-21T19:32:09Z"}
{"number": 723, "title": "Update `tests/builders/test_lr_schedules.py`", "files": ["src/oumi/builders/lr_schedules.py", "tests/builders/test_lr_schedules.py"], "area": "other", "area_votes": {"other": 1}, "body": "-- No functional changes. Just test changes to resolve obscure test failures: `initial_lr` not initialized in param_group https://discuss.pytorch.org/t/a-problem-occured-when-resuming-an-optimizer/28822\r\n-- Test changes: Add an extra Invocation of  `build_lr_schedule()`  for epoch 0 to set `initial_lr`\r\n-- Some `patch` imports changes (otherwise, `patch` wasn't effective)\r\n\r\nFixes OPE-618", "merged_at": "2024-10-20T04:35:03Z"}
{"number": 720, "title": "Move config files to new directory structure", "files": [".vscode/launch.json", "Makefile", "README.md", "configs/examples/bulk_inference/gcp_job.yaml", "configs/examples/bulk_inference/mistral_small_infer.yaml", "configs/examples/chatqa/chatqa_stage1_train.yaml", "configs/examples/chatqa/chatqa_stage2_train.yaml", "configs/examples/chatqa/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/accelerate.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/accelerate_ddp_sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/ddp/train.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/accelerate.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/ds_config.json", "configs/examples/fineweb_ablation_pretraining/deepspeed/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/deepspeed/train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/accelerate.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/oumi_gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/oumi_train.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/polaris_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/sky_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/trl_gcp_job.yaml", "configs/examples/fineweb_ablation_pretraining/fsdp/trl_train.yaml", "configs/examples/llama3_1/sft/405b_qlora/accelerate.yaml", "configs/examples/llama3_1/sft/405b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/405b_qlora/train.yaml", "configs/examples/llama3_1/sft/70b_lora/accelerate.yaml", "configs/examples/llama3_1/sft/70b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_lora/train.yaml", "configs/examples/llama3_1/sft/70b_qlora/accelerate.yaml", "configs/examples/llama3_1/sft/70b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/70b_qlora/train.yaml", "configs/examples/llama3_1/sft/8b_full/accelerate.yaml", "configs/examples/llama3_1/sft/8b_full/sky_job.yaml", "configs/examples/llama3_1/sft/8b_full/train.yaml", "configs/examples/llama3_1/sft/8b_lora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_lora/train.yaml", "configs/examples/llama3_1/sft/8b_qlora/accelerate.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/fsdp_train.yaml", "configs/examples/llama3_1/sft/8b_qlora/sky_job.yaml", "configs/examples/llama3_1/sft/8b_qlora/train.yaml", "configs/examples/misc/hello_world_gcp_job.yaml", "configs/examples/misc/hello_world_polaris_job.yaml", "configs/examples/misc/sky_init.sh", "configs/examples/misc/sky_ssh_job.yaml", "configs/examples/misc/vllm_gcp_job.yaml", "configs/examples/vision/blip2_opt_2.7b/sft_train.yaml", "configs/examples/vision/blip2_opt_2.7b/sky_job.yaml", "configs/examples/vision/llava_7b/infer.yaml", "configs/examples/vision/llava_7b/sft_sky_job.yaml", "configs/examples/vision/llava_7b/sft_train.yaml", "configs/projects/aya/evaluation/eval.yaml", "configs/projects/aya/evaluation/sky_job.yaml", "configs/projects/aya/sft/sky_job.yaml", "configs/projects/aya/sft/train.yaml", "configs/projects/zephyr/evaluation/eval.yaml", "configs/projects/zephyr/sft/full_sky_job.yaml", "configs/projects/zephyr/sft/full_train.yaml", "configs/projects/zephyr/sft/qlora_sky_job.yaml", "configs/projects/zephyr/sft/qlora_train.yaml", "configs/recipes/gpt2/evaluation/async_eval.yaml", "configs/recipes/gpt2/evaluation/async_sky_job.yaml", "configs/recipes/gpt2/inference/infer.yaml", "configs/recipes/gpt2/pretraining/accelerate.yaml", "configs/recipes/gpt2/pretraining/cpu_accelerate.yaml", "configs/recipes/gpt2/pretraining/fsdp_sky_job.yaml", "configs/recipes/gpt2/pretraining/mac_train.yaml", "configs/recipes/gpt2/pretraining/sky_job.yaml", "configs/recipes/gpt2/pretraining/train.yaml", "configs/recipes/llama3_1/evaluation/70b_eval.yaml", "configs/recipes/llama3_1/evaluation/70b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/70b_polaris_job.yaml", "configs/recipes/llama3_1/evaluation/8b_eval.yaml", "configs/recipes/llama3_1/evaluation/8b_gcp_job.yaml", "configs/recipes/llama3_1/evaluation/8b_polaris_job.yaml", "configs/recipes/llama3_1/inference/70b_eval.yaml", "configs/recipes/llama3_1/inference/8b_eval.yaml", "configs/recipes/llama3_1/sft/70b_full/accelerate.yaml", "configs/recipes/llama3_1/sft/70b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_full/sky_job.yaml", "configs/recipes/llama3_1/sft/70b_full/train.yaml", "configs/recipes/llama3_1/sft/70b_lora/accelerate.yaml", "configs/recipes/llama3_1/sft/70b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/sky_job.yaml", "configs/recipes/llama3_1/sft/70b_lora/train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/accelerate.yaml", "configs/recipes/llama3_1/sft/70b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_full/accelerate.yaml", "configs/recipes/llama3_1/sft/8b_full/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_full/longctx_train.yaml", "configs/recipes/llama3_1/sft/8b_full/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_full/sky_job.yaml", "configs/recipes/llama3_1/sft/8b_full/train.yaml", "configs/recipes/llama3_1/sft/8b_lora/accelerate.yaml", "configs/recipes/llama3_1/sft/8b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/polaris_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/sky_job.yaml", "configs/recipes/llama3_1/sft/8b_lora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/accelerate.yaml", "configs/recipes/llama3_1/sft/8b_qlora/gcp_job.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/evaluation/1b_eval.yaml", "configs/recipes/llama3_2/inference/1b_infer.yaml", "configs/recipes/llama3_2/inference/3b_infer.yaml", "configs/recipes/llama3_2/sft/1b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_full/accelerate.yaml", "configs/recipes/llama3_2/sft/3b_full/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_full/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_full/train.yaml", "configs/recipes/llama3_2/sft/3b_lora/accelerate.yaml", "configs/recipes/llama3_2/sft/3b_lora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_lora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/accelerate.yaml", "configs/recipes/llama3_2/sft/3b_qlora/fsdp_gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/gcp_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/polaris_job.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "configs/recipes/phi3/dpo/accelerate.yaml", "configs/recipes/phi3/dpo/fsdp_sky_job.yaml", "configs/recipes/phi3/dpo/mac_train.yaml", "configs/recipes/phi3/dpo/nvidia_24g_train.yaml", "configs/recipes/phi3/dpo/nvidia_80g_train.yaml", "configs/recipes/phi3/dpo/sky_job.yaml", "configs/recipes/phi3/dpo/train.yaml", "configs/recipes/phi3/evaluation/eval.yaml", "configs/recipes/phi3/evaluation/sky_job.yaml", "configs/recipes/phi3/sft/lora_train.yaml", "configs/recipes/phi3/sft/mac_lora_train.yaml", "docs/get_started/configuration.md", "docs/user_guides/launch/launch.md", "docs/user_guides/train/distributed_training.md", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Multinode Inference on Polaris.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Tuning Llama.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "scripts/benchmarks/benchmark_trainers.sh", "scripts/llama_e2e.py", "scripts/llm-as-judge/judge_dataset_polaris.py", "scripts/polaris/jobs/example_job.sh", "scripts/polaris/jobs/llama2b_pt_worker.sh", "scripts/polaris/jobs/llama8b_lora_eval.sh", "scripts/polaris/jobs/llama_tune.sh", "src/experimental/Oumi - Build Zephyr 7B.ipynb", "src/experimental/Oumi - Colab Setup Example.ipynb", "src/experimental/pretokenize/sky.yaml", "tests/core/configs/test_parse_configs.py"], "area": "configs", "area_votes": {"other": 2, "infra": 8, "docs": 11, "configs": 134}, "body": "# Describe your change\r\n\r\nExact actions taken in this PR:\r\n- Move all files from configs/accelerate/, configs/oumi/, and configs/skypilot/ to new directory structure\r\n- Some new files were created (ex. accelerate configs) so that each directory is self-contained\r\n- Renamed files\r\n- Updated important file path references\r\n- Cleaned up some unneeded files/comments\r\n\r\nThis replaces #718 which didn't preserve file history\r\n\r\nFuture PRs:\r\n- Finalize new directories (fix all links, add READMEs, etc.)\r\n- Migrate sky configs to job configs\r\n- Delete stale files in new directories\r\n\r\n## Related issues\r\n\r\nTowards OPE-399, OPE-275\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n", "merged_at": "2024-10-19T08:22:05Z"}
{"number": 719, "title": "Update index.rst", "files": ["docs/index.rst"], "area": "docs", "area_votes": {"docs": 1}, "body": "OPE-588 (kind of) removing the space between \"Oumi\" and the exclamation point in the \"Welcome to the documentation for Oumi!\".\r\n\r\n# Describe your change\r\n\r\n\r\n\r\n## Related issues\r\n\r\n\r\n\r\nTowards OPE-588\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [ ] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [ ] Did you link the issue(s) related to this PR in the section above?\r\n- [ ] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-20T00:43:08Z"}
{"number": 717, "title": "Make sure registry is case agnostic", "files": ["src/oumi/core/registry.py", "tests/core/test_registry.py"], "area": "other", "area_votes": {"other": 1}, "body": "# Describe your change\r\n\r\n- Update registry key to always be lowercase\r\n- This allows us to register and retrieve objects without having to worry about the exact case the authors used\r\n\r\nCloses OPE-576\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2024-10-19T01:17:49Z"}
{"number": 715, "title": "Update index.rst - switch expanded name Open Universal Machine Intelligence and replace with \"Oumi\"", "files": ["docs/index.rst"], "area": "docs", "area_votes": {"docs": 1}, "body": "OPE-588 - [Feedback] In oumi/docs switch remove the expanded name Open Universal Machine Intelligence and replace with \"Oumi\"\r\n\r\n# Describe your change\r\n\r\n\r\n\r\n## Related issues\r\n\r\nTowards OPE-588\r\n\r\n\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-18T20:58:51Z"}
{"number": 713, "title": "Add Llama inference configs", "files": ["configs/oumi/llama1b.infer.yaml", "configs/oumi/llama3b.infer.yaml", "configs/oumi/llama70b.infer.yaml", "configs/oumi/llama8b.infer.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "# Describe your change\r\n\r\nAdd Llama 3B/8B/70B inference configs. Also slightly adjust Llama 1B's inference config to be more useful.\r\n\r\n## Related issues\r\n\r\nTowards OPE-548\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?", "merged_at": "2024-10-18T19:13:20Z"}
{"number": 711, "title": "Add github issue templates.", "files": [".github/ISSUE_TEMPLATE/bug-report.yaml", ".github/ISSUE_TEMPLATE/feature-request.yaml"], "area": "infra", "area_votes": {"infra": 2}, "body": "# Describe your change\r\n\r\nAdded templates for feature requests and bugs. Both of these templates will use the new `triage` tag so we can filter which issues we've already triaged publicly.\r\n\r\n## Related issues\r\n\r\n\r\nFixes OPE-460\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n\r\n", "merged_at": "2024-10-18T17:50:12Z"}
{"number": 705, "title": "Update experimental async dataset feature", "files": ["configs/oumi/llama1b.sft.yaml", "configs/oumi/llama2b.pt.fsdp.oumi.yaml", "configs/oumi/llama2b.pt.fsdp.trl.yaml", "configs/oumi/llama2b.pt.yaml", "configs/oumi/llama3.8b.sft.longctx.yaml", "configs/oumi/llama3b.lora.yaml", "configs/oumi/llama3b.qlora.yaml", "configs/oumi/llama3b.sft.yaml", "configs/oumi/llama70b.lora.yaml", "configs/oumi/llama70b.qlora.yaml", "configs/oumi/llama70b.sft.yaml", "configs/oumi/llama8b.lora.yaml", "configs/oumi/llama8b.qlora.yaml", "configs/oumi/llama8b.sft.yaml", "scripts/benchmarks/benchmark_dataloader.py", "scripts/polaris/jobs/llama2b_pt_worker.sh", "src/oumi/builders/data.py", "src/oumi/core/configs/params/data_params.py", "src/oumi/core/datasets/__init__.py", "src/oumi/core/datasets/pretraining_async_text_dataset.py", "src/oumi/datasets/__init__.py", "tests/datasets/test_pretraining_async_text_dataset.py", "tests/integration/builders/test_data.py"], "area": "configs", "area_votes": {"configs": 15, "infra": 2, "other": 1, "data": 1}, "body": "# Update experimental dataset feature\r\n- Rename `experimental_use_async_dataset` to `use_async_dataset`.\r\n  - We've been using it consistently for a while now, time to drop the experimental flag.\r\n- Move `PretrainingAsyncTextDataset` to `oumi.core.datasets` instead of `oumi.datasets`, as it's not a dataset implementation    \r\n\r\n\r\n## Related issues\r\n\r\nOPE-598\r\n\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n", "merged_at": "2024-10-17T22:53:58Z"}
{"number": 704, "title": "Add main entrypoint, update vscode launch.json to use new cli", "files": [".vscode/launch.json", "src/oumi/__main__.py"], "area": "other", "area_votes": {"other": 2}, "body": "# Add main entrypoint, update vscode launch.json to use new cli\r\n\r\nThis PR updates the vscode launch.json to use the new `oumi` cli\r\n\r\n## Related issues\r\n\r\nOPE-599\r\n\r\n\r\n## Before submitting\r\n- [ ] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n", "merged_at": "2024-10-17T22:42:22Z"}
{"number": 700, "title": "Replace `logger.exception()` in `TextCollatorWithPadding` with `logger.error()`", "files": ["src/oumi/core/collators/text_collator_with_padding.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Reduce noise in test logs\r\n-- `logger.exception` looks like a false positive in test logs ", "merged_at": "2024-10-17T19:58:50Z"}
{"number": 699, "title": "Update new image CLI to support input image for VLLMs", "files": ["src/oumi/core/cli/infer.py", "tests/core/cli/test_cli_infer.py"], "area": "other", "area_votes": {"other": 1}, "body": "-- Porting https://github.com/oumi-ai/oumi/pull/686 to the new CLI.\r\n\r\nTowards OPE-401, OPE-467", "merged_at": "2024-10-17T22:54:51Z"}
{"number": 698, "title": "Update our dev set up instructions to not conflict with the new CLI", "files": ["docs/development/dev_setup.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Any devs who set up an `oumi` shortcut in their .zshrc will hit a naming collision when trying to use the new `oumi` CLI.", "merged_at": "2024-10-17T18:20:40Z"}
{"number": 697, "title": "Add a dummy sft dataset for unit tests", "files": ["src/oumi/datasets/debug.py"], "area": "data", "area_votes": {"data": 1}, "body": "**Changes**\r\n- Add a dummy sft datasets for unit tests\r\n\r\nTowards OPE-494,  OPE-598", "merged_at": "2024-10-17T18:28:41Z"}
{"number": 696, "title": "[Clean-up] Remove dead code (custom eval)", "files": ["src/oumi/evaluation/__init__.py", "src/oumi/evaluation/compute_metrics.py", "src/oumi/evaluation/infer_prob.py", "tests/evaluation/test_infer_prob.py"], "area": "evaluation", "area_votes": {"evaluation": 2}, "body": "", "merged_at": "2024-10-17T21:30:12Z"}
{"number": 695, "title": "Add back the polling spinner to the Launch CLI", "files": ["src/oumi/core/cli/launch.py", "tests/core/cli/test_cli_launch.py"], "area": "other", "area_votes": {"other": 1}, "body": "Adds back the polling spinner functionality that was removed when migrating from threadpools -> multiprocess polling.\r\n\r\nFixes OPE-584", "merged_at": "2024-10-17T18:22:37Z"}
{"number": 693, "title": "Create pull_request_template.md", "files": [".github/pull_request_template.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "# Describe your change\r\n\r\nAdd a default PR template for contributors\r\n\r\n## Related issues\r\n\r\n\r\nFixes OPE-461\r\n\r\n\r\n## Before submitting\r\n- [x] This PR only changes documentation. (You can ignore the following checks in that case)\r\n- [x] Did you read the [contributor guideline](https://github.com/oumi-ai/oumi/blob/main/CONTRIBUTING.md) Pull Request guidelines?\r\n- [x] Did you link the issue(s) related to this PR in the section above?\r\n- [x] Did you add / update tests where needed?\r\n\r\n\r\n## Reviewers\r\n\r\nAt least one review from a member of `oumi-ai/oumi-staff` is required.\r\n\r\n", "merged_at": "2024-10-17T22:02:16Z"}
{"number": 692, "title": "Set `model.tokenizer_pad_token` for gpt2 in Oumi Inference example in `vsconfig`", "files": [".vscode/launch.json"], "area": "other", "area_votes": {"other": 1}, "body": "\r\n\r\nTowards OPE-597", "merged_at": "2024-10-17T17:39:39Z"}
{"number": 691, "title": "Move sft and preference tuning datasets to their own module", "files": ["src/oumi/builders/data.py", "src/oumi/datasets/__init__.py", "src/oumi/datasets/preference_tuning/__init__.py", "src/oumi/datasets/preference_tuning/trl_dpo_preprocessor.py", "src/oumi/datasets/sft/__init__.py", "src/oumi/datasets/sft/alpaca.py", "src/oumi/datasets/sft/aya.py", "src/oumi/datasets/sft/chatqa.py", "src/oumi/datasets/sft/chatrag_bench.py", "src/oumi/datasets/sft/dolly.py", "src/oumi/datasets/sft/magpie.py", "src/oumi/datasets/sft/sft_jsonlines.py", "src/oumi/datasets/sft/ultrachat.py", "tests/datasets/test_text_jsonlines_dataset.py"], "area": "data", "area_votes": {"other": 1, "data": 9}, "body": "**Changes**\r\n- Similar to `pretraining`, move the text `sft` and `preference_tuning` datasets to their own module.\r\n- This makes it easier to find the right dataset in code and documentation\r\n\r\nTowards OPE-598", "merged_at": "2024-10-17T17:30:52Z"}
{"number": 688, "title": "Add basic validation for data collators", "files": ["src/oumi/core/configs/params/data_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Currently, max 1 collator is allowed for all splits\r\n\r\nFixes OPE-554", "merged_at": "2024-10-17T16:19:36Z"}
{"number": 684, "title": "Training configurations llama3.1 + ChatML-Template", "files": ["configs/oumi/tasks/llama3.1/train/ddp/lora/8B.sft/skypilot.yaml", "configs/oumi/tasks/llama3.1/train/ddp/lora/8B.sft/train.yaml", "configs/oumi/tasks/llama3.1/train/ddp/qlora/8B.sft/skypilot.yaml", "configs/oumi/tasks/llama3.1/train/ddp/qlora/8B.sft/train.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/full/8B.sft/fsdp.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/full/8B.sft/skypilot.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/full/8B.sft/train.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/lora/70B.sft/fsdp.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/lora/70B.sft/skypilot.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/lora/70B.sft/train.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/405B.sft/fsdp.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/405B.sft/skypilot.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/405B.sft/train.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/70B.sft/fsdp.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/70B.sft/skypilot.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/70B.sft/train.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/8B.sft/fsdp.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/8B.sft/skypilot.yaml", "configs/oumi/tasks/llama3.1/train/fsdp/qlora/8B.sft/train.yaml", "pyproject.toml", "src/oumi/core/configs/params/training_params.py", "src/oumi/datasets/chat_templates/chat_ml.jinja", "tests/core/configs/test_parse_configs.py"], "area": "configs", "area_votes": {"configs": 20, "infra": 1, "data": 1}, "body": "Contributing independent folders containing complete information for full runs of training different llama 3.1 models.\r\n\r\nThese configurations combine elements from [torchtune](https://github.com/pytorch/torchtune/tree/main/recipes/configs), [llama-recipies](https://github.com/meta-llama/llama-recipes) and the original repo of [alpaca](https://github.com/tatsu-lab/stanford_alpaca?tab=readme-ov-file#fine-tuning) (which is the dataset used across all configs). There are comments inside the .yaml's delinating the differences.\r\n\r\nCompared to most of [our existing configurations](https://github.com/oumi-ai/oumi/tree/main/configs/oumi), the key differences are: \r\n1) they train *base* models\r\n2) they use the chat_ml chat-template\r\n3) they train for >= 1 epoch | as suggested by the above libraries\r\n\r\nMinor addition: adds the chat-ml template in our library. This is a widely used template; and could become our default standard or at least expected to be utilized a lot.\r\n\r\nTowards OPE-547\r\n\r\n\r\n\r\n\r\n", "merged_at": "2024-10-18T02:27:01Z"}
{"number": 683, "title": "Small edits to Llama configs", "files": ["configs/oumi/jobs/gcp/llama3b_sft.yaml", "configs/oumi/llama8b.sft.yaml", "configs/skypilot/sky_ssh.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "Fixes OPE-547\r\n\r\n- Cap Llama 3B SFT model max length to 512 to prevent OOMs, similar to Polaris config\r\n- Set Llama 8B SFT epochs to 3 to match torchtune\r\n- Mount GCS dir to dev GCP job", "merged_at": "2024-10-16T23:19:42Z"}
{"number": 682, "title": "Add cff file for citations", "files": ["CITATION.cff"], "area": "docs", "area_votes": {"docs": 1}, "body": "**Changes**\r\n- Add a citation file in cff format\r\n\r\nYou can learn more about the CFF format [here](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-citation-files). It's supported natively by GitHub and will show up the UI\r\n\r\n\r\nCloses OPE-593", "merged_at": "2024-10-16T21:25:42Z"}
{"number": 681, "title": "Add Llama 3B DDP configs", "files": ["configs/oumi/jobs/gcp/llama3b_lora.yaml", "configs/oumi/jobs/gcp/llama3b_qlora.yaml", "configs/oumi/jobs/gcp/llama3b_sft.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "Fixes OPE-547", "merged_at": "2024-10-16T21:15:52Z"}
{"number": 680, "title": "Adds control for an explicit quantization-compute dtype", "files": ["configs/examples/llama3_1/sft/405b_qlora/train.yaml", "configs/examples/llama3_1/sft/8b_qlora/train.yaml", "configs/projects/zephyr/sft/qlora_train.yaml", "configs/recipes/llama3_1/sft/70b_qlora/train.yaml", "configs/recipes/llama3_1/sft/8b_qlora/train.yaml", "configs/recipes/llama3_2/sft/3b_qlora/train.yaml", "src/oumi/builders/models.py", "src/oumi/core/configs/params/model_params.py", "src/oumi/core/configs/params/peft_params.py"], "area": "configs", "area_votes": {"configs": 8, "other": 1}, "body": "Exposes and handles a parameter that controls the precision of the computation of a quantitated model (wrapping BitsAndBytes: `bnb_4bit_compute_dtype`).\r\n\r\nThis enables finetuning with quantized models that operate with a compute type that is specified by the user **explicitly**. -- besides this being handy for speedups (e.g., when the input is high-precision but your learning is happening with low-precision parameters) -- this is also necessary when we are loading quantized models where we want to instantiate them with Oumi following their original quantization (arbitrary) configuration.", "merged_at": "2024-10-27T05:20:45Z"}
{"number": 679, "title": "Cleanup vscode launch.json", "files": [".vscode/launch.json"], "area": "other", "area_votes": {"other": 1}, "body": "**Changes**\r\n- Keep a minimal set of launch config, one for each major `oumi` command.\r\n- Remove older commands / syntax\r\n- The selected configs need a refresh, but that will done as a follow-up PR\r\n\r\n\r\n![image](https://github.com/user-attachments/assets/1b4d31e6-6943-41bb-afc5-c1505316ae74)\r\n\r\n\r\nCloses OPE-274", "merged_at": "2024-10-16T19:56:37Z"}
{"number": 676, "title": "Add image support to `llama3-instruct.jinja` chat template", "files": ["src/oumi/datasets/chat_templates/llama3-instruct.jinja"], "area": "data", "area_votes": {"data": 1}, "body": "-- Add support for image tokens\r\n-- Should have no effect for data that has no images (only `message['type'] == 'text'`)\r\n\r\nTowards OPE-468", "merged_at": "2024-10-16T19:56:29Z"}
{"number": 675, "title": "Rename emails and website url to Oumi", "files": ["CODE_OF_CONDUCT.md", "README.md", "docs/development/code_of_conduct.md", "docs/development/dev_setup.md", "notebooks/Oumi - A Tour.ipynb", "pyproject.toml"], "area": "docs", "area_votes": {"docs": 5, "infra": 1}, "body": "Towards OPE-364", "merged_at": "2024-10-15T23:02:49Z"}
{"number": 670, "title": "[docs] Add script to generate module summaries", "files": ["Makefile", "docs/_docsummaries.sh", "docs/_summarize_module.py", "docs/conf.py", "docs/datasets/local_datasets.md", "docs/datasets/preference_tuning.md", "docs/datasets/pretraining.md", "docs/datasets/sft.md", "docs/datasets/vl_sft.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/judge/judge.md"], "area": "docs", "area_votes": {"infra": 1, "docs": 10}, "body": "**Changes**\r\n- Add a script to generate a summary table for a module. \r\n  - For example, to list all the inference engines, datasets, models, etc that we support and render as a markdown table  \r\n- This avoids having to manually update the documentation tables.\r\n- Update docs to include summaries for datasets, judges, and inference engines\r\n\r\nTowards OPE-589\r\n\r\nNote: Merged directly to unblock. This PR has no doc content change to review and does not impact the package runtime.", "merged_at": "2024-10-14T02:02:14Z"}
{"number": 668, "title": "[docs] Add content links and references", "files": ["USAGE.md", "docs/_doclinks.config", "docs/advanced/custom_datasets.md", "docs/index.rst", "docs/user_guides/evaluate/evaluate.md", "docs/user_guides/infer/infer.md", "docs/user_guides/judge/custom_judge.md", "docs/user_guides/judge/judge.md", "docs/user_guides/judge/oumi_judge.md", "docs/user_guides/launch/launch.md", "docs/user_guides/train/train.md"], "area": "docs", "area_votes": {"docs": 11}, "body": "**Changes**\r\n- This PR updates the sphinx index to leverage existing content from access the repo (notebooks, and readmes)\r\n- Remove redundant files\r\n\r\nNote: Merged directly to unblock. This PR has no doc content change to review and does not impact the package runtime.\r\n\r\nTowards OPE-589", "merged_at": "2024-10-13T22:46:42Z"}
{"number": 667, "title": "Add dataset submodules", "files": ["docs/api/oumi.datasets.pretraining.rst", "docs/api/oumi.datasets.vision_language.rst", "src/oumi/datasets/pretraining/__init__.py", "src/oumi/datasets/vision_language/__init__.py"], "area": "docs", "area_votes": {"docs": 2}, "body": "**Changes**\r\n- Subfolders for `pretraining` and `vision_language` datasets are now modules.\r\n- The individual datasets can still be import from `oumi.datasets`, this change is mostly to have them visible in the docs page.\r\n- Add apidoc for each module.\r\n\r\nTowards OPE-589", "merged_at": "2024-10-14T15:29:49Z"}
{"number": 666, "title": "[docs][tiny] fix minor doc typos", "files": ["docs/about/citations.md", "docs/conf.py", "docs/development/contributing.md", "docs/get_started/installation.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "**Changes**\r\n- Fix a typo, broken link, and citations rendering\r\n\r\nNote: Merged directly to unblock. This PR has no doc content change to review and does not impact the package runtime.\r\n\r\nTowards OPE-589", "merged_at": "2024-10-13T22:21:15Z"}
{"number": 664, "title": "Update notebooks to include a descriptive title", "files": ["notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Datasets Tutorial.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Multinode Inference on Polaris.ipynb", "notebooks/Oumi - Oumi Judge.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Tuning Llama.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "src/experimental/Oumi - Build Zephyr 7B.ipynb", "src/experimental/Oumi - Colab Setup Example.ipynb"], "area": "docs", "area_votes": {"docs": 13}, "body": "**Changes**\r\n- Update notebooks to include a `H1` title. This is needed so that they render properly in the sphinx documentation\r\n- Move notebooks that are meant of an internal audience to the experimental folder to avoid clutter in the public examples\r\n\r\nTowards OPE-589", "merged_at": "2024-10-14T15:51:19Z"}
{"number": 663, "title": "[Inference] Bug in generation config stop tokens", "files": ["src/oumi/core/configs/params/generation_params.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/inference/test_generation_params.py"], "area": "inference", "area_votes": {"configs": 1, "inference": 5}, "body": "- In `NativeTextInferenceEngine`, we generate a `transformers.GenerationConfig` from our `generation_params`, where we pass as \"stop\" argument the token(s) to terminate the sequence. However, there is no `stop` argument in the `transformers.GenerationConfig`'s `__init__`; to the best of my understanding this argument should be a no-op.\r\n- Since the generation does not have a stop token, it picks up the default one from Meta. In the model that I am currently investigating (trained from base), this defaults to `128001`; generation thinks our model is the base model.\r\n- The right way to determine the stop token is by `stop_strings` or `eos_token_id`. \r\n- We also need to pass in the tokenizer in `model.generate` kwargs (as `tokenizer`); this is a requirement if we are adjusting the stop tokens.\r\n- Future work (P1): Offer users the option to pick up the generation config from the model checkpoint (i.e., `generation_config.json`) and merge it with the `generation_params` defined in our config.", "merged_at": "2024-10-15T11:48:22Z"}
{"number": 662, "title": "Fix broken link in readme (dev_setup)", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2024-10-13T19:22:48Z"}
{"number": 661, "title": "[docs] fix broken links", "files": ["docs/_doclinks.config", "docs/development/contributing.md", "docs/development/style_guide.md", "docs/get_started/configuration.md", "docs/index.rst", "docs/user_guides/train/distributed_training.md", "src/oumi/core/configs/__init__.py"], "area": "docs", "area_votes": {"docs": 6}, "body": "**Changes**\r\n- Fix misc broken links in the docs\r\n\r\nTowards OPE-589\r\n\r\n**Note**: Merged directly to unblock. This PR has no doc content change to review and does not impact the package runtime.", "merged_at": "2024-10-13T02:08:19Z"}
{"number": 660, "title": "[Docs] fix rendering issues", "files": ["docs/citations.bib", "docs/conf.py", "docs/index.rst", "src/oumi/core/configs/inference_config.py", "src/oumi/datasets/pretraining/dolma.py", "src/oumi/datasets/pretraining/fineweb_edu.py", "src/oumi/datasets/pretraining/pile.py", "src/oumi/datasets/pretraining/red_pajama_v1.py", "src/oumi/datasets/pretraining/red_pajama_v2.py", "src/oumi/datasets/pretraining/slim_pajama.py", "src/oumi/datasets/pretraining/starcoder.py", "src/oumi/datasets/pretraining/the_stack.py", "src/oumi/datasets/pretraining/tiny_stories.py", "src/oumi/datasets/pretraining/tiny_textbooks.py", "src/oumi/datasets/pretraining/wikipedia.py", "src/oumi/datasets/pretraining/wikitext.py", "src/oumi/datasets/pretraining/youtube_commons.py"], "area": "data", "area_votes": {"docs": 3, "configs": 1, "data": 13}, "body": "**Changes**\r\n- Fix misc rendering issues\r\n- Minor content changes\r\n\r\nTowards OPE-589\r\n\r\n**Note**: Merged directly to unblock. This PR has no doc content change to review and does not impact the package runtime.", "merged_at": "2024-10-13T01:11:47Z"}
{"number": 658, "title": "[Docs] Add documentation placeholders", "files": ["CONTRIBUTING.md", "docs/_doclinks.config", "docs/about/acknowledgements.md", "docs/about/changelog.md", "docs/about/license.md", "docs/advanced/custom_datasets.md", "docs/advanced/custom_models.md", "docs/advanced/custom_vl_datasets.md", "docs/advanced/performance_optimization.md", "docs/advanced/quantization.md", "docs/cli/commands.md", "docs/conf.py", "docs/datasets/local_datasets.md", "docs/datasets/preference_tuning.md", "docs/datasets/pretraining.md", "docs/datasets/sft.md", "docs/datasets/vl_sft.md", "docs/development/code_of_conduct.md", "docs/development/contributing.md", "docs/development/dev_setup.md", "docs/development/style_guide.md", "docs/faq/gpu_sizing.md", "docs/faq/oom.md", "docs/faq/troubleshooting.md", "docs/get_started/configuration.md", "docs/get_started/installation.md", "docs/get_started/quickstart.md", "docs/get_started/tutorials.md", "docs/get_started/usage.md", "docs/index.rst", "docs/models/cambrian.md", "docs/models/recipes.md", "docs/user_guides/evaluate/evaluation.md", "docs/user_guides/evaluate/lm_harness.md", "docs/user_guides/infer/inference.md", "docs/user_guides/infer/inference_engines.md", "docs/user_guides/judge/custom_judge.md", "docs/user_guides/judge/judge.md", "docs/user_guides/judge/oumi_judge.md", "docs/user_guides/launch/launch.md", "docs/user_guides/train/distributed_training.md", "docs/user_guides/train/trainers.md", "docs/user_guides/train/training.md", "docs/user_guides/train/training_config.md"], "area": "docs", "area_votes": {"docs": 44}, "body": "**Changes**\r\n- Add placeholder sections for all doc pages\r\n- Minor content changes, to be reviewed later\r\n\r\n**Note**: Merged directly to unblock. This PR has no doc content change to review and does not impact the package runtime.\r\n\r\nTowards OPE-589\r\n", "merged_at": "2024-10-12T21:52:03Z"}
{"number": 655, "title": "[Docs] Refactor docs pipeline", "files": ["Makefile", "docs/_doclinks.config", "docs/_manage_doclinks.py", "docs/about/acknowledgements.md", "docs/about/changelog.md", "docs/about/license.md", "docs/advanced/custom_models.md", "docs/advanced/new_multimodal_dataset.md", "docs/cli/commands.md", "docs/conf.py", "docs/development/code_of_conduct.md", "docs/development/dev_setup.md", "docs/development/git_workflow.md", "docs/development/style_guide.md", "docs/examples/tutorials.md", "docs/faq/gpu_sizing.md", "docs/faq/oom.md", "docs/faq/troubleshooting.md", "docs/get_started/configuration.md", "docs/get_started/installation.md", "docs/index.rst", "docs/links/cloud_training.rst", "docs/links/contributing.rst", "docs/links/dev_setup.rst", "docs/links/distributed_training.rst", "docs/links/new_multimodal_dataset.rst", "docs/links/usage.rst", "docs/tutorials.rst", "docs/user_guides/distributed_training.md", "docs/user_guides/launch.md"], "area": "docs", "area_votes": {"infra": 1, "docs": 29}, "body": "**Changes**\r\n- Update index.rst with the new outline (pages to be filled later)\r\n- Add a script to manage documentation links. \r\n  - Sphinx requires all documentation pages to be under the `docs/` folder, but we have many pages that are hosted elsewhere (e.g. `CONTRIBUTING.md` in root, all notebooks under `notebooks`).\r\n  - the `_manage_doclinks` script will copy files to the right location when generating the docs \r\n- Update inter sphinx refs to latest hf index\r\n- Remove deprecated manual rst link files \r\n\r\n**Note**: Merged directly to unblock. This PR has no doc content change to review and does not impact the package runtime.\r\n\r\nTowards OPE-589", "merged_at": "2024-10-12T16:45:30Z"}
{"number": 654, "title": "Update sphinx docs", "files": [".gitignore", "Makefile", "docs/.sphinx/Makefile", "docs/.sphinx/make.bat", "docs/_templates/api/package.rst.jinja", "docs/api/modules.rst", "docs/api/oumi.builders.rst", "docs/api/oumi.core.callbacks.rst", "docs/api/oumi.core.cli.rst", "docs/api/oumi.core.configs.params.rst", "docs/api/oumi.core.configs.rst", "docs/api/oumi.core.datasets.rst", "docs/api/oumi.core.inference.rst", "docs/api/oumi.core.launcher.rst", "docs/api/oumi.core.models.rst", "docs/api/oumi.core.rst", "docs/api/oumi.core.tokenizers.rst", "docs/api/oumi.core.trainers.rst", "docs/api/oumi.core.types.rst", "docs/api/oumi.datasets.pretraining.rst", "docs/api/oumi.datasets.rst", "docs/api/oumi.datasets.vision_language.rst", "docs/api/oumi.evaluation.rst", "docs/api/oumi.inference.rst", "docs/api/oumi.judges.rst", "docs/api/oumi.launcher.clients.rst", "docs/api/oumi.launcher.clouds.rst", "docs/api/oumi.launcher.clusters.rst", "docs/api/oumi.launcher.rst", "docs/api/oumi.models.experimental.cambrian.model.language_model.phi3.rst", "docs/api/oumi.models.experimental.cambrian.model.language_model.rst", "docs/api/oumi.models.experimental.cambrian.model.multimodal_encoder.rst", "docs/api/oumi.models.experimental.cambrian.model.multimodal_projector.rst", "docs/api/oumi.models.experimental.cambrian.model.rst", "docs/api/oumi.models.experimental.cambrian.rst", "docs/api/oumi.models.experimental.rst", "docs/api/oumi.models.rst", "docs/api/oumi.performance.rst", "docs/api/oumi.rst", "docs/api/oumi.utils.rst", "docs/conf.py", "docs/index.rst", "docs/links/cloud_training.rst", "docs/links/contributing.rst", "docs/links/dev_setup.rst", "docs/links/distributed_training.rst", "docs/links/new_multimodal_dataset.rst", "docs/links/usage.rst", "docs/tutorials.rst"], "area": "docs", "area_votes": {"infra": 2, "docs": 47}, "body": "**Changes**\r\n- In preparation of our docs' overhaul:\r\n  - Move everything under `docs` instead of `docs/.sphinx`\r\n  - Update makefile to match\r\n  - Move api reference from `apidoc` -> `api`\r\n  - Add docs for newly introduced models (judge, etc) \r\n\r\nTowards OPE-589", "merged_at": "2024-10-12T04:04:50Z"}
{"number": 651, "title": "Add a description to the Launch CLI", "files": ["src/oumi/core/cli/main.py"], "area": "other", "area_votes": {"other": 1}, "body": "Before:\r\n![Screenshot 2024-10-11 at 11 30 30\u202fAM](https://github.com/user-attachments/assets/fa9dd49d-cf8d-44da-bf63-2c4e64807b8b)\r\n\r\nAfter:\r\n![Screenshot 2024-10-11 at 11 30 17\u202fAM](https://github.com/user-attachments/assets/ca3e415f-4819-4524-a1a3-d50483c37a1a)\r\n", "merged_at": "2024-10-11T21:30:13Z"}
{"number": 648, "title": "Add support for `add_generation_prompt` in LLAVA chat template", "files": ["src/oumi/datasets/chat_templates/llava.jinja"], "area": "data", "area_votes": {"data": 1}, "body": "-- Leads to better infer responses\r\n-- Consistent with the original LLAVA template: https://linear.app/oumi/issue/OPE-561#comment-ddfde552\r\n\r\nTowards OPE-401", "merged_at": "2024-10-11T15:42:30Z"}
{"number": 646, "title": "Enable dataloaders for VLLM-s (llava and blip2)", "files": ["configs/oumi/blip2_opt_2.7b.sft.yaml", "configs/oumi/llava.7b.sft.yaml", "configs/skypilot/sky_blip2_opt_2.7b.sft.yaml", "configs/skypilot/sky_llava.7b.sft.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "-- offloading image data loading to workers helps a lot with GPU utilization for `TRL_SFT` (now close to 100% , previously very uneven ~50%)\r\n-- and tune few other params \r\n\r\nTowards OPE-467, OPE-551", "merged_at": "2024-10-10T22:46:33Z"}
{"number": 642, "title": "Tune few training params for LLAVA and blip2 models", "files": ["configs/oumi/blip2_opt_2.7b.sft.yaml", "configs/oumi/llava.7b.sft.yaml", "configs/skypilot/sky_llava.7b.sft.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "-- Tune batch size, steps, and few other params \r\n-- For LLAVA using `bs=8,gas=8` to mimic this example: https://github.com/huggingface/trl/blob/b3f93f0bad85c808ea76ceb2c07706794a31e74f/examples/scripts/sft_vlm.py#L23\r\n\r\nTowards OPE-467, OPE-551", "merged_at": "2024-10-10T20:07:01Z"}
{"number": 641, "title": "Add popular pre-training dataset classes", "files": ["src/oumi/core/datasets/iterable_dataset.py", "src/oumi/datasets/__init__.py", "src/oumi/datasets/pretraining/c4.py", "src/oumi/datasets/pretraining/dolma.py", "src/oumi/datasets/pretraining/falcon_refinedweb.py", "src/oumi/datasets/pretraining/fineweb_edu.py", "src/oumi/datasets/pretraining/pile.py", "src/oumi/datasets/pretraining/red_pajama_v1.py", "src/oumi/datasets/pretraining/red_pajama_v2.py", "src/oumi/datasets/pretraining/slim_pajama.py", "src/oumi/datasets/pretraining/starcoder.py", "src/oumi/datasets/pretraining/the_stack.py", "src/oumi/datasets/pretraining/tiny_stories.py", "src/oumi/datasets/pretraining/tiny_textbooks.py", "src/oumi/datasets/pretraining/wikipedia.py", "src/oumi/datasets/pretraining/wikitext.py", "src/oumi/datasets/pretraining/youtube_commons.py", "tests/integration/datasets/test_pretraining_datasets_full_epoch.py", "tests/integration/datasets/test_sft_datasets_full_epoch.py", "tests/integration/datasets/test_sft_datasets_load_datasets.py"], "area": "data", "area_votes": {"data": 16}, "body": "**Changes**\r\n- Add a list of 15 most popular pre-training datasets.\r\n- All have the same structure, the created classes are just wrappers to enable having documentation and integration tests\r\n\r\n**Note**\r\n- Testing all the datasets takes time, and still working on it. About ~20% done\r\n\r\nCloses OPE-434, OPE-433", "merged_at": "2024-10-10T20:51:16Z"}
{"number": 637, "title": "Add all Llama FSDP GCP configs", "files": ["configs/oumi/jobs/gcp/llama3b_lora_fsdp.yaml", "configs/oumi/jobs/gcp/llama3b_qlora_fsdp.yaml", "configs/oumi/jobs/gcp/llama3b_sft_fsdp.yaml", "configs/oumi/jobs/gcp/llama70b_lora_fsdp.yaml", "configs/oumi/jobs/gcp/llama70b_qlora_fsdp.yaml", "configs/oumi/jobs/gcp/llama70b_sft_fsdp.yaml", "configs/oumi/jobs/gcp/llama8b_lora.yaml", "configs/oumi/jobs/gcp/llama8b_lora_fsdp.yaml", "configs/oumi/jobs/gcp/llama8b_qlora_fsdp.yaml", "configs/oumi/jobs/gcp/llama8b_sft_fsdp.yaml", "configs/oumi/llama70b.qlora.yaml", "configs/oumi/llama8b.qlora.yaml", "src/oumi/core/configs/params/peft_params.py"], "area": "configs", "area_votes": {"configs": 13}, "body": "Towards OPE-574\r\n\r\nThis is for all training modes (FFT/Lora/QLora) and all model sizes (3B/8B/70B). Renamed existing configs that did FSDP training (ex. Llama 70B Lora) to include FSDP in the name for clarity.\r\n\r\nNote that the QLora configs currently don't work due to OPE-577. This will be fixed in a follow-up PR.", "merged_at": "2024-10-12T02:30:53Z"}
{"number": 635, "title": "Add distribution mode flag to llama_tune", "files": [".vscode/launch.json", "configs/accelerate/llama.fsdp.mixedprec.yaml", "configs/accelerate/llama.fsdp.yaml", "configs/accelerate/llama70b.fsdp.yaml", "configs/accelerate/llama70b.lora.yaml", "configs/oumi/jobs/gcp/llama2b_fsdp_trl.yaml", "configs/oumi/jobs/gcp/llama8b_sft.yaml", "configs/oumi/jobs/polaris/llama3b_lora.yaml", "configs/oumi/jobs/polaris/llama3b_qlora.yaml", "configs/oumi/jobs/polaris/llama3b_sft.yaml", "configs/oumi/jobs/polaris/llama70b_lora.yaml", "configs/oumi/jobs/polaris/llama70b_sft.yaml", "configs/oumi/jobs/polaris/llama8b_lora.yaml", "configs/oumi/jobs/polaris/llama8b_sft.yaml", "configs/oumi/llama2b.pt.fsdp.trl.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_llama8b_sft.yaml", "notebooks/Oumi - Running Jobs Remotely.ipynb", "scripts/polaris/jobs/llama2b_pt_worker.sh", "scripts/polaris/jobs/llama_tune.sh"], "area": "configs", "area_votes": {"other": 1, "configs": 16, "docs": 1, "infra": 2}, "body": "Towards OPE-547\r\n\r\n- Rename `llama8b.fsdp.yaml` to `llama.fsdp.yaml` since it's also the config for llama 3B\r\n- Rename `llama.fsdp.yaml` to `llama.fsdp.mixedprec.yaml` to make clear the only difference is mixed precision\r\n- Add a distribution mode flag to `llama_tune.sh` so we can control whether we run ddp/fsdp\r\n\r\nTested existing configs on Polaris", "merged_at": "2024-10-10T04:15:21Z"}
{"number": 631, "title": "Add config example for long context fine-tuning", "files": ["configs/oumi/llama3.8b.sft.longctx.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "**Changes**\r\nThis PR adds a config optimized to fine-tune llama8b with 32K context.\r\n\r\n**Usage**\r\nThis was tested on `8xA100-40GB` using the latest `oumi` version.\r\n\r\n```bash\r\naccelerate launch \\\r\n    --config_file configs/accelerate/llama8b.fsdp.yaml \\\r\n    --use_fsdp \\\r\n    --num_processes 8 \\\r\n    --dynamo_backend inductor \\\r\n    --mixed_precision no \\\r\n    -m oumi.train \\\r\n    -c \"configs/oumi/llama3.8b.sft.longctx.yaml\"\r\n```", "merged_at": "2024-10-10T03:36:14Z"}
{"number": 630, "title": "Add `pip install -U uv;` to `make setup` for existing envs", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "-- Otherwise, it failed for my existing env (`uv` was not previously installed).\r\n-- Overhead should be minimal if `uv` already xists", "merged_at": "2024-10-09T23:08:46Z"}
{"number": 629, "title": "[tiny] Fix bug with Polaris job num", "files": ["scripts/polaris/jobs/llama_tune.sh", "scripts/polaris/polaris_init.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "Towards OPE-413\r\n\r\nWe need to export the `OUMI_JOBNUM` env var for it to be accessible in `llama_tune.sh` since it's a child process.\r\nTested this works on Polaris.", "merged_at": "2024-10-09T21:56:55Z"}
{"number": 628, "title": "[tiny] Add flops for T4 GPU", "files": ["src/oumi/performance/mfu.py"], "area": "training", "area_votes": {"training": 1}, "body": "Towards OPE-547\r\n\r\nNeeded to unblock Llama 3B SFT on Colab", "merged_at": "2024-10-09T20:54:44Z"}
{"number": 627, "title": "Use a timestamp for job directories on Polaris", "files": ["src/oumi/launcher/clusters/polaris_cluster.py", "tests/launcher/clusters/test_polaris_cluster.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Previously the Oumi Launcher used the `job.name` parameter as a directory when copying bits. I've updated this approach to use the current time stamp (including milliseconds).\r\n\r\nEx): `09102024_130424513094` of the form `ddMMYYYY_HHMMSSmmmmmm`\r\n\r\n\r\nFixes OPE-532", "merged_at": "2024-10-09T21:27:09Z"}
{"number": 626, "title": "Add Llama 3b sft/lora/qlora configs for Polaris", "files": ["configs/oumi/jobs/polaris/llama3b_lora.yaml", "configs/oumi/jobs/polaris/llama3b_qlora.yaml", "configs/oumi/jobs/polaris/llama3b_sft.yaml", "configs/oumi/llama70b.lora.yaml", "configs/oumi/llama8b.lora.yaml", "scripts/polaris/jobs/llama_tune.sh"], "area": "configs", "area_votes": {"configs": 5, "infra": 1}, "body": "Towards OPE-547\r\n\r\nThis expands the existing llama_tune.sh Polaris job. Later, we can add an FSDP/DDP option as well, along with 8B/70B QLora configs.\r\n\r\nThis also fixes the params for Llama 8B/70B Lora to be more in line with torchtune's.\r\n\r\nHad to reduce model max length for Llama 3B to 512 to prevent OOMs.\r\n\r\nTested all three runs work on Polaris.", "merged_at": "2024-10-09T19:14:36Z"}
{"number": 625, "title": "Add check if installation is successful", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "- Add check if installation is successful\r\n\r\nCloses OPE-567", "merged_at": "2024-10-09T19:26:35Z"}
{"number": 624, "title": "Update two VLLM configs.", "files": ["configs/oumi/blip2_opt_2.7b.sft.yaml", "configs/oumi/llava.7b.sft.yaml", "configs/skypilot/sky_blip2_opt_2.7b.sft.yaml", "configs/skypilot/sky_llava.7b.sft.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "-- Set `llava` chat template in both  (better than default) TODO: Define a better config for blip2\r\n-- Update configs to use `HuggingFaceH4/llava-instruct-mix-vsft`  dataset\r\n\r\nTowards OPE-467, OPE-545", "merged_at": "2024-10-09T22:19:01Z"}
{"number": 623, "title": "Judge Notebook 2: Custom Judge", "files": ["notebooks/Oumi - Custom Judge.ipynb", "notebooks/Oumi - Oumi Judge.ipynb"], "area": "docs", "area_votes": {"docs": 2}, "body": "", "merged_at": "2024-10-10T16:09:21Z"}
{"number": 622, "title": "Make sure conda env is registered", "files": ["Makefile", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "**Changes**\r\n- Make sure conda env is registered, so that it can be used by Jupyter lab directly", "merged_at": "2024-10-09T17:58:29Z"}
{"number": 621, "title": "Update several notebooks with the new EvaluationConfig format.", "files": ["notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Colab Setup Example.ipynb"], "area": "docs", "area_votes": {"docs": 2}, "body": "Update notebooks to reflect the recent changes to Evaluation.", "merged_at": "2024-10-09T17:31:21Z"}
{"number": 620, "title": "Add miniconda installation target, cleanup unused make commands", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "**Changes**\r\n- Add a make target to install Miniconda, as this is a common issue when onboarding users. Tested on both Mac (arm) and Linux (GCP)\r\n- Remove unused targets: train, eval, infer, as it's simpler to use the oumi launcher/\r\n\r\nTowards OPE-567, Closes OPE-327", "merged_at": "2024-10-09T17:23:02Z"}
{"number": 619, "title": "[tiny] alpaca - minor reproducibility boost", "files": ["src/oumi/datasets/alpaca.py"], "area": "data", "area_votes": {"data": 1}, "body": "Varies the system template as in the original/author's [codebase](https://github.com/tatsu-lab/stanford_alpaca/blob/main/train.py#L31). This should help (tiny) with reproducibility.\r\n ", "merged_at": "2024-10-09T20:39:08Z"}
{"number": 618, "title": "Turn off pretty-printing exceptions in our CLI", "files": ["src/oumi/core/cli/main.py"], "area": "other", "area_votes": {"other": 1}, "body": "Fixes OPE-563", "merged_at": "2024-10-09T15:30:23Z"}
{"number": 617, "title": "[tiny] add gpt2 chat template, and update tests to use it", "files": ["src/oumi/datasets/chat_templates/gpt2.jinja", "tests/integration/infer/test_infer.py", "tests/integration/infer/test_native_text_inference_engine.py"], "area": "data", "area_votes": {"data": 1}, "body": "**Changes**\r\n- Add a dummy template for gpt2. This template ignores all the roles and just concatenates all the inputs, and is similar to what was used in `transformers <4.45.0`\r\n- Update tests that relied on hardcoded gpt2 responses to use the template\r\n\r\nCloses OPE-115\r\nTowards OPE-490", "merged_at": "2024-10-09T04:01:14Z"}
{"number": 616, "title": "[tiny] Fix .gitignore", "files": [".gitignore"], "area": "infra", "area_votes": {"infra": 1}, "body": "Comments must be on their own line: https://git-scm.com/docs/gitignore#_pattern_format", "merged_at": "2024-10-09T03:01:52Z"}
{"number": 614, "title": "Update makefile to use uv, add Jupyter target", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "**Changes**\r\n- Update makefile to use uv to install dependencies (much faster than pip)\r\n- Add a Jupyter target, to make sure it's run in the expected environment \r\n\r\nTowards OPE-567", "merged_at": "2024-10-09T16:57:04Z"}
{"number": 612, "title": "Add support for the Launch command suite in the new CLI", "files": ["src/oumi/core/cli/launch.py", "src/oumi/core/cli/main.py", "tests/core/cli/test_cli_launch.py", "tests/core/cli/test_cli_main.py"], "area": "other", "area_votes": {"other": 2}, "body": "Example commands:\r\n\r\n`oumi launch status`\r\n`oumi launch which`\r\n`oumi launch up --config foo`\r\n`oumi launch run --config foo`\r\n`oumi launch down --cluster a --cloud b`\r\n`oumi launch stop --cluster a --cloud b --id c`\r\n\r\nTowards OPE-500 and OPE-473", "merged_at": "2024-10-08T23:44:55Z"}
{"number": 610, "title": "More robust make setup", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "**Changes**\r\n- Checks if conda is installed and available, and if not print an error message with instructions to install\r\n- Make sure conda is initialized for the shell the user is using\r\n- Reduce the number of installed dependencies from `[all]` to `[train,dev]`", "merged_at": "2024-10-08T23:25:02Z"}
{"number": 609, "title": "Fix `apply_chat_template` issue in `VisionLanguageSftDataset`", "files": ["src/oumi/core/datasets/vision_language_dataset.py", "src/oumi/datasets/chat_templates/llava.jinja"], "area": "data", "area_votes": {"data": 2}, "body": "-- Update LLAVA jinja template to match the latest `Message` structure \r\n-- Removed `image_placeholder` dict . Let' use `Message` everywhere for consistency.\r\n\r\nThis is a quick fix, a more robust longer term solution may be needed. \r\n\r\nTowards OPE-545\r\nFixes OPE-561", "merged_at": "2024-10-08T23:10:01Z"}
{"number": 607, "title": "explicitly specify the model's dtype in LMH", "files": ["src/oumi/core/configs/params/model_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "Update some basic parameters when utilizing LMH via Oumi:\r\n1.  Explicitly specify the torch.dtype of the neural model that will be instantiated with LMH. \r\n2. Enabling the loading of quantized model in 4 or 8 bits.\r\n\r\nTowards OPE-564.\r\n\r\nNote (future todo). We will revisit the interoperability between the two libraries. For instance, there might exist parameters in the [model_kwargs](https://github.com/oumi-ai/oumi/pull/607/files#diff-5d668620d9ee30efe51708c9716b83a3cfdf73c1760020fe689218e9a00aeacfR184\r\n) that are not relevant for LMH but are relevant for Oumi. Also, shortly we should directly pass the PeftConfig instead of using the ``load_in_4bit``, etc.", "merged_at": "2024-10-09T20:39:44Z"}
{"number": 605, "title": "Add Llama 3B configs", "files": ["configs/oumi/llama3b.lora.yaml", "configs/oumi/llama3b.qlora.yaml", "configs/oumi/llama3b.sft.yaml", "src/oumi/train.py"], "area": "configs", "area_votes": {"configs": 3, "training": 1}, "body": "Towards OPE-547\r\n\r\nWe can run these configs directly on mac, ex. `oumi-train -c configs/oumi/llama3b.sft.yaml \"training.optimizer=adamw_torch\" \"training.max_steps=10\" \"model.model_max_length=128\"`", "merged_at": "2024-10-08T20:47:47Z"}
{"number": 603, "title": "Set docstring for `NVidiaGpuRuntimeInfo` struct", "files": ["src/oumi/utils/device_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "-- belated PR feedback follow-up\r\n\r\nTowards OPE-369", "merged_at": "2024-10-08T16:40:11Z"}
{"number": 602, "title": "Add Llama train/eval/infer E2E integration test", "files": ["configs/oumi/llama1b.eval.yaml", "configs/oumi/llama1b.infer.yaml", "configs/oumi/llama1b.sft.yaml", "scripts/llama_e2e.py", "tests/integration/builders/test_data.py"], "area": "configs", "area_votes": {"configs": 3, "infra": 1}, "body": "Towards OPE-503\r\n\r\nTo run: `python scripts/llama_e2e.py`\r\n\r\nThis test confirms if train/eval/infer can successfully run, without testing quality. This is intended to be run as a script. We can make it more advanced afterwards if desired.", "merged_at": "2024-10-08T05:17:18Z"}
{"number": 601, "title": "Update accelerate version to 1.0.0", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Fixes OPE-496\r\n\r\n1.0.0 just released today. I tested that it still solves the FSDP model saving option, but should fix the issue Jeremy encountered with AdamW: https://linear.app/oumi/issue/OPE-496#comment-6b7c2880", "merged_at": "2024-10-07T22:39:06Z"}
{"number": 600, "title": "Add generation params to inference engines", "files": ["src/oumi/core/configs/params/generation_params.py", "src/oumi/core/configs/params/remote_params.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/inference/test_anthropic_inference_engine.py", "tests/inference/test_generation_params.py", "tests/integration/infer/test_infer.py", "tests/integration/infer/test_native_text_inference_engine.py"], "area": "inference", "area_votes": {"configs": 2, "inference": 5}, "body": "# Add Generation Parameters to Inference Engines\r\n\r\nThis PR introduces a set of missing generation parameters, and updates all the current inference engines to support them, or if not supported by the engine, logs a warning that the parameter will be ignored. \r\n\r\nThe following parameters are added: temperature, top_p, frequency_penalty, presence_penalty, stop sequences, logit_bias, and min_p.\r\n\r\nTowards OPE-328\r\n\r\n## Changes\r\n\r\n- Updated `GenerationParams` class with new parameters\r\n- Implemented parameter support in AnthropicInferenceEngine, LlamaCppInferenceEngine, NativeTextInferenceEngine, RemoteInferenceEngine, and VLLMInferenceEngine\r\n\r\n\r\n## Usage\r\n\r\n### Setting Generation Parameters\r\n\r\n```python\r\nfrom oumi.core.configs.params.generation_params import GenerationParams\r\n\r\nparams = GenerationParams(\r\n    max_new_tokens=100,\r\n    temperature=0.7,\r\n    top_p=0.9,\r\n    frequency_penalty=0.1,\r\n    presence_penalty=0.1,\r\n    stop=[\"END\"],\r\n    logit_bias={50256: -100},  # Decrease likelihood of EOS token\r\n    min_p=0.05\r\n)\r\n```\r\n\r\n### Using Parameters with an Inference Engine\r\n\r\n```python\r\nfrom oumi.inference.llama_cpp_inference_engine import LlamaCppInferenceEngine\r\n\r\nengine = LlamaCppInferenceEngine(model_params, generation_params=params)\r\nresponse = engine.generate(conversation)\r\n```\r\n", "merged_at": "2024-10-08T16:52:19Z"}
{"number": 598, "title": "Fix cli infer test", "files": ["src/oumi/core/cli/infer.py", "tests/core/cli/test_cli_infer.py"], "area": "other", "area_votes": {"other": 1}, "body": "For PR-588", "merged_at": "2024-10-07T18:36:26Z"}
{"number": 597, "title": "Refactor Aya & Ultrachat to use oumi dataset sft classes", "files": ["src/oumi/builders/data.py", "src/oumi/datasets/aya.py", "src/oumi/datasets/common.py", "src/oumi/datasets/prompt_response_sft_preprocessor_factory.py", "src/oumi/datasets/ultrachat.py", "src/oumi/datasets/ultrachat_200k.py"], "area": "data", "area_votes": {"other": 1, "data": 5}, "body": "- Refactor Aya & Ultrachat dataset to use dataset classes instead of pre-processing functions\r\n\r\nCloses OPE-272", "merged_at": "2024-10-07T23:27:50Z"}
{"number": 596, "title": "Remove deprecated dataset code paths", "files": ["src/oumi/builders/data.py", "src/oumi/datasets/alpaca.py", "src/oumi/datasets/chatqa.py", "src/oumi/datasets/chatrag_bench.py"], "area": "data", "area_votes": {"other": 1, "data": 3}, "body": "- Remove deprecated dataset preprocessing functions for alpaca, chatqa\r\n- All dataset loading & preprocessing is done as part of their respective dataset class\r\n\r\nTowards OPE-272", "merged_at": "2024-10-07T23:12:31Z"}
{"number": 595, "title": "Update handling of GPU fan speed info", "files": ["src/oumi/utils/device_utils.py", "tests/utils/test_device_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "-- The function `nvmlDeviceGetFanSpeed()` fails on many systems (fails on GCP, Polaris but works on my local WSL machine w/ GPU) => Downgrade this warning to `logging.debug()` to reduce logs noise\r\n-- Update unit tests to allow `None`-s\r\n\r\nTowards OPE-369", "merged_at": "2024-10-07T17:29:11Z"}
{"number": 594, "title": "Add support for magpie dataset variants", "files": ["src/oumi/datasets/__init__.py", "src/oumi/datasets/magpie.py", "tests/integration/datasets/test_datasets_full_epoch.py", "tests/integration/datasets/test_dolly_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "- Add support for magpie dataset variants\r\n\r\nCloses OPE-431", "merged_at": "2024-10-07T17:50:25Z"}
{"number": 593, "title": "Judge Notebook 1: default judge", "files": ["notebooks/Oumi - Judge (Oumi): filtering a dataset.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2024-10-07T19:35:41Z"}
{"number": 591, "title": "Misc updates to Polaris launcher scripts", "files": ["scripts/polaris/jobs/llama2b_pt_worker.sh", "scripts/polaris/jobs/llama_tune.sh", "scripts/polaris/polaris_init.sh"], "area": "infra", "area_votes": {"infra": 3}, "body": "-- Add `-t` (telemetry) option to `scripts/polaris/jobs/llama_tune.sh`\r\n--  Move `OUMI_JOBNUM` computation to `polaris_init.sh` for reusability\r\n-- Define `$SHARED_TRAINING_PARAMS` in `llama_tune.sh`\r\n\r\nTowards OPE-413", "merged_at": "2024-10-07T16:54:33Z"}
{"number": 589, "title": "Add text jsonlines dataset class", "files": ["src/oumi/datasets/sft_jsonlines.py", "src/oumi/datasets/vision_language/vision_jsonlines.py", "tests/datasets/test_text_jsonlines_dataset.py", "tests/datasets/test_vision_language_jsonlines_dataset.py"], "area": "data", "area_votes": {"data": 2}, "body": "# Add TextSftJsonLinesDataset for text-based SFT data\r\n\r\n- This PR introduces a new `TextSftJsonLinesDataset` class for handling text-based Supervised Fine-Tuning (SFT) data in JSON Lines format. This is is similar to the `VisionLanguageJsonLinesDataset`. \r\n- They re-use the same logic, and we could consider refactoring as a mix-in. For now it was simpler to just duplicate that logic.\r\n- Also add more robust type validation to `VisionLanguageJsonLinesDataset`\r\n\r\nCloses OPE-427\r\n\r\n## Usage\r\n\r\nHere's how to use the new `TextSftJsonLinesDataset`:\r\n\r\n```python\r\nfrom oumi.datasets.sft_jsonlines import TextSftJsonLinesDataset\r\n\r\n# Load from a JSONL file\r\ndataset = TextSftJsonLinesDataset(dataset_path=\"path/to/your/data.jsonl\")\r\n\r\n# Or load from a list of dictionaries\r\ndata = [\r\n    {\r\n        \"messages\": [\r\n            {\"role\": \"user\", \"content\": \"Hello, how are you?\"},\r\n            {\"role\": \"assistant\", \"content\": \"I'm doing well, thank you! How can I assist you today?\"},\r\n            {\"role\": \"user\", \"content\": \"Can you explain what machine learning is?\"},\r\n            {\"role\": \"assistant\", \"content\": \"Certainly! Machine learning is a branch of artificial intelligence...\"},\r\n        ]\r\n    }\r\n]\r\ndataset = TextSftJsonLinesDataset(data=data)\r\n\r\n# Access conversations\r\nconversation = dataset.conversation(0)\r\nprint(conversation.messages[0].content)  # Output: \"Hello, how are you?\"\r\n```", "merged_at": "2024-10-05T01:41:53Z"}
{"number": 588, "title": "Set up a new version of the Oumi CLI using Typer", "files": ["pyproject.toml", "src/oumi/core/cli/cli_utils.py", "src/oumi/core/cli/evaluate.py", "src/oumi/core/cli/infer.py", "src/oumi/core/cli/main.py", "src/oumi/core/cli/train.py", "src/oumi/infer.py", "tests/core/cli/test_cli_evaluate.py", "tests/core/cli/test_cli_infer.py", "tests/core/cli/test_cli_main.py", "tests/core/cli/test_cli_train.py", "tests/core/cli/test_cli_utils.py"], "area": "other", "area_votes": {"infra": 1, "other": 5, "inference": 1}, "body": "This PR creates a new `oumi` command that accepts 3 subcommands:\r\n`oumi train`\r\n`oumi infer`\r\n`oumi evaluate`\r\n\r\nThese commands have full typer integration.\r\n\r\nI will be adding `oumi launch` as a follow-up as it's more involved.\r\nI will remove the old CLI when we have parity + update all references. \r\n\r\n\r\nTowards OPE-500 and OPE-473", "merged_at": "2024-10-07T17:00:13Z"}
{"number": 586, "title": "Add dataset class for dolly dataset", "files": ["src/oumi/datasets/__init__.py", "src/oumi/datasets/dolly.py", "tests/integration/datasets/test_dolly_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "- Add dataset class for dolly dataset\r\n\r\nCloses OPE-430", "merged_at": "2024-10-05T00:20:17Z"}
{"number": 584, "title": "Add `@override` annotations to methods of few Dataset subclasses", "files": ["src/oumi/core/datasets/iterable_dataset.py", "src/oumi/core/datasets/vision_language_dataset.py", "tests/core/datasets/test_vision_language_dataset.py"], "area": "data", "area_votes": {"data": 2}, "body": "-- For extra code clarity \r\n\r\nTowards OPE-353", "merged_at": "2024-10-04T18:54:15Z"}
{"number": 583, "title": "Update `ProfilerParams` docstrings to follow the new style", "files": ["src/oumi/core/configs/params/profiler_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- To simplify, and, for consistency, with all other params\r\n\r\nFixes OPE-136", "merged_at": "2024-10-04T18:08:02Z"}
{"number": 582, "title": "Remove AWS as a default dep", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Make AWS an optional installation.\r\n\r\nTowards OPE-530", "merged_at": "2024-10-04T17:25:54Z"}
{"number": 581, "title": "Define `DataCollationParams`", "files": ["configs/oumi/blip2_opt_2.7b.sft.yaml", "configs/oumi/llava.7b.sft.yaml", "src/oumi/core/configs/params/data_params.py"], "area": "configs", "area_votes": {"configs": 3}, "body": "-- This is needed to enable `visual_language` collator for multi-modal , but may have other applications as well\r\n-- Some collation options: https://huggingface.co/docs/transformers/en/main_classes/data_collator\r\n-- This PR only defines configs (there are options how to define this), it's not wired up in training loop yet.\r\n\r\nTowards OPE-353", "merged_at": "2024-10-05T01:56:29Z"}
{"number": 580, "title": "Set accelerate version to fix FSDP model saving", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Fixes OPE-496\r\n\r\nWould recommend everyone to rerun pip-install after pulling this change to resolve the issue on their machines.\r\n\r\nCurrent version is 0.34.2, which doesn't properly unwrap the FSDP model for saving (specifically the embedding and LM head). Reverting back to 0.33.0, which doesn't have this issue. 0.34.0 doesn't work because it has a different unrelated error.\r\n\r\nThe pre-release 1.0.0rc0 version fixes this, but we may not want to depend on this in our codebase. However, this likely means Accelerate fixed the issue, so we don't need to flag this to them.\r\n\r\nTested this on GCP with `oumi-launch -p configs/oumi/jobs/gcp/llama8b_sft.yaml -c fix`, and successfully loaded the trained model.", "merged_at": "2024-10-03T22:59:52Z"}
{"number": 579, "title": "Disable `sdpa` for `blip2`", "files": ["configs/oumi/blip2_opt_2.7b.sft.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "Error:  \r\n\r\n```\r\nValueError: Blip2ForConditionalGeneration does not support an attention implementation through torch.nn.functional.scaled_dot_product_attention yet. Please request the support for this architecture: https://github.com/huggingface/transformers/issues/28005. If you believe this error is a bug, please open an issue in Transformers GitHub repository and load your model with the argument `attn_implementation=\"eager\"` meanwhile. Example: `model = AutoModel.from_pretrained(\"openai/whisper-tiny\", attn_implementation=\"eager\")`\r\n```\r\n\r\nEnabled `sdpa` at the last moment yesterday, but it doesn't actually work for this model.\r\n\r\n https://github.com/huggingface/transformers/issues/28005\r\n\r\nTowards OPE-353\r\nFixes OPE-233", "merged_at": "2024-10-03T20:21:44Z"}
{"number": 576, "title": "Misc cleanups in `JsonlinesDataset`", "files": ["src/oumi/datasets/vision_language/vision_jsonlines.py", "tests/datasets/test_vision_language_jsonlines_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "-- Switch to `pathlib`\r\n-- Error message improvements\r\n-- Increase unit test coverage\r\n\r\nTowards OPE-353", "merged_at": "2024-10-03T18:13:18Z"}
{"number": 575, "title": "Split out cloud dependencies", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Make a unique build target for each cloud.", "merged_at": "2024-10-03T19:33:50Z"}
{"number": 574, "title": "Inference Engine async writes", "files": ["scripts/inference/gcp_inference.py", "src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/llama_cpp_inference_engine.py", "src/oumi/inference/native_text_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "src/oumi/inference/vllm_inference_engine.py", "tests/inference/test_llama_cpp_inference_engine.py", "tests/inference/test_remote_inference_engine.py", "tests/inference/test_vllm_inference_engine.py"], "area": "inference", "area_votes": {"infra": 1, "inference": 5}, "body": "This PR is based on https://github.com/oumi-ai/oumi/pull/558\r\n\r\nThis PR updates how writes are done for inference.\r\n\r\n- If all requests are successful, the final written file will have all responses in the same order as the provided input for all engines.\r\n- During inference, all requests are written to a `/scratch` directory containing a file with the same name. There is no guarantee of order on the values written to this file (this truly only matters for the InferenceEngines that leverage parallelism).\r\n\r\nNon-parallel engines will write to disk in-line (blocking). I've benchmarked this for medium sized files (100s of MB): appending a line of text to these files takes on average 1.788e-05 seconds", "merged_at": "2024-10-03T16:14:38Z"}
{"number": 572, "title": "Fix issues with package build pipeline", "files": [".github/workflows/gcp_artifacts.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Fix misc issues with the config\r\n\r\nTowards OPE-455", "merged_at": "2024-10-02T23:43:18Z"}
{"number": 571, "title": "Update model builder to use `default_chat_template` if available", "files": ["src/oumi/builders/models.py"], "area": "other", "area_votes": {"other": 1}, "body": "-- For some models e.g.,  `Salesforce/blip2-opt-2.7b`, `tokenizer.default_chat_template` is provided while `tokenizer.chat_template` is not.\r\n\r\nTowards OPE-353", "merged_at": "2024-10-02T21:53:32Z"}
{"number": 570, "title": "Add package build and deployment workflow to google artifact registry", "files": [".github/workflows/gcp_artifacts.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "**Changes**\r\n- Add package build and deployment workflow to google artifact registry\r\n\r\nTowards OPE-455\r\n", "merged_at": "2024-10-02T22:00:39Z"}
{"number": 569, "title": "Configure freeze_layer map in `minimal_multimodal_training.py`", "files": ["scripts/benchmarks/minimal_multimodal_training.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "Towards OPE-353, OPE-505", "merged_at": "2024-10-02T17:32:27Z"}
{"number": 568, "title": "Clean up legacy evaluate_oumi code paths", "files": ["USAGE.md", "configs/oumi/gpt2.asynceval.nvidia.yaml", "configs/oumi/llama3.8b.aya.eval.yaml", "configs/oumi/llama70b.eval.yaml", "configs/oumi/llama8b.eval.yaml", "configs/oumi/phi3.eval.lm_harness.yaml", "configs/oumi/phi3.eval.oumi.mac.yaml", "configs/oumi/phi3.eval.oumi.nvidia.yaml", "configs/oumi/zephyr.7b/sft/eval_mmlu.yaml", "notebooks/Oumi - Finetuning Tutorial.ipynb", "src/oumi/__init__.py", "src/oumi/core/configs/__init__.py", "src/oumi/core/configs/evaluation_config.py", "src/oumi/core/configs/params/evaluation_params.py", "src/oumi/evaluate.py", "src/oumi/evaluation/huggingface_leaderboard.py", "tests/integration/evaluate/test_evaluate.py", "tests/integration/evaluate/test_evaluate_async.py"], "area": "configs", "area_votes": {"docs": 2, "configs": 10, "evaluation": 2}, "body": "This PR comes in two parts:\r\n\r\n1) Clean up evaluate_oumi\r\n\r\n2) Update our evaluation config to have a section explicitly for lm_harness as we don't support evaluations on custom datasets currently.\r\n\r\nAs a followup we should:\r\n- Add support for custom (oumi) metrics like bleu and rouge. \r\n- When we have support for custom datasets, let users specify the dataset directly as well\r\n\r\nTowards OPE-418", "merged_at": "2024-10-02T19:14:29Z"}
{"number": 566, "title": "Update DEV_SETUP.md with Windows instructions", "files": ["docs/DEV_SETUP.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Point users to WSL for installation. Add instructions for Conda installation.", "merged_at": "2024-10-01T23:28:47Z"}
{"number": 557, "title": "Initial Cambrian integration", "files": [".pre-commit-config.yaml", "pyproject.toml", "src/oumi/builders/models.py", "src/oumi/models/experimental/cambrian/__init__.py", "src/oumi/models/experimental/cambrian/constants.py", "src/oumi/models/experimental/cambrian/mm_utils.py", "src/oumi/models/experimental/cambrian/model/__init__.py", "src/oumi/models/experimental/cambrian/model/builder.py", "src/oumi/models/experimental/cambrian/model/cambrian_arch.py", "src/oumi/models/experimental/cambrian/model/language_model/cambrian_llama.py", "src/oumi/models/experimental/cambrian/model/language_model/cambrian_phi3.py", "src/oumi/models/experimental/cambrian/model/language_model/phi3/__init__.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/__init__.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/base_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/builder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/clip_convnext_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/clip_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/dino_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/load.py", "src/oumi/models/experimental/cambrian/model/multimodal_encoder/siglip_encoder.py", "src/oumi/models/experimental/cambrian/model/multimodal_projector/builder.py", "src/oumi/models/experimental/cambrian/model/multimodal_projector/projectors.py", "src/oumi/models/experimental/cambrian/model/vision_sampler.py", "src/oumi/models/experimental/cambrian/utils.py"], "area": "other", "area_votes": {"infra": 2, "other": 18}, "body": "*  Update pyproject and pre-commit rules to ignore many ruff/lint errors  \r\n*  Excludes Cambrian `train` and `serve` for now.\r\n*  To facilitate future merges, all changes in Cambrian codebase are of these types: \r\n    * Deletions (files or lines) of unused or not-needed functionality. Also, deleted extra visual Encoders and text LLMs not used in the currently published Cambrian models to keep PR size under control.\r\n    * Edit Imports to use `oumi` package hierarchy\r\n* Updated Cambrian files to use `oumi` logger\r\n \r\nBased on https://github.com/cambrian-mllm/cambrian/tree/78b97086cd96d84e62f45c7cee1b87386615a9a4\r\n\r\nThis is an initial import. It allows loading of pre-trained Cambrian models (including vision towers) with weights published on HF :\r\n* `nyu-visionx/cambrian-phi3-3b`\r\n* `nyu-visionx/cambrian-8b`\r\n* `nyu-visionx/cambrian-13b`\r\n* `nyu-visionx/cambrian-34b`\r\n\r\nMore work is needed to enable tuning, and inference.\r\n\r\nTowards OPE-374, OPE-377", "merged_at": "2024-10-09T20:04:47Z"}
{"number": 555, "title": "Log dataset info: shape, columns, other metainfo", "files": ["src/oumi/core/datasets/base_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "", "merged_at": "2024-09-26T15:41:15Z"}
{"number": 554, "title": "Update experimental pretokenize_dataset tool", "files": ["src/experimental/pretokenize/process_dataset.py", "src/experimental/pretokenize/sky.yaml"], "area": "other", "area_votes": {"other": 2}, "body": "-- Rename the tool  to `process_dataset`, and add a new option to skip tokenization, in which case the tool  downloads HF datasets, reshards it , it writes to GCS file system\r\n-- Add new sharding params\r\n-- Add new dataset params (`split`, `subset`, `trust_remote_code`)\r\n-- Add SkyPilot example how to download and save COCO dataset\r\n\r\nTowards OPE-102, OPE-353", "merged_at": "2024-09-26T16:24:02Z"}
{"number": 553, "title": "Pass `split` param to `datasets.load_dataset()`", "files": ["src/oumi/core/datasets/base_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "- It enables additional features like Slicing (`--split=\"train[:20%]\"`) https://huggingface.co/docs/datasets/v1.10.1/splits.html#slicing-api\r\n- Allows optimization e.g., not reading all splits if not needed ", "merged_at": "2024-09-25T17:55:21Z"}
{"number": 550, "title": "Use `local_rank` to query GPU temperature", "files": ["src/oumi/performance/telemetry.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Previously, it would default to `device_index=0`\r\n\r\nFixes OPE-344", "merged_at": "2024-09-24T18:16:59Z"}
{"number": 549, "title": "Fix a bug for handling stopped sky clusters in the oumi launcher.", "files": ["src/oumi/launcher/clouds/sky_cloud.py", "tests/launcher/clouds/test_sky_cloud.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Previously `oumi-launch -a status` would throw an error if a cluster was suspended but not down. I've added a check to only list a cluster if it's UP and ignore it otherwise as it cannot have any jobs running while suspended.\r\n\r\nFixes OPE-371", "merged_at": "2024-09-24T18:22:05Z"}
{"number": 547, "title": "[tiny] Add fp paged_adam optimizer option", "files": ["src/oumi/builders/optimizers.py"], "area": "other", "area_votes": {"other": 1}, "body": "- Add fp paged_adam optimizer option", "merged_at": "2024-09-24T16:25:40Z"}
{"number": 546, "title": "[tiny] Allow conversation metadata to contain values other than str", "files": ["src/oumi/core/types/turn.py"], "area": "data", "area_votes": {"data": 1}, "body": "- Allow conversation metadata to contain values other than `str`\r\n- `str` is too restrictive, as we often need to store other types (`float`, `bool`, `dict`, ...)\r\n- Setting the value type to `Any` for now.", "merged_at": "2024-09-24T16:25:55Z"}
{"number": 545, "title": "[tiny] Add util to get install folder root dir", "files": ["src/oumi/builders/models.py", "src/oumi/utils/io_utils.py", "tests/builders/test_models.py", "tests/utils/test_io_utils.py"], "area": "other", "area_votes": {"other": 2}, "body": "- To be used for resources that depend on the root folder (e.g. chat_templates, judge configs, etc)\r\n", "merged_at": "2024-09-24T16:25:28Z"}
{"number": 544, "title": "[tiny] fix small typo", "files": ["src/oumi/inference/llama_cpp_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "- fix small typo. Model max length was supposed to be a power of 2", "merged_at": "2024-09-24T14:48:47Z"}
{"number": 543, "title": "Minor changes in `scripts/benchmarks/minimal_multimodal_training.py`", "files": ["scripts/benchmarks/minimal_multimodal_training.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "-- Use full name of Flickr dataset\r\n-- Support configurable `split`, and change default split of `nlphuji/flickr30k` to `test` (the only split available there)\r\n-- Update separator in `--test-fsdp` in sample commands\r\n\r\nTowards OPE-353\r\n", "merged_at": "2024-09-24T16:12:24Z"}
{"number": 541, "title": "Add an optional `-t` flag to scripts/polaris/jobs/llama2b_pt_worker.sh", "files": ["scripts/polaris/jobs/llama2b_pt_worker.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "-- Enables additional telemetry\r\n\r\nFixes OPE-344", "merged_at": "2024-09-23T21:57:20Z"}
{"number": 539, "title": "Auto-format shell scripts under `scripts`", "files": ["configs/skypilot/sky_llama8b_sft.yaml", "scripts/polaris/jobs/build_apptainer_from_docker.sh", "scripts/polaris/jobs/download_model_from_hf.sh", "scripts/polaris/jobs/llama2b_pt_job.sh", "scripts/polaris/jobs/llama2b_pt_worker.sh", "scripts/polaris/jobs/llama8b_lora_eval.sh", "scripts/polaris/jobs/llama_tune.sh", "scripts/polaris/jobs/vllm_job.sh", "scripts/polaris/jobs/vllm_worker.sh", "scripts/polaris/launcher.sh", "scripts/polaris/polaris_init.sh"], "area": "infra", "area_votes": {"configs": 1, "infra": 10}, "body": "", "merged_at": "2024-09-23T20:30:29Z"}
{"number": 537, "title": "[tiny] cleanup multimodal benchmark script", "files": ["scripts/benchmarks/minimal_multimodal_training.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "- Use default dataset builder\r\n\r\nTowards OPE-353", "merged_at": "2024-09-23T18:31:58Z"}
{"number": 536, "title": "Create CODE_OF_CONDUCT.md", "files": ["CODE_OF_CONDUCT.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Adopt the standard github code of conduct. We can change this going forward as necessary.", "merged_at": "2024-09-23T16:46:19Z"}
{"number": 535, "title": "Add conversation helper methods", "files": ["src/oumi/core/types/turn.py", "tests/core/types/test_turn.py"], "area": "data", "area_votes": {"data": 1}, "body": "\r\n**Changes**\r\nThis pull request introduces new helper methods for frequently used operations on the `Conversation` class, with the option to filter by role:\r\n- `Conversation.first_message`\r\n- `Conversation.last_message`\r\n- `Conversation.filter_messages`\r\n\r\n\r\n**Usage**\r\n```python\r\nfrom oumi.core.types import Conversation, Message, Role\r\n\r\n# Create a conversation with some messages\r\nmessages = [\r\n    Message(role=Role.USER, content=\"Hello\"),\r\n    Message(role=Role.ASSISTANT, content=\"Hi, how can I help you?\"),\r\n    Message(role=Role.USER, content=\"I need assistance with my account.\")\r\n]\r\nconversation = Conversation(messages=messages)\r\n\r\n# Get the first message in the conversation\r\nfirst_msg = conversation.first_message()\r\nprint(first_msg.content)  # Output: \"Hello\"\r\n\r\n# Get the last message from the user\r\nlast_user_msg = conversation.last_message(role=Role.USER)\r\nprint(last_user_msg.content)  # Output: \"I need assistance with my account.\"\r\n\r\n# Get the system messages\r\nsystem_messages = conversation.filter_messages(role=Role.SYSTEM)\r\nassert len(system_messages) == 0\r\n```", "merged_at": "2024-09-23T16:52:32Z"}
{"number": 533, "title": "Switch from Flash Attention 2 to PyTorch SDPA", "files": ["configs/oumi/chatqa/chatqa.stage1.yaml", "configs/oumi/chatqa/chatqa.stage2.yaml", "configs/oumi/gpt2.asynceval.nvidia.yaml", "configs/oumi/gpt2.pt.yaml", "configs/oumi/jobs/gcp/bulk_inference.yaml", "configs/oumi/jobs/gcp/llama2b_ddp.yaml", "configs/oumi/jobs/gcp/llama2b_fsdp_oumi.yaml", "configs/oumi/jobs/gcp/llama2b_fsdp_trl.yaml", "configs/oumi/jobs/gcp/llama70b_lora.yaml", "configs/oumi/jobs/gcp/llama70b_sft.yaml", "configs/oumi/jobs/gcp/llama8b_eval.yaml", "configs/oumi/jobs/gcp/llama8b_lora.yaml", "configs/oumi/jobs/gcp/llama8b_sft.yaml", "configs/oumi/llama2b.pt.fsdp.oumi.yaml", "configs/oumi/llama2b.pt.fsdp.trl.yaml", "configs/oumi/llama2b.pt.yaml", "configs/oumi/llama3.8b.aya.eval.yaml", "configs/oumi/llama3.8b.aya.sft.yaml", "configs/oumi/llama70b.eval.yaml", "configs/oumi/llama70b.lora.yaml", "configs/oumi/llama70b.sft.yaml", "configs/oumi/llama8b.eval.legacy.yaml", "configs/oumi/llama8b.eval.yaml", "configs/oumi/llama8b.lora.yaml", "configs/oumi/llama8b.sft.yaml", "configs/oumi/zephyr.7b/sft/eval_mmlu.yaml", "configs/oumi/zephyr.7b/sft/full.yaml", "configs/oumi/zephyr.7b/sft/qlora.yaml", "configs/skypilot/sky_chatqa.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_gpt2_eval.yaml", "configs/skypilot/sky_gpt2_fsdp.yaml", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama2b_ddp.yaml", "configs/skypilot/sky_llama2b_deepspeed.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_llama3_8b_aya_eval.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml", "configs/skypilot/sky_llama70b_lora.yaml", "configs/skypilot/sky_llama70b_sft.yaml", "configs/skypilot/sky_llama8b_lora.yaml", "configs/skypilot/sky_llama8b_sft.yaml", "configs/skypilot/sky_phi3_dpo.yaml", "configs/skypilot/sky_phi3_dpo_fsdp.yaml", "configs/skypilot/sky_phi3_eval.yaml", "configs/skypilot/sky_zephyr_7b_qlora_sft.yaml", "configs/skypilot/sky_zephyr_7b_sft.yaml", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "pyproject.toml", "src/oumi/builders/models.py", "src/oumi/core/configs/params/model_params.py"], "area": "configs", "area_votes": {"configs": 48, "docs": 2, "infra": 1, "other": 1}, "body": "Towards OPE-233, OPE-363\r\n\r\n- Use `attn_implementation` flag in HF Trainer, allowing use of both eager and sdpa implementations\r\n- Switch all configs to specify SDPA instead of flash attention. We could also delete this line altogether, since it defaults to SDPA, but we may eventually want to be explicit for the Oumi trainer.\r\n- Remove `flash-attn` from `pyproject.toml`, and create a `train_gpu` target.\r\n- Switch sky configs to use `train_gpu` target for a one-line pip install. Previously, we couldn't because `flash-attn` required a two-step install.\r\n- Switch remaining Llama 8B Lora configs to torchrun since it's DDP\r\n\r\nOverall, this change has no effect on runtime, except for a 10% speedup for 8B Lora: https://docs.google.com/spreadsheets/d/1EIgbdVhI2mLzxpbPGSMMixyUJSpDqUrCGMdIVzeBDQk/edit?gid=1819500808#gid=1819500808", "merged_at": "2024-09-24T16:40:13Z"}
{"number": 532, "title": "Fix a small bug in `infer_interactive()`:  only prints the first character", "files": ["src/oumi/infer.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "-- Also remove the outdated comment", "merged_at": "2024-09-21T01:34:54Z"}
{"number": 530, "title": "Auto-format `pyproject` and `pre-commit` configs", "files": [".pre-commit-config.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "Formatters:\r\n\r\n```\r\n\"[toml]\": {\r\n        \"editor.defaultFormatter\": \"tamasfe.even-better-toml\"\r\n    },\r\n    \"[yaml]\": {\r\n        \"editor.defaultFormatter\": \"redhat.vscode-yaml\"\r\n    },\r\n```", "merged_at": "2024-09-20T21:26:36Z"}
{"number": 529, "title": "Update Makefile", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "When reinstalling oumi from scratch, `conda activate oumi` failed for me until I ran `source ~/.zshrc`", "merged_at": "2024-09-20T21:29:42Z"}
{"number": 526, "title": "Update name typo", "files": ["LICENSE", "README.md", "docs/.sphinx/conf.py", "docs/.sphinx/index.rst", "notebooks/Oumi - A Tour.ipynb", "src/experimental/__init__.py", "src/oumi/__init__.py", "src/oumi/builders/__init__.py", "src/oumi/core/__init__.py", "src/oumi/core/callbacks/__init__.py", "src/oumi/core/configs/__init__.py", "src/oumi/core/datasets/__init__.py", "src/oumi/core/inference/__init__.py", "src/oumi/core/launcher/__init__.py", "src/oumi/core/models/__init__.py", "src/oumi/core/tokenizers/__init__.py", "src/oumi/core/trainers/__init__.py", "src/oumi/core/types/__init__.py", "src/oumi/datasets/__init__.py", "src/oumi/evaluation/__init__.py", "src/oumi/inference/__init__.py", "src/oumi/launcher/__init__.py", "src/oumi/launcher/clouds/__init__.py", "src/oumi/models/__init__.py"], "area": "docs", "area_votes": {"docs": 5}, "body": "Replace all instances of `Open Unified Machine Intelligence` -> `Open Universal  Machine Intelligence`", "merged_at": "2024-09-20T18:12:44Z"}
{"number": 523, "title": "Add anthropic inference engine", "files": ["src/oumi/core/configs/params/remote_params.py", "src/oumi/core/inference/base_inference_engine.py", "src/oumi/inference/__init__.py", "src/oumi/inference/anthropic_inference_engine.py", "src/oumi/inference/remote_inference_engine.py", "tests/inference/test_anthropic_inference_engine.py"], "area": "inference", "area_votes": {"configs": 1, "inference": 3}, "body": "# Add Anthropic Inference Engine\r\n\r\nThis PR introduces support for the Anthropic API, allowing users run remote inference with Anthropic's language models.\r\n\r\n## Changes\r\n\r\n1. Added `AnthropicInferenceEngine` class to handle Anthropic-specific API inputs / outputs.\r\n2. Updated `RemoteInferenceEngine` to be decouple generating auth headers from query.\r\n3. Modified `RemoteParams` to include an optional environment variable name for API keys.\r\n\r\nTowards OPE-397", "merged_at": "2024-09-20T17:46:46Z"}
{"number": 522, "title": "Rename website references to oumi.ai", "files": ["README.md", "docs/.sphinx/index.rst", "notebooks/Oumi - A Tour.ipynb", "notebooks/Oumi - Build Zephyr 7B.ipynb", "notebooks/Oumi - Colab Setup Example.ipynb", "notebooks/Oumi - Datasets Tutorial.ipynb", "notebooks/Oumi - Deploying a Job.ipynb", "notebooks/Oumi - Finetuning Tutorial.ipynb", "notebooks/Oumi - Launching Jobs on Custom Clusters.ipynb", "notebooks/Oumi - Multinode Inference on Polaris.ipynb", "notebooks/Oumi - Running Jobs Remotely.ipynb", "notebooks/Oumi - Tuning Llama.ipynb", "notebooks/Oumi - Using NanoGPT.ipynb", "notebooks/Oumi - Using vLLM Engine for Inference.ipynb", "pyproject.toml"], "area": "docs", "area_votes": {"docs": 14, "infra": 1}, "body": "OPE-346\r\n\r\nAlso lowercase notebook names. My previous attempt didn't go through; I had to run `git config core.ignorecase false` first.", "merged_at": "2024-09-20T17:39:00Z"}
{"number": 518, "title": "[Minor] Issues arose by \"newcomer\" exploration [1/K]", "files": ["README.md", "notebooks/OUMI - A Tour.ipynb"], "area": "docs", "area_votes": {"docs": 2}, "body": "README.md:\r\n- Improves comments/syntax.\r\n\r\nnotebooks/OUMI - A Tour.ipynb\r\n- Improves/adds comments.\r\n- Updates call arguments to oumi.infer.", "merged_at": "2024-09-19T18:31:23Z"}
{"number": 516, "title": "Freeze `lm-eval` and `torch` versions as a workaround for  OPE-390", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "-- Otherwise, training jobs fail to start on GCP \r\n\r\nTowards  OPE-390", "merged_at": "2024-09-19T18:54:35Z"}
{"number": 514, "title": "Update TOTAL_NUM_GPUS compare commands in SkyPilot configs", "files": ["configs/skypilot/sky_llama2b_ddp.yaml", "configs/skypilot/sky_llama2b_deepspeed.yaml", "configs/skypilot/sky_llama70b_lora.yaml", "configs/skypilot/sky_llama8b_lora.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "-- The current syntax leads to errors on GCP => switch to `test $X -lt $Y`\r\n-- Add more warnings if FSDP is used with 1 GPU", "merged_at": "2024-09-18T16:40:10Z"}
{"number": 512, "title": "Update final lema references", "files": ["README.md", "configs/oumi/jobs/gcp/llama70b_lora.yaml", "configs/oumi/jobs/gcp/llama70b_sft.yaml", "configs/oumi/jobs/gcp/llama8b_eval.yaml", "configs/skypilot/sky_gpt2_eval.yaml", "configs/skypilot/sky_llama70b_lora.yaml", "configs/skypilot/sky_llama70b_sft.yaml", "configs/skypilot/sky_phi3_eval.yaml", "configs/skypilot/sky_zephyr_7b_qlora_sft.yaml", "configs/skypilot/sky_zephyr_7b_sft.yaml", "src/experimental/pretokenize/sky.yaml"], "area": "configs", "area_votes": {"docs": 1, "configs": 9, "other": 1}, "body": "Towards OPE-364\r\n\r\nThis PR updates our README badges and our GCS bucket names. I copied over the contents of the old buckets into the new ones, and will delete the old buckets a few days after this PR is merged.\r\n\r\nThe only two lema references left after this refactor are our GCP project id `lema-dev` and our email `contact@openlema.com`.\r\n\r\n", "merged_at": "2024-09-17T20:12:38Z"}
{"number": 511, "title": "Rename remaining lema references in `docs/ `", "files": ["docs/.sphinx/_templates/apidoc/package.rst.jinja", "docs/.sphinx/conf.py", "docs/.sphinx/index.rst", "docs/.sphinx/tutorials.rst", "docs/CLOUD_TRAINING.md", "docs/DEV_SETUP.md", "docs/GIT_WORKFLOW.md", "docs/MODEL_DEFINITION.md", "docs/NEW_MULTIMODAL_DATASET.md", "src/oumi/datasets/mmlu.py"], "area": "docs", "area_votes": {"docs": 9, "data": 1}, "body": "Towards OPE-364\r\n\r\nThis includes our Markdown guides and some references left in Sphinx.", "merged_at": "2024-09-17T19:46:29Z"}
{"number": 510, "title": "Update conf.py", "files": ["docs/.sphinx/conf.py"], "area": "docs", "area_votes": {"docs": 1}, "body": "Update lema -> Oumi", "merged_at": "2024-09-17T16:41:02Z"}
{"number": 509, "title": "Re-generate Sphinx docs", "files": ["docs/.sphinx/apidoc/lema.core.callbacks.rst", "docs/.sphinx/apidoc/lema.core.configs.params.rst", "docs/.sphinx/apidoc/lema.core.rst", "docs/.sphinx/apidoc/lema.datasets.vision_language.rst", "docs/.sphinx/apidoc/lema.launcher.clients.rst", "docs/.sphinx/apidoc/lema.launcher.clusters.rst", "docs/.sphinx/apidoc/lema.performance.rst", "docs/.sphinx/apidoc/lema.rst", "docs/.sphinx/apidoc/lema.utils.rst", "docs/.sphinx/apidoc/modules.rst", "docs/.sphinx/apidoc/oumi.builders.rst", "docs/.sphinx/apidoc/oumi.core.callbacks.rst", "docs/.sphinx/apidoc/oumi.core.configs.params.rst", "docs/.sphinx/apidoc/oumi.core.configs.rst", "docs/.sphinx/apidoc/oumi.core.datasets.rst", "docs/.sphinx/apidoc/oumi.core.inference.rst", "docs/.sphinx/apidoc/oumi.core.launcher.rst", "docs/.sphinx/apidoc/oumi.core.models.rst", "docs/.sphinx/apidoc/oumi.core.rst", "docs/.sphinx/apidoc/oumi.core.tokenizers.rst", "docs/.sphinx/apidoc/oumi.core.trainers.rst", "docs/.sphinx/apidoc/oumi.core.types.rst", "docs/.sphinx/apidoc/oumi.datasets.rst", "docs/.sphinx/apidoc/oumi.datasets.vision_language.rst", "docs/.sphinx/apidoc/oumi.evaluation.rst", "docs/.sphinx/apidoc/oumi.inference.rst", "docs/.sphinx/apidoc/oumi.launcher.clients.rst", "docs/.sphinx/apidoc/oumi.launcher.clouds.rst", "docs/.sphinx/apidoc/oumi.launcher.clusters.rst", "docs/.sphinx/apidoc/oumi.launcher.rst", "docs/.sphinx/apidoc/oumi.models.rst", "docs/.sphinx/apidoc/oumi.performance.rst", "docs/.sphinx/apidoc/oumi.rst", "docs/.sphinx/apidoc/oumi.utils.rst", "docs/.sphinx/links/new_multimodal_dataset.rst"], "area": "docs", "area_votes": {"docs": 35}, "body": "Towards OPE-364\r\n\r\nCreated by running `make docs-rebuild`", "merged_at": "2024-09-17T02:29:03Z"}
{"number": 507, "title": "Rename configs/lema to configs/oumi", "files": [".vscode/launch.json", "README.md", "USAGE.md", "configs/accelerate/gpt2.fsdp.yaml", "configs/accelerate/llama.ddp.yaml", "configs/accelerate/llama.deepspeed.yaml", "configs/accelerate/llama.fsdp.yaml", "configs/accelerate/local_machine_no_distributed_cpu_only.yaml", "configs/accelerate/phi3.fsdp.dpo.yaml", "configs/oumi/chatqa/chatqa.stage1.yaml", "configs/oumi/chatqa/chatqa.stage2.yaml", "configs/oumi/gpt2.asynceval.nvidia.yaml", "configs/oumi/gpt2.pt.mac.yaml", "configs/oumi/gpt2.pt.yaml", "configs/oumi/jobs/gcp/hello_world.yaml", "configs/oumi/jobs/gcp/llama2b_fsdp.yaml", "configs/oumi/jobs/gcp/llama70b_lora.yaml", "configs/oumi/jobs/gcp/llama70b_sft.yaml", "configs/oumi/jobs/gcp/llama8b_eval.yaml", "configs/oumi/jobs/gcp/llama8b_lora.yaml", "configs/oumi/jobs/gcp/llama8b_sft.yaml", "configs/oumi/jobs/polaris/hello_world.yaml", "configs/oumi/jobs/polaris/llama2b_pt_job.yaml", "configs/oumi/jobs/polaris/llama70b_eval.yaml", "configs/oumi/jobs/polaris/llama70b_lora.yaml", "configs/oumi/jobs/polaris/llama70b_sft.yaml", "configs/oumi/jobs/polaris/llama8b_eval.yaml", "configs/oumi/jobs/polaris/llama8b_lora.yaml", "configs/oumi/jobs/polaris/llama8b_sft.yaml", "configs/oumi/jobs/polaris/vllm.yaml", "configs/oumi/llama2b.pt.fsdp.yaml", "configs/oumi/llama2b.pt.yaml", "configs/oumi/llama3.8b.aya.eval.yaml", "configs/oumi/llama3.8b.aya.sft.yaml", "configs/oumi/llama70b.eval.yaml", "configs/oumi/llama70b.lora.yaml", "configs/oumi/llama70b.sft.yaml", "configs/oumi/llama8b.eval.legacy.yaml", "configs/oumi/llama8b.eval.yaml", "configs/oumi/llama8b.lora.yaml", "configs/oumi/llama8b.sft.yaml", "configs/oumi/phi3.dpo.mac.32g.yaml", "configs/oumi/phi3.dpo.nvidia.24g.yaml", "configs/oumi/phi3.dpo.nvidia.80g.yaml", "configs/oumi/phi3.dpo.yaml", "configs/oumi/phi3.eval.lema.mac.yaml", "configs/oumi/phi3.eval.lema.nvidia.yaml", "configs/oumi/phi3.eval.lm_harness.yaml", "configs/oumi/phi3.lora.yaml", "configs/oumi/phi3.sft.nvidia.24g.yaml", "configs/oumi/visionlanguage.sft.config.yaml", "configs/oumi/zephyr.7b/sft/eval_mmlu.yaml", "configs/oumi/zephyr.7b/sft/full.yaml", "configs/oumi/zephyr.7b/sft/qlora.yaml", "configs/skypilot/sky_chatqa.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_gpt2_eval.yaml", "configs/skypilot/sky_gpt2_fsdp.yaml", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama2b_ddp.yaml", "configs/skypilot/sky_llama2b_deepspeed.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_llama3_8b_aya_eval.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml", "configs/skypilot/sky_llama70b_lora.yaml", "configs/skypilot/sky_llama70b_sft.yaml", "configs/skypilot/sky_llama8b_lora.yaml", "configs/skypilot/sky_llama8b_sft.yaml", "configs/skypilot/sky_phi3_dpo.yaml", "configs/skypilot/sky_phi3_dpo_fsdp.yaml", "configs/skypilot/sky_phi3_eval.yaml", "configs/skypilot/sky_zephyr_7b_qlora_sft.yaml", "configs/skypilot/sky_zephyr_7b_sft.yaml", "notebooks/OUMI - A Tour.ipynb", "notebooks/OUMI - Build Zephyr 7B.ipynb", "notebooks/OUMI - Colab Setup Example.ipynb", "notebooks/OUMI - Datasets Tutorial.ipynb", "notebooks/OUMI - Deploying a Job.ipynb", "notebooks/OUMI - Finetuning Tutorial.ipynb", "notebooks/OUMI - Launching Jobs on Custom Clusters.ipynb", "notebooks/OUMI - Multinode Inference on Polaris.ipynb", "notebooks/OUMI - Running Jobs Remotely.ipynb", "notebooks/OUMI - Tuning Llama.ipynb", "notebooks/OUMI - Using NanoGPT.ipynb", "notebooks/OUMI - Using vLLM Engine for Inference.ipynb", "scripts/benchmarks/benchmark_trainers.sh", "scripts/benchmarks/minimal_fsdp_training.py", "scripts/benchmarks/minimal_multimodal_training.py", "scripts/llm-as-judge/judge_dataset_polaris.py", "scripts/polaris/jobs/example_job.sh", "scripts/polaris/jobs/llama2b_pt_worker.sh", "scripts/polaris/jobs/llama8b_lora_eval.sh", "scripts/polaris/jobs/llama_tune.sh", "src/experimental/pretokenize/sky.yaml", "src/oumi/builders/data.py", "src/oumi/builders/oumi_data.py", "src/oumi/core/trainers/__init__.py", "src/oumi/core/trainers/oumi_trainer.py", "tests/builders/test_oumi_data.py", "tests/core/configs/test_parse_configs.py", "tests/core/trainers/test_oumi_trainer.py"], "area": "configs", "area_votes": {"other": 4, "docs": 14, "configs": 70, "infra": 8, "training": 1}, "body": "Towards OPE-364\r\n\r\nThe main goal of this PR is to rename lema to oumi in all remaining dir/filename references.\r\n\r\n- Rename `configs/lema`, and caught stray references the automatic VSCode refactor didn't catch\r\n- Rename `lema_data.py` and `lema_trainer.py`\r\n- Rename notebooks\r\n\r\nTested:\r\n- Cmd+P in Vscode shows all files with lema in the name. The only ones left are docs/.sphinx/*, `configs/clouds/gcp/lema_dev_iam_custom_role.yaml`, and configs for the LeMa eval framework.\r\n- Polaris jobs still work", "merged_at": "2024-09-16T23:59:53Z"}
{"number": 502, "title": "Replace `pip install flash-attn` with `.[gpu]` target", "files": ["configs/lema/jobs/gcp/llama2b_fsdp.yaml", "configs/lema/jobs/gcp/llama70b_lora.yaml", "configs/lema/jobs/gcp/llama70b_sft.yaml", "configs/lema/jobs/gcp/llama8b_eval.yaml", "configs/lema/jobs/gcp/llama8b_lora.yaml", "configs/lema/jobs/gcp/llama8b_sft.yaml", "configs/skypilot/sky_chatqa.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_gpt2_eval.yaml", "configs/skypilot/sky_gpt2_fsdp.yaml", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama2b_ddp.yaml", "configs/skypilot/sky_llama2b_deepspeed.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_llama3_8b_aya_eval.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml", "configs/skypilot/sky_llama70b_lora.yaml", "configs/skypilot/sky_llama70b_sft.yaml", "configs/skypilot/sky_llama8b_lora.yaml", "configs/skypilot/sky_llama8b_sft.yaml", "configs/skypilot/sky_phi3_dpo.yaml", "configs/skypilot/sky_phi3_dpo_fsdp.yaml", "configs/skypilot/sky_phi3_eval.yaml", "configs/skypilot/sky_zephyr_7b_qlora_sft.yaml", "configs/skypilot/sky_zephyr_7b_sft.yaml", "notebooks/LeMa - Running Jobs Remotely.ipynb", "scripts/polaris/launcher.sh", "src/lema/launcher/clusters/polaris_cluster.py", "tests/launcher/clusters/test_polaris_cluster.py"], "area": "configs", "area_votes": {"configs": 25, "docs": 1, "infra": 1, "launcher": 1}, "body": "Towards OPE-363\r\n\r\nIdeally we could combine the two install lines into one, but it doesn't seem to work since flash-attn expects pytorch to already be present. This PR makes our install setup more generalized, and also enables Liger kernel usage in all our jobs.", "merged_at": "2024-09-16T17:40:45Z"}
{"number": 501, "title": "Update `TelemetryCallback` to write JSON with GPU temperature summary", "files": ["src/lema/core/callbacks/telemetry_callback.py", "src/lema/performance/telemetry.py", "tests/performance/test_telemetry.py"], "area": "training", "area_votes": {"training": 2}, "body": "-- The summary aggregates all ranks, and includes rank index with max GPU temp observed during training\r\n-- Add related helper function to `TelemetryTracker` \r\n\r\nTowards OPE-344", "merged_at": "2024-09-16T23:08:48Z"}
{"number": 500, "title": "Use HF's built-in gradient checkpointing argument", "files": ["configs/lema/llama2b.pt.yaml", "configs/lema/llama70b.lora.yaml", "configs/lema/llama70b.sft.yaml", "configs/lema/llama8b.lora.yaml", "configs/lema/llama8b.sft.yaml", "src/lema/core/configs/params/training_params.py", "src/lema/core/trainers/lema_trainer.py", "src/lema/train.py", "tests/core/trainers/test_lema_trainer.py"], "area": "configs", "area_votes": {"configs": 6, "training": 2}, "body": "Fixes OPE-361\r\n\r\nSwitch to using the HF `gradient_checkpointing` arg instead of handling gradient checkpointing logic ourselves. I'm not sure why we chose to handle it ourselves initially, as the logic is [equivalent](https://github.com/huggingface/transformers/blob/7bb1c99800d235791dace10305731f377db8077b/src/transformers/trainer.py#L2151).\r\n\r\nAn advantage of switching to HF's arg is they'll now do the [arg validation check](https://github.com/huggingface/transformers/blob/0963229e287501bed52ae1dabc17922524de6992/src/transformers/trainer.py#L4846) for gradient checkpointing for FSDP. I also disable `enable_gradient_checkpointing` for Llama 8b/70b SFT and 70b Lora, since it's already enabled in the accelerate FSDP config, to pass the arg validation check.\r\n\r\nTested the following to ensure there's no regression:\r\n- Lema trainer FSDP\r\n- HF trainer FSDP\r\n- HF trainer Lora training\r\n\r\nAnother relevant warning in HF: https://github.com/huggingface/transformers/blob/7bb1c99800d235791dace10305731f377db8077b/src/transformers/training_args.py#L1854", "merged_at": "2024-09-16T20:40:22Z"}
{"number": 499, "title": "Add simpler builder for single dataset use cases", "files": ["src/lema/builders/__init__.py", "src/lema/builders/data.py", "tests/integration/builders/test_data.py"], "area": "other", "area_votes": {"other": 1}, "body": "### Changes\r\n- Loading a single dataset in a notebook or script is very cumbersome, as we need to create 4 nested objects to pass the relevant data\r\n- This PR introduces wrappers around `build_dataset_mixture` for the case where we have a single dataset:\r\n  -  `build_dataset`: if the user just wants to pass kwargs\r\n  - `build_dataset_from_params`: if the users has a `DatasetParams` object ready\r\n\r\n**Before**\r\n```python\r\nconfig = TrainingConfig(\r\n    data=DataParams(\r\n        train=DatasetSplitParams(\r\n            datasets=[\r\n                DatasetParams(\r\n                    dataset_name=\"mydataset\",\r\n                    split=\"train\",\r\n                )\r\n            ]\r\n        )\r\n    ),\r\n)\r\ndataset = build_dataset_from_mixture(config, tokenizer=tokenizer)\r\n```\r\n\r\n**After**\r\n```python\r\ndataset = build_dataset(dataset_name=\"mydataset\", split=\"train\", tokenizer=tokenizer)\r\n\r\n# OR\r\n\r\nparams = DatasetParams(\r\n   dataset_name=\"mydataset\",\r\n   split=\"train\"\r\n)\r\ndataset = build_dataset_from_params(params, tokenizer=tokenizer)\r\n```", "merged_at": "2024-09-16T18:18:48Z"}
{"number": 495, "title": "Increase the rsync timeout from 40s to 300s", "files": ["src/lema/launcher/clients/polaris_client.py", "tests/launcher/clients/test_polaris_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Increase the rsync timeout from 40s to 300s\r\n\r\nFixes OPE-360", "merged_at": "2024-09-13T21:31:13Z"}
{"number": 493, "title": "Add documentation to peft_params", "files": ["src/lema/core/configs/params/base_params.py", "src/lema/core/configs/params/peft_params.py"], "area": "configs", "area_votes": {"configs": 2}, "body": "**Changes**\r\n- Add docstring to `peft_params` class", "merged_at": "2024-09-13T20:45:40Z"}
{"number": 492, "title": "Introduce `BaseTrainerCallback` alias", "files": ["src/lema/builders/callbacks.py", "src/lema/core/callbacks/__init__.py", "src/lema/core/callbacks/base_trainer_callback.py", "src/lema/core/callbacks/hf_mfu_callback.py", "src/lema/core/callbacks/mfu_callback.py", "src/lema/core/callbacks/profiler_step_callback.py", "src/lema/core/callbacks/telemetry_callback.py"], "area": "training", "area_votes": {"other": 1, "training": 5}, "body": "-- Start using it for existing code\r\n-- It's consistent with `Base Tokenizer` and the first step towards OPE-250\r\n\r\n\r\nTowards OPE-250", "merged_at": "2024-09-13T18:41:24Z"}
{"number": 491, "title": "Switch to using official UV action with dependency caching", "files": [".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "**Changes**\r\n- Switch to using official UV action with dependency caching\r\n- Reduces build time by 1-2 minutes with a cache hit\r\n- No need to manually install uv in the setup section", "merged_at": "2024-09-13T18:25:57Z"}
{"number": 488, "title": "Minor fixes in DISTRIBUTED_TRAINING.md", "files": ["docs/DISTRIBUTED_TRAINING.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "-- Correct few FSDP links to reflect the new directory structure (`docs` sub-dir)\r\n-- Remove the instruction (`Set num_processes: N where N is the number of GPUs.`) . No longer necessary", "merged_at": "2024-09-13T17:52:32Z"}
{"number": 487, "title": "Add missing documentation for model_params", "files": ["src/lema/core/configs/params/model_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "Add missing documentation for model_params", "merged_at": "2024-09-13T00:04:17Z"}
{"number": 486, "title": "Cleanup FSDP wrap class auto guesser", "files": ["src/lema/utils/torch_naming_heuristics.py", "tests/utils/test_torch_naming_heuristics.py"], "area": "other", "area_votes": {"other": 1}, "body": "**Changes**\r\n- Simplify the logic with a higher level heuristic.\r\n- Add unit tests\r\n- Test with more model architectures", "merged_at": "2024-09-13T00:04:05Z"}
{"number": 484, "title": "Remove device_map for model init from config", "files": ["src/lema/builders/models.py"], "area": "other", "area_votes": {"other": 1}, "body": "Received this error for Llama 2B PT jobs: `LlamaForCausalLM.__init__() got an unexpected keyword argument 'device_map'`. Was introduced in #459.", "merged_at": "2024-09-12T20:39:02Z"}
{"number": 483, "title": "Small typo fix in the vllm notebook", "files": ["notebooks/LeMa - Using vLLM Engine for Inference.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "Small typo fix in the vllm notebook", "merged_at": "2024-09-12T23:30:49Z"}
{"number": 476, "title": "Add test coverage target, update pyproject.toml metadata", "files": [".gitignore", "Makefile", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 3}, "body": "**Changes**\r\n- Add `make coverage` target to our Makefile to compute test coverage. As of this PR, we're at 75% coverage\r\n- Update pip package metadata", "merged_at": "2024-09-11T23:25:45Z"}
{"number": 475, "title": "Rebuild docs, add multi-modal tutorial", "files": ["docs/.sphinx/apidoc/lema.core.inference.rst", "docs/.sphinx/apidoc/lema.core.rst", "docs/.sphinx/apidoc/lema.datasets.rst", "docs/.sphinx/apidoc/lema.datasets.vision_language.rst", "docs/.sphinx/apidoc/lema.inference.rst", "docs/.sphinx/apidoc/lema.rst", "docs/.sphinx/index.rst", "docs/.sphinx/links/new_multimodal_dataset.rst"], "area": "docs", "area_votes": {"docs": 8}, "body": "**Changes**\r\n- Add link to the multi-modal dataset tutorial\r\n- [auto-generated] rebuild docs to include the newly added modules (inference, fsdp, multimodal)\r\n\r\n", "merged_at": "2024-09-11T23:25:29Z"}
{"number": 473, "title": "Removed duplicate task_done call", "files": ["scripts/polaris/jobs/python/vllm_parallel_inference.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "", "merged_at": "2024-09-11T20:51:11Z"}
{"number": 472, "title": "Update chat_template_builder", "files": ["src/lema/builders/models.py", "src/lema/utils/io_utils.py", "tests/builders/test_models.py"], "area": "other", "area_votes": {"other": 2}, "body": "## Changes\r\n\r\n- Update `build_chat_template` to load templates from any .jinja file in the template folder\r\n- Add `load_file` utility function in `io_utils.py`\r\n- Add tests for `build_chat_template`", "merged_at": "2024-09-11T19:14:04Z"}
{"number": 471, "title": "Create a local inference engine for vLLM", "files": ["pyproject.toml", "src/lema/core/inference/base_inference_engine.py", "src/lema/inference/__init__.py", "src/lema/inference/native_text_inference_engine.py", "src/lema/inference/vllm_inference_engine.py", "tests/inference/test_vllm_inference_engine.py"], "area": "inference", "area_votes": {"infra": 1, "inference": 3}, "body": "This PR creates a local inference engine for vLLM. A few caveats to note:\r\n\r\n- The pip package of vLLM does not support CPU inference (local GPU is required)\r\n- vLLM does not support MPS (mac hardware) for inference\r\n- Additional params are required for LoRA models\r\n\r\nThis inference engine must be run on linux or in a valid cuda environment.\r\n\r\nTODO: Verify functionality on Polaris / GCP\r\n\r\nTowards OPE-320\r\n", "merged_at": "2024-09-11T23:33:27Z"}
{"number": 469, "title": "Fixed issue with metadata extraction failure", "files": ["scripts/polaris/jobs/python/vllm_parallel_inference.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "Issue came up when renaming variables that went untested", "merged_at": "2024-09-11T02:43:04Z"}
{"number": 468, "title": "Combine telemetry from all ranks", "files": ["src/lema/core/callbacks/telemetry_callback.py", "src/lema/core/distributed.py", "src/lema/core/trainers/lema_trainer.py", "src/lema/performance/telemetry.py", "src/lema/train.py", "tests/core/test_distributed.py", "tests/core/trainers/test_lema_trainer.py", "tests/performance/test_telemetry.py"], "area": "training", "area_votes": {"training": 5}, "body": "-- Remove duplicate telemetry save in LEMA loop\r\n-- Add `lema.distributed.all_gather_object()` helper function (also, remove `dist` alias in the file and use fully-qualified name `torch.distributed`. The alias seems to cause problems for Mocks in tests)\r\n-- Include hostname into telemetry (can be useful for debugging)\r\n-- Update `TelemetryCallback` to honor `training.telemetry.collect_telemetry_for_all_ranks`\r\n\r\n\r\nTowards OPE-344", "merged_at": "2024-09-11T16:54:07Z"}
{"number": 467, "title": "Update README.md", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Update the status of models to be more verbose instead of just emojis", "merged_at": "2024-09-10T21:22:49Z"}
{"number": 466, "title": "Update inference to pass the generation config to inference engines.", "files": ["src/lema/core/configs/generation_config.py", "src/lema/core/inference/base_inference_engine.py", "src/lema/infer.py", "src/lema/inference/native_text_inference_engine.py", "tests/integration/infer/test_native_text_inference_engine.py"], "area": "inference", "area_votes": {"configs": 1, "inference": 3}, "body": "This is a followup from our discussion yesterday to pass single-purpose configs for our top-level nouns.\r\n\r\nKey changes:\r\n- I'm now piping the generation config down the chain rather than using kwargs.\r\n- If an inference config is not specified the default config will be used.\r\n- Added support for reading input files as jsonl (assuming our conversation input format)\r\n\r\nTowards OPE-320", "merged_at": "2024-09-10T19:55:24Z"}
{"number": 465, "title": "Fix several broken links and update installation instructions", "files": ["USAGE.md", "notebooks/LeMa - A Tour.ipynb"], "area": "docs", "area_votes": {"docs": 2}, "body": "- Fix two broken links in notebooks\r\n- Update the installation instruction to be simpler to run", "merged_at": "2024-09-09T23:55:07Z"}
{"number": 464, "title": "Update README to make installation steps more prominent", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Addresses feedback from Manos regarding first time installation (make Git and Conda instructions more clear).", "merged_at": "2024-09-09T23:06:22Z"}
{"number": 462, "title": "Update README.md", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Minor changes to Features description section", "merged_at": "2024-09-09T22:44:05Z"}
{"number": 460, "title": "Add integration tests for native inference (not using the CLI).", "files": ["src/lema/inference/native_text_inference_engine.py", "tests/integration/infer/test_native_text_inference_engine.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "Add integration tests for #456\r\n\r\nTowards OPE-320", "merged_at": "2024-09-06T22:58:44Z"}
{"number": 458, "title": "[tiny] Add missing default value to hf_trainer", "files": ["src/lema/core/trainers/hf_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "", "merged_at": "2024-09-05T23:50:34Z"}
{"number": 456, "title": "Add the base classes for inference. Pull out logic from `infer` to a native text inference engine.", "files": ["pyproject.toml", "src/lema/core/inference/__init__.py", "src/lema/core/inference/base_inference_engine.py", "src/lema/infer.py", "src/lema/inference/__init__.py", "src/lema/inference/native_text_inference_engine.py", "tests/integration/infer/test_infer.py"], "area": "inference", "area_votes": {"infra": 1, "inference": 3}, "body": "Create a BaseInferenceEngine interface that simply exposes an \"infer(...)\" method.\r\n\r\nPull out the logic from infer into a local inference engine.\r\n\r\nTODOs (in follow up PRs):\r\n- Add tests for that NativeTextInferenceEngine class\r\n- Add an `InferenceEngine` class that builds each InferenceEngine under the hood so users don't need to know about each class.\r\n\r\nTowards OPE-320", "merged_at": "2024-09-06T21:04:57Z"}
{"number": 455, "title": "[tiny] enable ruff format on save with notebooks", "files": [".vscode/settings.json"], "area": "other", "area_votes": {"other": 1}, "body": "- Update default `vscode` config to enable notebook cell formatting using the default formatter", "merged_at": "2024-09-05T19:42:20Z"}
{"number": 454, "title": "Simplify record_function annotation in LEMA training loop", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- The current annotation `microstep_X_of_Y` is too verbose, adds visual clutter\r\n\r\nFixes OPE-293", "merged_at": "2024-09-05T17:59:00Z"}
{"number": 452, "title": "Judge inference script for Polaris", "files": ["scripts/llm-as-judge/generate_prompts.py", "scripts/llm-as-judge/judge_dataset_polaris.py", "scripts/llm-as-judge/judge_dataset_polaris_postprocess.py", "scripts/polaris/jobs/python/vllm_inference.py", "scripts/polaris/jobs/python/vllm_parallel_inference.py"], "area": "infra", "area_votes": {"infra": 5}, "body": "", "merged_at": "2024-09-06T16:36:47Z"}
{"number": 451, "title": "[tiny] Fix inference notebook", "files": ["notebooks/LeMa - Multinode Inference on Polaris.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "scripts/polaris/jobs/vllm_job.sh requires the REPO and MODEL env vars", "merged_at": "2024-09-05T17:47:51Z"}
{"number": 449, "title": "Copy changes from PR-446 into GCP launcher config", "files": ["configs/lema/jobs/gcp/llama8b_eval.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Refer to https://github.com/openlema/lema/pull/446\r\n-- Add support for multi-node `LM_HARNESS` evals on GCP. \r\n  NOTE: multi-node may need more work, I got NCCL error at the end of eval \r\n-- Fix misconfigured GCP mount (should be under `storage_mounts:`) \r\n\r\nTowards OPE-313, OPE-140", "merged_at": "2024-09-05T00:44:19Z"}
{"number": 447, "title": "Add Llama 70B eval script", "files": ["configs/lema/jobs/polaris/llama70b_eval.yaml", "configs/lema/jobs/polaris/llama8b_eval.yaml", "configs/lema/llama70b.eval.yaml", "configs/lema/llama8b.eval.yaml", "notebooks/LeMa - Tuning Llama.ipynb", "scripts/polaris/jobs/llama_tune.sh", "src/lema/core/configs/params/model_params.py"], "area": "configs", "area_votes": {"configs": 5, "docs": 1, "infra": 1}, "body": "Towards OPE-313, OPE-140\r\n\r\nWe need to set `parallelize=True` as an argument for LM Harness for models too large for one GPU. I created a new `model.shard_for_eval` param for this (feedback welcome).\r\n\r\nI also updated the Llama notebook to have an eval section", "merged_at": "2024-09-05T01:01:55Z"}
{"number": 444, "title": "[docs] Update format + add missing docs to data_params.py", "files": ["src/lema/core/configs/params/data_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "- Update format\r\n- Add missing docs to data_params\r\n- Add notice about deprecated and experimental features", "merged_at": "2024-09-04T20:01:25Z"}
{"number": 442, "title": "Add Llama 8B eval script", "files": ["configs/lema/jobs/gcp/llama8b_eval.yaml", "configs/lema/jobs/polaris/llama2b_pt_job.yaml", "configs/lema/jobs/polaris/llama70b_lora.yaml", "configs/lema/jobs/polaris/llama70b_sft.yaml", "configs/lema/jobs/polaris/llama8b_eval.yaml", "configs/lema/jobs/polaris/llama8b_lora.yaml", "configs/lema/jobs/polaris/llama8b_sft.yaml", "configs/lema/jobs/polaris/vllm.yaml", "configs/lema/llama8b.eval.yaml", "configs/lema/llama8b.lora.eval.yaml", "scripts/polaris/jobs/llama8b_lora_eval.sh"], "area": "configs", "area_votes": {"configs": 10, "infra": 1}, "body": "Towards OPE-313, OPE-140", "merged_at": "2024-09-04T18:26:48Z"}
{"number": 441, "title": "[tiny] Fix lema loop performance gap", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "This is by far the silliest bug I've at lema so far \ud83e\udd72: we we're compiling the model, but not using the compiled version \ud83e\udd26 \r\n\r\nWith this change, tokens/s/gpu on an A100 go from `11665` -> `13583` on the tested config, in-line with `TRL_SFT` ", "merged_at": "2024-09-04T01:54:56Z"}
{"number": 440, "title": "Add a mkdir to polaris init.", "files": ["src/lema/launcher/clients/polaris_client.py", "tests/launcher/clients/test_polaris_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Fixes an issue where a missing lema_launcher dir would cause the polaris launcher to fail.", "merged_at": "2024-09-03T23:31:15Z"}
{"number": 439, "title": "Enable Liger for Llama 8B SFT", "files": ["configs/lema/llama70b.lora.yaml", "configs/lema/llama70b.sft.yaml", "configs/lema/llama8b.lora.yaml", "configs/lema/llama8b.sft.yaml", "scripts/polaris/jobs/llama_tune.sh"], "area": "configs", "area_votes": {"configs": 4, "infra": 1}, "body": "Towards OPE-331\r\n\r\nWith our default config, enabling Liger for 8B SFT has a noticeable improvement to training speed, which is not the case for other configs.\r\n\r\nAlso switched to explicitly training for 1 epoch instead of putting the calculated steps needed. Also adjusted the save steps for the 8B configs to be more reasonable given the total number of steps and training speed.", "merged_at": "2024-09-03T17:33:55Z"}
{"number": 438, "title": "Updated Parallel Inference job", "files": ["notebooks/LeMa - Multinode Inference on Polaris.ipynb", "scripts/polaris/jobs/python/vllm_inference.py", "scripts/polaris/jobs/python/vllm_parallel_inference.py", "scripts/polaris/jobs/vllm_job.sh", "scripts/polaris/jobs/vllm_parallel_inference.py", "scripts/polaris/jobs/vllm_worker.sh"], "area": "infra", "area_votes": {"docs": 1, "infra": 5}, "body": "Updates parallel inference code to be configurable and better with resource usage and metric writing. Sets parallel to be the default.\r\n\r\nTests run using first 10k examples from OpenOrca dataset.\r\n\r\n8B (81x increase \ud83d\ude80 ):\r\nSingle-threaded (old): ~104 total tok/s\r\n250 workers (new): ~8124 total tok/s\r\n\r\n70B (86x increase \ud83d\ude80 ):\r\nSingle-threaded (old): ~45 total tok/s\r\n250 workers (new): ~3881 total tok/s\r\n\r\nFixes OPE-314", "merged_at": "2024-09-03T20:33:26Z"}
{"number": 436, "title": "Update documentation index", "files": ["docs/.sphinx/index.rst", "docs/.sphinx/links/cloud_training.rst", "docs/.sphinx/links/contributing.rst", "docs/.sphinx/links/dev_setup.rst", "docs/.sphinx/links/distributed_training.rst", "docs/.sphinx/links/usage.rst"], "area": "docs", "area_votes": {"docs": 6}, "body": "- Add details, style changes, include markdown docs from lema", "merged_at": "2024-08-30T01:51:22Z"}
{"number": 435, "title": "[tiny] Only log to console on global leader", "files": ["src/lema/utils/logging.py"], "area": "other", "area_votes": {"other": 1}, "body": "Fixes OPE-332\r\n\r\nCurrently, on GCP/Polaris, every process will log to stderr, which results in a lot of duplicate lines. Now, only the global leader will log to stderr, while every process will still log to their own separate file in the output dir.\r\n\r\nTested this on Polaris and GCP", "merged_at": "2024-08-30T02:11:46Z"}
{"number": 434, "title": "Add GitHub badges, readme typos", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "- Add GitHub badges\r\n- Fix typos from previous PR (Thanks @wizeng23 !)", "merged_at": "2024-08-29T22:51:47Z"}
{"number": 433, "title": "Fix markdown lint errors", "files": ["docs/CLOUD_TRAINING.md", "docs/DEV_SETUP.md", "docs/DISTRIBUTED_TRAINING.md", "docs/GIT_WORKFLOW.md", "docs/MODEL_DEFINITION.md"], "area": "docs", "area_votes": {"docs": 5}, "body": "- Fix additional lint errors in our docs\r\n- Towards enabling the pre-commit linter check", "merged_at": "2024-08-29T23:05:00Z"}
{"number": 432, "title": "Update main readme file", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Update main readme file", "merged_at": "2024-08-29T21:36:56Z"}
{"number": 430, "title": "[tiny] Breakdown main Readme into multiple docs", "files": ["README.md", "docs/CLOUD_TRAINING.md", "docs/DEV_SETUP.md", "docs/DISTRIBUTED_TRAINING.md"], "area": "docs", "area_votes": {"docs": 4}, "body": "- Move dev setup guide, cloud/skypilot guide, and distributed training into their own documents\r\n- To make reviewing easier, this PR does not contain any content changes. Those are coming in a follow-up PR", "merged_at": "2024-08-29T20:22:43Z"}
{"number": 429, "title": "Refresh markdown docs", "files": ["CONTRIBUTING.md", "STYLE_GUIDE.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "**Changes**\r\n- Fix linting errors detected by `markdownlint`\r\n- Fill a couple todos\r\n- Minor style changes", "merged_at": "2024-08-29T16:40:12Z"}
{"number": 428, "title": "Add Llama 70B SFT config", "files": ["configs/accelerate/llama70b.fsdp.yaml", "configs/accelerate/llama70b.lora.yaml", "configs/accelerate/llama8b.fsdp.yaml", "configs/lema/jobs/gcp/llama70b_lora.yaml", "configs/lema/jobs/gcp/llama70b_sft.yaml", "configs/lema/jobs/gcp/llama8b_lora.yaml", "configs/lema/jobs/gcp/llama8b_sft.yaml", "configs/lema/jobs/polaris/llama70b_lora.yaml", "configs/lema/jobs/polaris/llama70b_sft.yaml", "configs/lema/llama70b.sft.yaml", "configs/skypilot/sky_llama70b_lora.yaml", "configs/skypilot/sky_llama70b_sft.yaml", "configs/skypilot/sky_llama8b_lora.yaml", "configs/skypilot/sky_llama8b_sft.yaml", "notebooks/LeMa - Tuning Llama.ipynb", "scripts/polaris/jobs/llama_tune.sh"], "area": "configs", "area_votes": {"configs": 14, "docs": 1, "infra": 1}, "body": "Towards OPE-150\r\n\r\nWe get 130 s/it on 4 nodes\r\n\r\nReduced from 3 to 2 nodes for 70B lora since I don't see OOM errors on Polaris with a couple runs\r\n\r\nSwitched from pip install huggingface_hub[hf_transfer] to pip install hf_transfer since the former doesn't work on Polaris (as Matt pointed out)", "merged_at": "2024-08-29T18:33:13Z"}
{"number": 427, "title": "Add docs-rebuild command to Makefile", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "To fully re-build the docs:\r\n\r\n```bash\r\nmake docs-rebuild # cleanup everything, rebuild apidocs, rebuild html\r\nmake docs # just rebuild html\r\nmake docs-serve # serve existing html\r\n```", "merged_at": "2024-08-29T16:17:15Z"}
{"number": 426, "title": "[tiny] Move apidocs into their own folder", "files": ["docs/.sphinx/apidoc/lema.builders.rst", "docs/.sphinx/apidoc/lema.core.callbacks.rst", "docs/.sphinx/apidoc/lema.core.configs.params.rst", "docs/.sphinx/apidoc/lema.core.configs.rst", "docs/.sphinx/apidoc/lema.core.datasets.rst", "docs/.sphinx/apidoc/lema.core.launcher.rst", "docs/.sphinx/apidoc/lema.core.models.rst", "docs/.sphinx/apidoc/lema.core.rst", "docs/.sphinx/apidoc/lema.core.tokenizers.rst", "docs/.sphinx/apidoc/lema.core.trainers.rst", "docs/.sphinx/apidoc/lema.core.types.rst", "docs/.sphinx/apidoc/lema.datasets.rst", "docs/.sphinx/apidoc/lema.evaluation.rst", "docs/.sphinx/apidoc/lema.launcher.clients.rst", "docs/.sphinx/apidoc/lema.launcher.clouds.rst", "docs/.sphinx/apidoc/lema.launcher.clusters.rst", "docs/.sphinx/apidoc/lema.launcher.rst", "docs/.sphinx/apidoc/lema.models.rst", "docs/.sphinx/apidoc/lema.performance.rst", "docs/.sphinx/apidoc/lema.rst", "docs/.sphinx/apidoc/lema.utils.rst", "docs/.sphinx/apidoc/modules.rst", "docs/.sphinx/index.rst", "docs/.sphinx/tutorials.rst"], "area": "docs", "area_votes": {"docs": 24}, "body": "- To avoid confusion with the docs we manually edit.\r\n  - `sphinx-apidoc --remove-old` used to delete `tutorials` which is manually created\r\n- Minor cleanup of the tutorials doc to remove sphinx warnings", "merged_at": "2024-08-29T16:17:01Z"}
{"number": 425, "title": "Add sphinx api doc template for packages", "files": ["docs/.sphinx/_templates/apidoc/package.rst.jinja"], "area": "docs", "area_votes": {"docs": 1}, "body": "- Customize default apidoc template (found in `sphinx/templates/apidoc`) for Lema:\r\n  - Do not display submodules\r\n  - Do not display `package` | `namespace` suffix ", "merged_at": "2024-08-29T00:58:57Z"}
{"number": 424, "title": "Add automatically generated apidoc RSTs", "files": ["docs/.sphinx/index.rst", "docs/.sphinx/lema.builders.rst", "docs/.sphinx/lema.core.callbacks.rst", "docs/.sphinx/lema.core.configs.params.rst", "docs/.sphinx/lema.core.configs.rst", "docs/.sphinx/lema.core.datasets.rst", "docs/.sphinx/lema.core.launcher.rst", "docs/.sphinx/lema.core.models.rst", "docs/.sphinx/lema.core.rst", "docs/.sphinx/lema.core.tokenizers.rst", "docs/.sphinx/lema.core.trainers.rst", "docs/.sphinx/lema.core.types.rst", "docs/.sphinx/lema.datasets.rst", "docs/.sphinx/lema.evaluation.rst", "docs/.sphinx/lema.launcher.clients.rst", "docs/.sphinx/lema.launcher.clouds.rst", "docs/.sphinx/lema.launcher.clusters.rst", "docs/.sphinx/lema.launcher.rst", "docs/.sphinx/lema.models.rst", "docs/.sphinx/lema.performance.rst", "docs/.sphinx/lema.rst", "docs/.sphinx/lema.utils.rst", "docs/.sphinx/modules.rst"], "area": "docs", "area_votes": {"docs": 23}, "body": "**Changes**\r\n- Refresh apidoc RST files. \r\n- These are now automatically generated without any manual edits using the following command:\r\n```bash\r\ncd lema/docs/.sphinx\r\nsphinx-apidoc -o ./ ../../src/lema --remove-old --force --module-first --implicit-namespaces  --maxdepth 2 -t _templates/apidoc\r\n```", "merged_at": "2024-08-29T00:59:12Z"}
{"number": 423, "title": "Script to generate judge prompts.", "files": ["scripts/llm-as-judge/generate_prompts.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "", "merged_at": "2024-08-29T20:13:58Z"}
{"number": 420, "title": "Cleanup doc RSTs", "files": ["docs/.sphinx/lema.builders.rst", "docs/.sphinx/lema.core.configs.rst", "docs/.sphinx/lema.core.datasets.rst", "docs/.sphinx/lema.core.launcher.rst", "docs/.sphinx/lema.core.models.rst", "docs/.sphinx/lema.core.tokenizers.rst", "docs/.sphinx/lema.core.trainers.rst", "docs/.sphinx/lema.core.types.rst", "docs/.sphinx/lema.evaluation.rst", "docs/.sphinx/lema.models.rst"], "area": "docs", "area_votes": {"docs": 10}, "body": "**Changes**\r\n- Remove hardcoded overview section from RST files, as it's redundant with the docstrings\r\n- Prefer generating docs from existing docstrings, as it's more convenient that editing the auto-generated RSTs\r\n\r\n**Example after change**\r\n![image](https://github.com/user-attachments/assets/1c89b482-ff8e-4045-98ec-68d8f030a9e1)\r\n", "merged_at": "2024-08-28T23:14:48Z"}
{"number": 419, "title": "[tiny] Enable D104 rule", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "**Changes**\r\n- Adding docstrings for packages is now required going forward\r\n- PR #415 fixed docs for all current packages\r\n\r\nTowards OPE-326", "merged_at": "2024-08-28T20:28:53Z"}
{"number": 418, "title": "Update the CLI to look for open SSH tunnels as a way of preserving Polaris state", "files": ["src/lema/launcher/clients/polaris_client.py", "src/lema/launcher/clouds/polaris_cloud.py", "tests/launcher/clients/test_polaris_client.py", "tests/launcher/clouds/test_polaris_cloud.py"], "area": "launcher", "area_votes": {"launcher": 2}, "body": "Running the launcher via a python client is stateful as the main python process persists throughout the session.\r\nHowever, when running via the CLI, the python session is terminated immediately upon the command existing.\r\nTo enable subsequent polaris commands, (status, getjob, etc) I've updated the polaris cloud to look for open SSH tunnels on init and create clusters as necessary.\r\n\r\n\r\nTowards OPE-283", "merged_at": "2024-08-28T22:34:02Z"}
{"number": 417, "title": "Improve launcher polling by running tasks in a subprocess.", "files": ["src/lema/launch.py", "tests/test_launch.py"], "area": "other", "area_votes": {"other": 1}, "body": "Previously printing and polling were done in the same thread. This caused our pretty-printed message on the console to animate much slower than intended.\r\n\r\nI've moved execution to a thread to ensure printing happens as expected.\r\n\r\nTowards OPE-283", "merged_at": "2024-08-28T20:02:06Z"}
{"number": 416, "title": "[tiny] sphinx conf update", "files": ["docs/.sphinx/conf.py"], "area": "docs", "area_votes": {"docs": 1}, "body": "**Changes**\r\n- Include magic function in documentation: we use a few magic functions as part of our regular code paths (e.g. `__getitem__` for datasets), which are excluded by default. This change enables docstring generation for magic functions\r\n- Add logging for docstring generation (not very informative now, but will be useful when doctests are enabled in a future PR)", "merged_at": "2024-08-28T19:43:25Z"}
{"number": 414, "title": "Script to run inference with Llama/GPT judges.", "files": ["scripts/llm-as-judge/judge_dataset.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "", "merged_at": "2024-08-28T16:46:07Z"}
{"number": 412, "title": "Fix a deadlock in the Polaris launcher for users with 500+ jobs.", "files": ["src/lema/launcher/clients/polaris_client.py", "tests/launcher/clients/test_polaris_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Per the python [docs](https://docs.python.org/3/library/subprocess.html#subprocess.Popen.wait) `Popen` has a potential deadlock:\r\n```\r\nThis will deadlock when using stdout=PIPE or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use [Popen.communicate()](https://docs.python.org/3/library/subprocess.html#subprocess.Popen.communicate) when using pipes to avoid that.\r\n```\r\n\r\nThis is resolved by using `run` instead.", "merged_at": "2024-08-28T16:25:42Z"}
{"number": 411, "title": "Update sphinx comments to docstrings", "files": ["src/lema/core/configs/async_evaluation_config.py", "src/lema/core/configs/job_config.py"], "area": "configs", "area_votes": {"configs": 2}, "body": "`job_config.py` and `async_evaluation_config.py` already have comments, this PR reformats as docstrings.", "merged_at": "2024-08-27T23:41:41Z"}
{"number": 410, "title": "Add missing docstrings to top-level configs", "files": ["src/lema/core/configs/evaluation_config.py", "src/lema/core/configs/generation_config.py", "src/lema/core/configs/inference_config.py", "src/lema/core/configs/training_config.py"], "area": "configs", "area_votes": {"configs": 4}, "body": "Add missing docstrings to top-level configs", "merged_at": "2024-08-28T17:02:50Z"}
{"number": 409, "title": "Add missing docstrings to TrainingParams", "files": ["src/lema/core/configs/params/training_params.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "- Add missing docstrings to TrainingParams\r\n- Reformat existing sphinx style comments into multiline docstrings", "merged_at": "2024-08-27T23:42:05Z"}
{"number": 407, "title": "Autostop sky clusters after 30 min of no activity", "files": ["src/lema/core/launcher/base_cloud.py", "src/lema/launcher/clients/sky_client.py", "src/lema/launcher/clouds/local_cloud.py", "src/lema/launcher/clouds/polaris_cloud.py", "src/lema/launcher/clouds/sky_cloud.py", "src/lema/launcher/launcher.py", "tests/launcher/clients/test_sky_client.py", "tests/launcher/clouds/test_sky_cloud.py", "tests/launcher/test_launcher.py"], "area": "launcher", "area_votes": {"launcher": 6}, "body": "Set sky clusters to autostop after 30 minutes of no activity. A cluster will not stop while a job is actively running.\r\n\r\nFixes OPE-317", "merged_at": "2024-08-27T20:43:15Z"}
{"number": 406, "title": "Ensure that jobs are queued on existing clusters when users call UP", "files": ["src/lema/launch.py", "tests/test_launch.py"], "area": "other", "area_votes": {"other": 1}, "body": "If the user specifies a cluster in their UP command, we should reuse existing clusters if the names match.\r\n\r\nNote that this ignores the resources in their job config for the new job, but this is simply a nicety via our CLI. The user can work around this by switching cluster names.\r\n\r\nFixes OPE-323", "merged_at": "2024-08-27T18:33:32Z"}
{"number": 403, "title": "Add docs and gpu install targets", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "- Add new install targets: `docs` and `gpu`\r\n  - `docs` contains the dependencies that are only used for docs generation\r\n   - The GPU target contains all dependencies that fail to install on CPU. ", "merged_at": "2024-08-27T16:31:06Z"}
{"number": 402, "title": "Increase from 2 to 3 nodes for Llama 70B Lora", "files": ["configs/accelerate/llama70b.lora.yaml", "configs/lema/jobs/polaris/llama70b_lora.yaml", "configs/lema/llama70b.lora.yaml", "configs/skypilot/sky_llama70b_lora.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "Towards OPE-309\r\n\r\nThis is to avoid OOM issues which show up occasionally with 2 nodes on GCP and Polaris. It's flaky with the same config, and I'm unsure of the reason, but there's more memory breathing room with 3 nodes. However, the downside is scheduling this job in Polaris can get much harder since queue times are longer outside the debug queues.\r\n\r\nThis also adds in k_proj, which was in the torchtune reference config but I didn't add in earlier PRs: https://github.com/pytorch/torchtune/blob/main/recipes/configs/llama3_1/70B_lora.yaml", "merged_at": "2024-08-27T15:06:04Z"}
{"number": 400, "title": "Add vscode launch config for accelerate distributed training", "files": [".vscode/launch.json"], "area": "other", "area_votes": {"other": 1}, "body": "- Add vscode launch config for accelerate distributed training\r\n- This can be used to debug distributed jobs launched with accelerate in vscode debugger", "merged_at": "2024-08-26T23:42:33Z"}
{"number": 399, "title": "Update trainer save model", "files": ["src/lema/core/trainers/base_trainer.py", "src/lema/core/trainers/hf_trainer.py", "src/lema/core/trainers/lema_trainer.py", "src/lema/train.py"], "area": "training", "area_votes": {"training": 4}, "body": "Towards OPE-311\r\n\r\n**Changes**\r\n- Remove redundant `is_world_process_zero` checks\r\n- With FSDP, save the final checkpoint in `FULL_STATE_DICT` format\r\n\r\n\r\n**Usage**\r\n- Training llama 8b for 10 steps, 1 full state checkpoint, 1 full state final save\r\n```\r\nreal    3m48.424s\r\nuser    27m7.338s\r\nsys     2m40.138s\r\n```\r\n- Training llama 8b for 10 steps, 1 sharded checkpoint, 1 full state final save\r\n```\r\nreal    5m50.184s\r\nuser    12m40.055s\r\nsys     2m45.310s\r\n```\r\n", "merged_at": "2024-08-27T01:07:40Z"}
{"number": 398, "title": "Fix FSDP model initialization", "files": ["src/lema/builders/models.py"], "area": "other", "area_votes": {"other": 1}, "body": "Towards OPE-309\r\n\r\nWe should set device_map to None to let Accelerate init FSDP models properly. This change resolves the OOM we experienced.\r\n\r\nI chose to detect FSDP using the ACCELERATE_USE_FSDP env variable. The other method is using `trainer.is_fsdp_enabled` for the transformer and TRL trainers, but the trainer hasn't been initialized yet at the time of model initialization.", "merged_at": "2024-08-26T23:01:40Z"}
{"number": 396, "title": "Update sample commands to point to the preemptable queue", "files": ["configs/lema/jobs/polaris/llama8b_lora.yaml", "configs/lema/jobs/polaris/llama8b_sft.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "8b models should not be run on the debug queue if they require more than 1hr walltime.\r\n\r\n\r\nReference on walltimes: https://docs.alcf.anl.gov/polaris/running-jobs/", "merged_at": "2024-08-26T19:33:25Z"}
{"number": 395, "title": "Check ignored docstring rules", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Our auto-generated docstrings are fairly bare. We have several rules that are ignored that we need to iteratively fix: D100, D101, D104. \r\n\r\nOur codebase contains 130 violations total.\r\n\r\nTowards OPE-326", "merged_at": "2024-08-26T17:21:45Z"}
{"number": 393, "title": "Update Llama notebook to include 8B SFT", "files": ["notebooks/LeMa - Tuning Llama.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "Towards OPE-145\r\n\r\nWhile we have configs for 70B LoRA, there's currently issues getting it to train.", "merged_at": "2024-08-26T18:22:18Z"}
{"number": 391, "title": "Raise `NOT_IMPLEMENTED` if `adapter_model` is configured for `LM_HARNESS` eval", "files": ["src/lema/evaluate.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "\r\n\r\nTowards OPE-313", "merged_at": "2024-08-24T01:39:08Z"}
{"number": 390, "title": "Fix missing new line at the end of `Makefile`", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "", "merged_at": "2024-08-24T01:26:11Z"}
{"number": 388, "title": "Add Llama 70b lora config", "files": ["configs/accelerate/llama70b.lora.yaml", "configs/lema/jobs/gcp/llama70b_lora.yaml", "configs/lema/jobs/gcp/llama8b_lora.yaml", "configs/lema/jobs/polaris/llama70b_lora.yaml", "configs/lema/jobs/polaris/llama8b_lora.yaml", "configs/lema/jobs/polaris/llama8b_sft.yaml", "configs/lema/llama70b.lora.yaml", "configs/skypilot/sky_llama70b_lora.yaml", "scripts/polaris/jobs/llama8b_lora.sh", "scripts/polaris/jobs/llama_tune.sh", "scripts/polaris/jobs/multinode_example_worker.sh"], "area": "configs", "area_votes": {"configs": 8, "infra": 3}, "body": "Fixes OPE-309\r\n\r\nAlso updates the Llama polaris worker bash script to support 8b/70b and sft/lora.", "merged_at": "2024-08-24T23:59:59Z"}
{"number": 387, "title": "Add docs-serve makefile command", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "Add a new makefile command to serve the documentation locally.\r\n\r\n**Usage**\r\n```bash\r\nmake docs  # generate the docs\r\n\r\nmake docs-serve  # launch a local http server with the docs\r\n```", "merged_at": "2024-08-23T23:55:24Z"}
{"number": 386, "title": "Update main makefile to generate docs", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "Update main Makefile to generate docs.\r\n\r\nTo use it:\r\n```make docs```\r\n\r\nTo see all the options:\r\n```make docs-help```", "merged_at": "2024-08-23T23:48:04Z"}
{"number": 385, "title": "Update SkyPilot GCP script to download  the right model version", "files": ["configs/skypilot/sky_llama8b_lora.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "\r\nTowards  OPE-312", "merged_at": "2024-08-23T22:10:11Z"}
{"number": 384, "title": "Remove special case for saving PEFT models", "files": ["src/lema/core/trainers/hf_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Switch to using  `_hf_trainer.save_model(output_dir)` in HF trainer path\r\n-- In my test, both variants produce files of same size in output directory, so I think it's best to use a public method `save_model()` (?)\r\n\r\nTowards OPE-213\r\nFixes OPE-313\r\n\r\nBEFORE:\r\n```\r\n\"-rw-r--r-- 1 community_ai     662 Aug 23 21:29 adapter_config.json\r\n-rw-r--r-- 1 community_ai 6832728 Aug 23 21:29 adapter_model.safetensors\r\ndrwxr-sr-x 2 community_ai    4096 Aug 23 21:28 logs\r\ndrwxr-sr-x 3 community_ai    4096 Aug 23 21:28 profiler\r\n-rw-r--r-- 1 community_ai    5111 Aug 23 21:29 README.md\r\n-rw-r--r-- 1 community_ai     325 Aug 23 21:29 special_tokens_map.json\r\n-rw-r--r-- 1 community_ai   55380 Aug 23 21:29 tokenizer_config.json\r\n-rw-r--r-- 1 community_ai 9085657 Aug 23 21:29 tokenizer.json\r\n-rw-r--r-- 1 community_ai    1108 Aug 23 21:29 trainer_state.json\r\n-rw-r--r-- 1 community_ai    5560 Aug 23 21:29 training_args.bin\"\r\n```\r\n\r\nAFTER:\r\n```\r\n\"-rw-r--r-- 1 community_ai     662 Aug 23 21:38 adapter_config.json\r\n-rw-r--r-- 1 community_ai 6832728 Aug 23 21:38 adapter_model.safetensors\r\ndrwxr-sr-x 2 community_ai    4096 Aug 23 21:36 logs\r\ndrwxr-sr-x 3 community_ai    4096 Aug 23 21:37 profiler\r\n-rw-r--r-- 1 community_ai    5111 Aug 23 21:38 README.md\r\n-rw-r--r-- 1 community_ai     325 Aug 23 21:38 special_tokens_map.json\r\n-rw-r--r-- 1 community_ai   55380 Aug 23 21:38 tokenizer_config.json\r\n-rw-r--r-- 1 community_ai 9085657 Aug 23 21:38 tokenizer.json\r\n-rw-r--r-- 1 community_ai    1108 Aug 23 21:38 trainer_state.json\r\n-rw-r--r-- 1 community_ai    5560 Aug 23 21:38 training_args.bin\"\r\n```\r\n\r\n", "merged_at": "2024-08-26T17:52:27Z"}
{"number": 383, "title": "Clean up Sky configs", "files": ["README.md", "configs/lema/chatqa/chatqa.stage1.yaml", "configs/lema/chatqa/chatqa.stage2.yaml", "configs/lema/jobs/gcp/hello_world.yaml", "configs/lema/jobs/gcp/llama2b_fsdp.yaml", "configs/lema/jobs/gcp/llama8b_lora.yaml", "configs/lema/jobs/polaris/hello_world.yaml", "configs/lema/jobs/polaris/llama8b_lora.yaml", "configs/lema/jobs/polaris/multinode_example.yaml", "configs/lema/jobs/polaris/vllm.yaml", "configs/skypilot/sky_chatqa.yaml", "configs/skypilot/sky_eval.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_gpt2_eval.yaml", "configs/skypilot/sky_gpt2_fsdp.yaml", "configs/skypilot/sky_init.sh", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama2b_ddp.yaml", "configs/skypilot/sky_llama2b_deepspeed.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_llama3_8b_aya_eval.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml", "configs/skypilot/sky_llama70b_fsdp.yaml", "configs/skypilot/sky_llama8b_lora.yaml", "configs/skypilot/sky_llama8b_sft.yaml", "configs/skypilot/sky_phi3_dpo.yaml", "configs/skypilot/sky_phi3_dpo_fsdp.yaml", "configs/skypilot/sky_phi3_eval.yaml", "configs/skypilot/sky_ssh.yaml", "configs/skypilot/sky_zephyr_7b_qlora_sft.yaml", "configs/skypilot/sky_zephyr_7b_sft.yaml", "notebooks/LeMa - A Tour.ipynb", "notebooks/LeMa - Deploying a Job.ipynb", "notebooks/LeMa - Running Jobs Remotely.ipynb", "scripts/polaris/jobs/llama8b_lora.sh", "scripts/polaris/jobs/multinode_example_worker.sh", "scripts/polaris/jobs/vllm_job.sh", "scripts/polaris/jobs/vllm_worker.sh", "src/experimental/pretokenize/sky.yaml"], "area": "configs", "area_votes": {"docs": 4, "configs": 30, "infra": 4, "other": 1}, "body": "Fixes OPE-319\r\n\r\nLarge PR to clean up Sky configs. Please provide feedback if I should be keeping anything I'm deleting!\r\nFor full disclosure, I did not test my changes by running the configs since so many were updated, but I did read through the whole PR to verify all changes were intended. IMO I'm not too worried about any potential breakage since it mainly affects old Sky configs, but if you want anything tested please LMK.\r\n\r\nList of all functional changes:\r\n- Set WANDB_PROJECT to lema-train or lema-eval (seems like better names than `lema-train-test` which we had before)\r\n- Changed reference to 8B to 8B instruct in `configs/lema/jobs/gcp/llama8b_lora.yaml`\r\n- Deleted reference to runpod flash attention install and switched all configs to normal install, since we don't run on Runpod anymore\r\n- Renamed `gpt2.chatqa.stage*.yaml` to `chatqa.stage*.yaml` since they currently aren't using gpt\r\n- Renamed `sky.yaml` to `sky_phi3_dpo.yaml`, and `sky_eval.yaml` to `sky_phi3_eval.yaml`\r\n- Changed around some config values in old configs we had (ex. gpt2) to reasonable values, ex. settting 2 GPUs for Llama2b DDP\r\n- Changed config run names away from `lema-train-example`\r\n- Removed `pip install '.[train]'` calls in the run section, which should only be used during debugging\r\n- Renamed POLARIS_GPUS_PER_NODE to POLARIS_NUM_GPUS_PER_NODE to be in line with SKYPILOT_NUM_GPUS_PER_NODE\r\n\r\nChanges that don't affect behavior:\r\n\r\n- Removed comments from Sky configs that were adding a lot of bloat. The relevant information can be found in our [README](https://github.com/openlema/lema/blob/main/README.md) or https://github.com/openlema/lema/wiki/Clouds-Setup.\r\n- Removed out commented params that we likely won't use (ex. `/artifacts lema-dev-private`).\r\n- Made all configs as consistent as possible with each other\r\n- Add num_nodes to FSDP/Deepspeed configs to hint how to do multi-node training\r\n- Added more details to our README\r\n", "merged_at": "2024-08-23T22:37:41Z"}
{"number": 382, "title": "Minor tuning of llama8b configs", "files": ["configs/lema/llama8b.lora.yaml", "configs/lema/llama8b.sft.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "-- Reduce LoRA batch size from 3 to 2.  3 is still flakey: still fails sometimes on GCP but works OK on Polaris (?). Also, for consistency with `torchtune`'s  `bs=2x32`\r\n-- Increase `save_steps` from `100` to `200`\r\n-- Auto-formatting \r\n\r\nTowards OPE-313, OPE-312", "merged_at": "2024-08-23T20:19:49Z"}
{"number": 380, "title": "Update llama8b GCP launcher script to allow Spot VMs", "files": ["configs/lema/jobs/gcp/llama8b_lora.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Spot VM-s for A100 40GB GPUs are plentiful on GCP, and cost ~3X less.\r\n\r\nTowards OPE-149", "merged_at": "2024-08-23T16:50:06Z"}
{"number": 379, "title": "Add Llama 8b SFT config", "files": ["configs/accelerate/llama8b.fsdp.yaml", "configs/lema/jobs/polaris/llama8b_lora.yaml", "configs/lema/llama3.8b.aya.eval.yaml", "configs/lema/llama3.8b.aya.sft.yaml", "configs/lema/llama8b.lora.yaml", "configs/lema/llama8b.sft.yaml", "configs/skypilot/sky_llama8b_sft.yaml", "src/lema/core/trainers/hf_trainer.py"], "area": "configs", "area_votes": {"configs": 7, "training": 1}, "body": "Found out that the final model save is a no-op for FSDP with SHARDED_STATE_DICT. However, trying to save the full state dict times out with 8b. Will fix in a later PR. Setting FULL_STATE_DICT for now is faster across the board than SHARDED_STATE_DICT", "merged_at": "2024-08-23T19:00:18Z"}
{"number": 378, "title": "Add `TelemetryCallback.include_timer_metrics` param: `False` by default", "files": ["src/lema/core/callbacks/telemetry_callback.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- The flag controls whether to add timer stats to reported metrics.  The timings stats can be verbose/distracting, and only useful in limited cases, so `False` by default.  The timings will be written to a file at the end of training regardless  of the value of this flag.\r\n\r\nTowards OPE-254, OPE-293", "merged_at": "2024-08-23T16:17:31Z"}
{"number": 374, "title": "Initial notebook for llama 8b LoRa tuning.", "files": ["notebooks/LeMa - Tuning Llama.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "Add a brief notebook with a description on how to run Llama 8b LoRA tuning.\r\n\r\nThis notebook will be updated as folks finalize the best configurations for each flavor of Llama given our experiments.\r\n\r\nTowards OPE-145", "merged_at": "2024-08-23T20:47:54Z"}
{"number": 373, "title": "Add `empty_device_cache_steps` param and configure it for Llama8b model", "files": ["configs/lema/llama8b.lora.yaml", "src/lema/core/configs/params/training_params.py"], "area": "configs", "area_votes": {"configs": 2}, "body": "-- It helps to keep VRAM usage under control.\r\n-- Otherwise, mem usage slowly grows for the model and leads to CUDA OOM after ~140 steps with bs=3.\r\n\r\n\r\nTested: https://wandb.ai/lema-train-test/lema-train-test/runs/7osxgqb4?nw=nwuserxrdaukar\r\n![EmptyCacheEffect_Capture](https://github.com/user-attachments/assets/8c0738e1-2343-4da1-a112-2c2148135e2f)\r\n\r\nThe periodic drops in memory usage correspond to `torch.cuda.empty_cache()` calls\r\n\r\nTowards OPE-149, OPE-312", "merged_at": "2024-08-23T00:58:24Z"}
{"number": 372, "title": "Disable MFU computation for PEFT", "files": ["src/lema/train.py"], "area": "training", "area_votes": {"training": 1}, "body": "Otherwise, getting insane numbers:\r\n\r\n```\r\n(lema-train-example, pid=5692) wandb:                            train/train_mfu 238.27972\r\n(lema-train-example, pid=5692) wandb:                       train/train_step_mfu 258.8923\r\n```\r\n\r\nTowards OPE-149\r\nFixes OPE-230", "merged_at": "2024-08-23T00:57:39Z"}
{"number": 370, "title": "Add vllm parallel inference to improve throughput", "files": ["scripts/polaris/jobs/vllm_parallel_inference.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "Adds MultithreadedRatelimitedClient and reworks inference to be parallelized.\r\n\r\nFor quantized 70B:\r\n1 thread: 25:29 / 100 samples\r\n50 threads: 1:22 / 100 samples\r\n\r\nTried 100 threads but can't create that many. Likely 50 is enough.\r\n\r\nTowards OPE-314", "merged_at": "2024-08-22T22:39:40Z"}
{"number": 369, "title": "Fix interpolation when using the launcher CLI for various sky configs.", "files": ["src/lema/core/configs/base_config.py", "tests/integration/configs/test_parse_configs.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "Similar fix as #314 but applied to `from_yaml_and_arg_list`\r\n\r\nFixes OPE-285", "merged_at": "2024-08-22T17:31:35Z"}
{"number": 366, "title": "jgreer013/vllm-inference", "files": ["scripts/polaris/jobs/build_apptainer_from_docker.sh", "scripts/polaris/jobs/download_model_from_hf.sh", "scripts/polaris/jobs/vllm_inference.py", "scripts/polaris/jobs/vllm_job.sh", "scripts/polaris/jobs/vllm_worker.sh"], "area": "infra", "area_votes": {"infra": 5}, "body": "Add polaris scripts for vllm inference (and helper scripts)", "merged_at": "2024-08-22T15:10:11Z"}
{"number": 365, "title": "Minor update to Llama70B", "files": ["configs/accelerate/llama70b.fsdp.yaml", "configs/lema/llama70b.pt.yaml", "configs/skypilot/sky_llama70b_fsdp.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "Towards OPE-150\r\n\r\nTo get Llama70B working, you need to set `device_map=\"cpu\"` in `transformers.AutoModelForCausalLM.from_pretrained` in models.py. Finding a way to programatically do this for FSDP runs will take more experimentation, and I plan to do that in a separate PR.\r\n\r\nWe get ~60s/it with this setup, while torchtune gets 12s/it, using as similar of a config as possible on the same cluster. It's possible they're not packing their data, which could be the difference.", "merged_at": "2024-08-21T19:38:38Z"}
{"number": 364, "title": "Reorder model compilation and DDP/FSDP wrapping", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Per DDP docs `torch.compile` should be called after DDP wrapping: https://pytorch.org/docs/stable/notes/ddp.html#example\r\n-- FSDP example: https://github.com/foundation-model-stack/fms-fsdp/blob/948cb2a97ebcbec066d880dce618f6fe917372f3/main_training.py#L110\r\n\r\n\r\nTowards OPE-296", "merged_at": "2024-08-21T19:40:07Z"}
{"number": 363, "title": "Basic plumbing for GPU temperature telemetry", "files": ["src/lema/core/callbacks/telemetry_callback.py", "src/lema/core/trainers/lema_trainer.py", "src/lema/performance/telemetry.py", "tests/performance/test_telemetry.py"], "area": "training", "area_votes": {"training": 3}, "body": "-- Update `TelemetryCallback` to store GPU temperatures, and report temperature stats as a metric (controlled by the new `track_gpu_temperature` param)\r\n-- Update LEMA training loop to log GPU temps.\r\n-- Update `Telemetry.print_summary()` to print stats as a single value (so stats from different ranks aren't interleaved) \r\n\r\nTowards OPE-252", "merged_at": "2024-08-21T17:29:46Z"}
{"number": 362, "title": "Minor improvements in logging and instrumentations in `train.py`", "files": ["src/lema/train.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- No functionality changes\r\n\r\nTowards OPE-252", "merged_at": "2024-08-21T16:24:34Z"}
{"number": 361, "title": "Add Llama70B FSDP config", "files": ["configs/accelerate/llama70b.fsdp.yaml", "configs/lema/llama2b.pt.yaml", "configs/lema/llama70b.pt.yaml", "configs/lema/llama8b.pt.yaml", "configs/skypilot/sky_llama70b_fsdp.yaml", "configs/skypilot/sky_llama8b.yaml"], "area": "configs", "area_votes": {"configs": 6}, "body": "Towards OPE-150\r\n\r\nThis config doesn't work yet (OOMs on model instantiation), but is being checked in so that others can reproduce my findings.", "merged_at": "2024-08-21T01:41:56Z"}
{"number": 360, "title": "Add another `bareer()` call before train()", "files": ["src/lema/train.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Making sure all workers start training at the same time approximately", "merged_at": "2024-08-20T23:40:35Z"}
{"number": 358, "title": "Add Llama 8B config", "files": ["configs/lema/jobs/gcp/llama2b_fsdp.yaml", "configs/lema/jobs/gcp/llama8b.yaml", "configs/lema/llama2b.pt.yaml", "configs/lema/llama8b.pt.yaml", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama2b_ddp.yaml", "configs/skypilot/sky_llama2b_deepspeed.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_llama8b.yaml", "notebooks/LeMa - Running Jobs Remotely.ipynb", "scripts/polaris/jobs/multinode_example_worker.sh"], "area": "configs", "area_votes": {"configs": 9, "docs": 1, "infra": 1}, "body": "Towards OPE-149\r\n\r\nThis is just an initial config that should run. I haven't optimized the params to maximize tok/s yet.\r\n\r\nAlso moved some Llama2b param values which aren't often changed during experimentation from skypilot configs to the base configs since the sky config commands were getting quite long.", "merged_at": "2024-08-20T22:44:01Z"}
{"number": 356, "title": "Define `ddp1gpu` Polaris mode: Spawn 1 `torchrun` process per GPU (4 `torchrun`-s per node)", "files": ["scripts/polaris/jobs/multinode_example_job.sh", "scripts/polaris/jobs/multinode_example_worker.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "-- In `ddp1gpu` mode: GPU visibility is limited to 1 GPU for DDP as recommended in [Polaris docs](https://docs.alcf.anl.gov/polaris/data-science-workflows/frameworks/pytorch/#multi-gpu-multi-node-scale-up)\r\n-- Tested on 1 and 2 nodes so far: In my initial test, the setup yields roughly the same DDP speed as before. Still adding for more validation.\r\n\r\nTowards OPE-231, OPE-252", "merged_at": "2024-08-20T16:44:11Z"}
{"number": 351, "title": "Update LEMA training loop to log wandb url", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "Towards OPE-257", "merged_at": "2024-08-19T18:22:11Z"}
{"number": 350, "title": "Fix a minor bug in `TelemetryCallback.on_train_end`", "files": ["src/lema/core/callbacks/telemetry_callback.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Remove one irrelevant condition, which is always false\r\n\r\nTowards OPE-254, OPE-293", "merged_at": "2024-08-17T00:06:14Z"}
{"number": 349, "title": "Various improvements for our autogenerated docs", "files": ["docs/.sphinx/conf.py", "docs/.sphinx/index.rst", "docs/.sphinx/lema.builders.rst", "docs/.sphinx/lema.core.datasets.rst", "docs/.sphinx/lema.core.rst", "docs/.sphinx/lema.core.types.rst", "docs/.sphinx/lema.datasets.rst", "docs/.sphinx/lema.evaluation.rst", "docs/.sphinx/lema.launcher.rst", "docs/.sphinx/lema.models.rst", "docs/.sphinx/lema.rst", "docs/.sphinx/modules.rst", "docs/.sphinx/tutorials.rst"], "area": "docs", "area_votes": {"docs": 13}, "body": "Various docs improvements:\r\n\r\n- Added a summary to all packages\r\n- Added a tutorial page pointing to our notebooks, broken down by topic.\r\n- Removed unprofessional styling\r\n\r\n\r\nTowards OPE-300\r\n\r\n\"Screenshot\r\n", "merged_at": "2024-08-16T23:02:53Z"}
{"number": 348, "title": "Polaris: update sample `tail` command to use `-n200`", "files": ["scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "Towards OPE-231", "merged_at": "2024-08-16T23:13:54Z"}
{"number": 347, "title": "Polaris: Enable NCCL debug logging at WARNING level", "files": ["scripts/polaris/jobs/multinode_example_job.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "\r\nTowards OPE-231", "merged_at": "2024-08-16T22:17:56Z"}
{"number": 345, "title": "Add a new notebook for getting started.", "files": ["README.md", "notebooks/LeMa - A Tour.ipynb", "notebooks/LeMa - Finetuning Tutorial.ipynb", "notebooks/LeMa - Running Jobs Remotely.ipynb"], "area": "docs", "area_votes": {"docs": 4}, "body": "Add a new notebook for getting started. This notebook give a brief overview of LeMa's python APIs.\r\n\r\nUpdated several other notebooks to remove field output.\r\n\r\nTowards OPE-145", "merged_at": "2024-08-16T22:18:26Z"}
{"number": 344, "title": "Add a link to our documentation via the readme.", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Add a link to our hosted documentation via our README.\r\n\r\nTowards OPE-300", "merged_at": "2024-08-16T16:27:48Z"}
{"number": 343, "title": "Create `TelemetryCallback`", "files": ["src/lema/core/callbacks/telemetry_callback.py", "src/lema/performance/telemetry.py", "src/lema/train.py"], "area": "training", "area_votes": {"training": 3}, "body": "-- Computes sub-steps/steps/epochs timings, and reports them as metrics\r\n-- Can also save the metrics to JSON files \r\n\r\nTowards OPE-254, OPE-293\r\n", "merged_at": "2024-08-16T22:39:08Z"}
{"number": 342, "title": "Enable gradient scaling for fp16 mixed-precision training", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "We had the correct GradScaler calls in the Lema trainer code before, but didn't enable it.", "merged_at": "2024-08-16T08:24:25Z"}
{"number": 341, "title": "Disable weight decay for layernorm/biases in Lema trainer", "files": ["src/lema/builders/models.py", "src/lema/builders/optimizers.py", "src/lema/utils/torch_naming_heuristics.py", "src/lema/utils/torch_utils.py", "tests/utils/test_torch_naming_heuristics.py"], "area": "other", "area_votes": {"other": 4}, "body": "Fixes OPE-301\r\n\r\nWe were previously weight-decaying all trainable params, but we should be skipping biases and layernorms. I'm referencing this implementation: https://huggingface.co/docs/transformers/main/en/perf_train_gpu_one#8-bit-adam\r\n\r\nHF essentially has the same logic behind the scenes: https://github.com/huggingface/transformers/blob/ab7e893d09285e0b235e4886401cfab93487169c/src/transformers/trainer.py#L1054\r\n\r\nA similar approach in nanoGPT: https://github.com/karpathy/build-nanogpt/blob/6104ab1b53920f6e2159749676073ff7d815c1fa/train_gpt2.py#L179\r\n\r\n", "merged_at": "2024-08-16T19:09:46Z"}
{"number": 340, "title": "Minor doc formatting updates.", "files": ["docs/.sphinx/lema.core.rst", "docs/.sphinx/lema.core.types.rst", "docs/.sphinx/lema.datasets.rst", "docs/.sphinx/lema.evaluation.rst", "docs/.sphinx/lema.launcher.rst", "docs/.sphinx/lema.models.rst", "docs/.sphinx/lema.rst", "src/lema/launcher/launcher.py"], "area": "docs", "area_votes": {"docs": 7, "launcher": 1}, "body": "Minor doc formatting updates so we don't show duplicate class definitions for subpackages / submodules.\r\n\r\nTowards OPE-300", "merged_at": "2024-08-15T23:14:20Z"}
{"number": 339, "title": "Add 8-bit Adam optimizer to Lema trainer", "files": ["src/lema/builders/optimizers.py"], "area": "other", "area_votes": {"other": 1}, "body": "Fixes OPE-302\r\n\r\nThese names match those used by HF.", "merged_at": "2024-08-15T23:43:18Z"}
{"number": 338, "title": "Update  ProfilerStepCallback to add `microstep` profiler annotations", "files": ["src/lema/core/callbacks/profiler_step_callback.py", "src/lema/train.py"], "area": "training", "area_votes": {"training": 2}, "body": "-- Explicit microstep marks can be useful to directly see step durations, associate other events, etc \r\n\r\nTowards OPE-292", "merged_at": "2024-08-15T19:56:43Z"}
{"number": 337, "title": "Fix dataclass strings to be parsable by our docs generator.", "files": ["src/lema/core/types/base_cluster.py", "src/lema/core/types/configs.py", "src/lema/core/types/params/data_params.py", "src/lema/core/types/params/job_resources.py", "src/lema/core/types/params/model_params.py", "src/lema/core/types/params/profiler_params.py", "src/lema/core/types/params/training_params.py"], "area": "data", "area_votes": {"data": 7}, "body": "Comments will be explicitly picked up if prefaced with `#:`\r\n\r\nSamples of when docstrings will be picked up:\r\n\r\n```python\r\nclass Foo:\r\n    \"\"\"Docstring for class Foo.\"\"\"\r\n\r\n    #: Doc comment for class attribute Foo.bar.\r\n    #: It can have multiple lines.\r\n    bar = 1\r\n\r\n    flox = 1.5   #: Doc comment for Foo.flox. One line only.\r\n\r\n    baz = 2\r\n    \"\"\"Docstring for class attribute Foo.baz.\"\"\"\r\n\r\n    def __init__(self):\r\n        #: Doc comment for instance attribute qux.\r\n        self.qux = 3\r\n\r\n        self.spam = 4\r\n        \"\"\"Docstring for instance attribute spam.\"\"\"\r\n```\r\n(source: https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#directive-autoattribute)\r\nTowards OPE-300", "merged_at": "2024-08-15T19:32:18Z"}
{"number": 335, "title": "Set up Sphinx-based doc generation for LeMa", "files": [".gitignore", "docs/.sphinx/Makefile", "docs/.sphinx/conf.py", "docs/.sphinx/index.rst", "docs/.sphinx/lema.builders.rst", "docs/.sphinx/lema.core.datasets.rst", "docs/.sphinx/lema.core.rst", "docs/.sphinx/lema.core.types.rst", "docs/.sphinx/lema.datasets.rst", "docs/.sphinx/lema.evaluation.rst", "docs/.sphinx/lema.launcher.clouds.rst", "docs/.sphinx/lema.launcher.rst", "docs/.sphinx/lema.models.rst", "docs/.sphinx/lema.rst", "docs/.sphinx/make.bat", "docs/.sphinx/modules.rst", "pyproject.toml"], "area": "docs", "area_votes": {"infra": 2, "docs": 15}, "body": "We can now generate documentation from our docstrings. I've elected to use Google style parsing as that aligns with our current style guide.\r\n\r\nTODOs:\r\n- Miscellaneous cleanup of our internal docstrings\r\n- Clean up which modules we expose\r\n- Publish the generated docs on our website.\r\n\r\n\r\nTowards OPE-300", "merged_at": "2024-08-15T17:36:03Z"}
{"number": 333, "title": "Add PyTorch profiler annotation for each step/micro-step", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Makes it easier to understand where each step starts/stops, and how long it takes.\r\n-- No effect on functional behavior\r\n\r\nTowards OPE-293", "merged_at": "2024-08-14T21:15:47Z"}
{"number": 332, "title": "Enable `HfMfuTrainerCallback` if supported", "files": ["src/lema/core/callbacks/hf_mfu_callback.py", "src/lema/performance/mfu.py", "src/lema/train.py", "tests/performance/test_mfu.py"], "area": "training", "area_votes": {"training": 3}, "body": "-- Refactor training callback creation into a helper function\r\n-- Bugfixes in `HfMfuTrainerCallback`\r\n-- We may consider having a separate param to control `HfMfuTrainerCallback` enablement in the future to avoid confusing users by reporting multiple MFU metrics, but for now it's OK to report both \r\n\r\nTowards OPE-254\r\nFixes OPE-278", "merged_at": "2024-08-14T23:59:00Z"}
{"number": 331, "title": "Update README.md", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "Update with feedback from Manos.\r\n\r\nTowards OPE-146", "merged_at": "2024-08-14T18:21:50Z"}
{"number": 330, "title": "Update LEMA training loop to count tokens on CPU", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Counting tokens on CPU is fast (`~330us`),  and saves one GPU-to-CPU transfer\r\n\r\nTowards OPE-296", "merged_at": "2024-08-14T00:30:27Z"}
{"number": 328, "title": "fix synchronization issues in LEMA training loop", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "Decorators like `@local_leader_only()`, `@global_leader_only()` proved to be error-prone, so replacing them with simpler `is_local_process_zero()`, `is_world_process_zero()` calls.\r\n\r\nFor example, the following snippet results in synchronization error because `global_leader_only()` uses `barrier()` internally, and, thus, it assumes it's called by all workers, but in this example, it's only called by the main process.  This can happen during nested calls, which makes such mistakes not obvious.\r\n\r\n```\r\n@global_leader_only()\r\ndef log(self, message: str):\r\n  logger.info(message)\r\n\r\ndef bazz():\r\n  log(\"bzz\")\r\n\r\ndef foo():\r\n  if is_world_process_zero():\r\n     bazz()\r\n```\r\n\r\nFixes OPE-297\r\n\r\n\r\n", "merged_at": "2024-08-14T00:30:13Z"}
{"number": 325, "title": "Fix nanoGPT notebook", "files": ["notebooks/LeMa - Using NanoGPT.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "Towards OPE-290\r\n\r\nWe needed to change the output of the `forward()` call to be compatible with the HF trainer. I also fixed a bug with saving the model that surfaced after by switching to the lema trainer.", "merged_at": "2024-08-12T23:12:41Z"}
{"number": 324, "title": "Add support for logging stdout and stderr for Local runs.", "files": ["src/lema/launcher/clients/local_client.py", "tests/launcher/clients/test_local_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Users can now set the environment variable \"LEMA_LOGGING_DIR\" to write output files during job runs.\r\nOutput files are written using following format:\r\n\r\nSUBMISSIONTIME_JOBID.stdout\r\n\r\n\r\nFixes OPE-289", "merged_at": "2024-08-12T21:41:19Z"}
{"number": 323, "title": "Remove unbalanced call to `barrier()` in `HuggingFaceTrainer.save_model`", "files": ["src/lema/core/trainers/hf_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- Synchronization is now done at higher level in `train()` function", "merged_at": "2024-08-10T17:33:46Z"}
{"number": 321, "title": "Create a tutorial for custom clouds.", "files": ["notebooks/LeMa - Launching Jobs on Custom Clusters.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "Create a brief tutorial for registering a dummy cloud with lema.\r\n\r\n\r\nFixes OPE-241", "merged_at": "2024-08-12T16:28:28Z"}
{"number": 320, "title": "Create a simpler tutorial for running jobs.", "files": ["notebooks/LeMa - Deploying a Job.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "This tutorial runs a simple Hello World job, and refers to the advanced tutorial / the Finetuning tutorial for fields of the JobConfig and TrainConfig, respectively.\r\n\r\n\r\nFixes OPE-242", "merged_at": "2024-08-09T22:38:19Z"}
{"number": 319, "title": "Add  pytorch profiler (`-p`) option to `multinode_example_worker.sh` script", "files": ["scripts/polaris/jobs/multinode_example_worker.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "-- Makes it easier to enable PyTorch profiler \r\n\r\nTowards OPE-254, OPE-136", "merged_at": "2024-08-09T20:45:35Z"}
{"number": 318, "title": "Fix a small typo in Lema README", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2024-08-09T20:22:55Z"}
{"number": 317, "title": "Add a 'done' field to the LeMa job status object.", "files": ["src/lema/core/types/base_cluster.py", "src/lema/launcher/clients/local_client.py", "src/lema/launcher/clients/polaris_client.py", "src/lema/launcher/clients/sky_client.py", "src/lema/launcher/clusters/sky_cluster.py", "tests/launcher/clients/test_local_client.py", "tests/launcher/clients/test_polaris_client.py", "tests/launcher/clients/test_sky_client.py", "tests/launcher/clouds/test_local_cloud.py", "tests/launcher/clouds/test_polaris_cloud.py", "tests/launcher/clouds/test_sky_cloud.py", "tests/launcher/clusters/test_local_cluster.py", "tests/launcher/clusters/test_polaris_cluster.py", "tests/launcher/clusters/test_sky_cluster.py", "tests/launcher/test_launcher.py"], "area": "launcher", "area_votes": {"data": 1, "launcher": 4}, "body": "Adding a `done` field in addition to `status`. `done` is a boolean that simply indicates if the job has terminated. It is up to each cloud to set this value properly. `status` is still a string that can give cloud-specific finegrain status.\r\n\r\nThe addition of this field lets us easily poll jobs in tutorials.", "merged_at": "2024-08-09T19:46:42Z"}
{"number": 316, "title": "Add simple benchmark script for distributed operations", "files": ["scripts/benchmarks/benchmark_nccl.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "Adding a benchmark / sanity check with simple distributed operations, to a) make sure the distributed env, nccl and network are configured properly and (b) get a measure of the inter and intra node speed for basic ops: barrier, all_reduce, all_gather", "merged_at": "2024-08-09T17:09:12Z"}
{"number": 315, "title": "[bugfix] GPU workers not waiting for global leader to save final checkpoint", "files": ["src/lema/core/trainers/lema_trainer.py", "src/lema/train.py"], "area": "training", "area_votes": {"training": 2}, "body": "Towards OPE-257", "merged_at": "2024-08-09T16:27:40Z"}
{"number": 314, "title": "Fix interpolation when loading lema configs.", "files": ["configs/lema/jobs/gcp/llama2b_fsdp.yaml", "configs/lema/jobs/polaris/hello_world.yaml", "configs/lema/jobs/polaris/multinode_example.yaml", "src/lema/core/types/base_config.py"], "area": "configs", "area_votes": {"configs": 3, "data": 1}, "body": "Omegaconf would throw an error if the `setup` for `run` fields in a lema job config contained variable interpolation ( ${myvar}).\r\n\r\nWe can circumvent this by reading the file and manually replacing these instances with a delimited version.\r\n\r\nNote that in the final config, the fields will NOT have the delimiter. OmegaConf removes it after initial parsing.\r\n\r\n\r\n\r\nFixes OPE-285", "merged_at": "2024-08-09T16:10:57Z"}
{"number": 313, "title": "Add finetuning tutorial", "files": ["notebooks/LeMa - Finetuning Tutorial.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "First pass at a fine-tuning tutorial.\r\n\r\nIn this tutorial, we'll fine-tune a large language model to improve it's ability to generate and explain complex python code. \r\n\r\nWe'll cover the following topics:\r\n1. Prerequisites\r\n2. Data Preparation & Sanity Checks [WIP]\r\n3. Training Config Preparation\r\n4. Launching Training\r\n5. Monitoring Progress [WIP]\r\n6. Evaluation\r\n7. Analysing Results [WIP]\r\n8. Inference", "merged_at": "2024-08-09T14:53:05Z"}
{"number": 312, "title": "Remove some leftover occurrences of `builtin_` prefix in HF MFU callback", "files": ["src/lema/core/callbacks/hf_mfu_callback.py"], "area": "training", "area_votes": {"training": 1}, "body": "Towards OPE-278", "merged_at": "2024-08-08T19:42:04Z"}
{"number": 310, "title": "Add a convenience method for listing all registered clouds.", "files": ["src/lema/launcher/__init__.py", "src/lema/launcher/launcher.py", "tests/launcher/test_launcher.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Add a convenience method for listing all registered clouds. This is available as a class method in the launcher as well as a top-level module method.", "merged_at": "2024-08-08T19:26:53Z"}
{"number": 309, "title": "Create a local cloud for the LeMa launcher.", "files": ["src/lema/launcher/clouds/__init__.py", "src/lema/launcher/clouds/local_cloud.py", "tests/launcher/clouds/test_local_cloud.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Adds a local cloud for running local jobs. This consumes the local client / cluster.\r\n\r\nFixes OPE-281", "merged_at": "2024-08-08T19:41:04Z"}
{"number": 307, "title": "[ALCF] Reverse Polaris GPU order to match CPU/GPU affinities", "files": ["scripts/polaris/jobs/multinode_example_worker.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "Polaris docs recommend reversing GPU order (CUDA_VISIBLE_DEVICES=3,2,1,0) to match CPU/GPU affinities:\r\n\r\n- https://github.com/argonne-lcf/GettingStarted/blob/498550dd942a0a5188065e6131c285380419585e/Examples/Polaris/affinity_gpu/set_affinity_gpu_polaris.sh#L5\r\n- https://docs.alcf.anl.gov/polaris/hardware-overview/machine-overview/#polaris-device-affinity-information\r\n\r\nIn my initial tests, the change makes minor difference e.g., MFU 0.503 ->0.505/0.509 for 4 node DDP, [data](https://docs.google.com/spreadsheets/d/11GfKfjjWaNypDWgXYJ4Kgo6wrJlxVr_sMaWIJNjYeLQ/edit?gid=109378257#gid=109378257 ). More tests enqueued. Regardless, the change should make no harm, let's follow the recommendation.\r\n\r\nTowards OPE-231, OPE-252", "merged_at": "2024-08-08T19:40:26Z"}
{"number": 306, "title": "Add a local cluster for running local jobs.", "files": ["src/lema/launcher/clusters/local_cluster.py", "src/lema/launcher/clusters/polaris_cluster.py", "tests/launcher/clusters/test_local_cluster.py"], "area": "launcher", "area_votes": {"launcher": 2}, "body": "Adds a local cluster for running local jobs. This consumes the local client.\r\n\r\nTowards OPE-281", "merged_at": "2024-08-08T18:05:02Z"}
{"number": 305, "title": "Add a client for running local jobs via the launcher.", "files": ["src/lema/launcher/clients/local_client.py", "tests/launcher/clients/test_local_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Add a client for running jobs.\r\n\r\nThe client creates a daemon thread that polls a queue. Note that this client only allows one job to run at once. Other jobs are queued.\r\n\r\nJobs are executed in a subprocess using `Popen`\r\n\r\n\r\nTowards OPE-281", "merged_at": "2024-08-08T16:31:07Z"}
{"number": 304, "title": "Increase the default value of `ProfilerParams.row_limit` from 20 to 50", "files": ["src/lema/core/types/params/profiler_params.py"], "area": "data", "area_votes": {"data": 1}, "body": "-- `20` can be too little\r\n\r\nTowards OPE-136", "merged_at": "2024-08-07T18:52:14Z"}
{"number": 303, "title": "Mini guide on using basic lema functionality", "files": ["USAGE.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2024-08-07T19:21:20Z"}
{"number": 301, "title": "Compute MFU based of HF `total_flos` (alternative way to compute MFU)", "files": ["src/lema/core/callbacks/hf_mfu_callback.py", "src/lema/performance/mfu.py", "tests/performance/test_mfu.py"], "area": "training", "area_votes": {"training": 2}, "body": "Context:  https://linear.app/openlema/issue/OPE-253/[alcf]-add-hfu-metric#comment-5e222d54\r\n\r\nIf embedding params aren't subtracted from `train/train_mfu`:\r\n```\r\ntrain/builtin_train_mfu 0.55036\r\ntrain/builtin_train_step_mfu 0.5705\r\n\r\ntrain/train_mfu 0.55626\r\ntrain/train_step_mfu 0.57661\r\n```\r\n\r\nif embedding params are excluded (the current default):\r\n```\r\ntrain/builtin_train_mfu 0.54719\r\ntrain/builtin_train_step_mfu 0.56742\r\n\r\ntrain/train_mfu 0.51983\r\ntrain/train_step_mfu 0.53905\r\n``` \r\n\r\nTowards OPE-278", "merged_at": "2024-08-07T20:14:18Z"}
{"number": 300, "title": "Prevent HF version bump", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Temporary fix for the test failures here: https://github.com/openlema/lema/actions/runs/10275212628/job/28433447244?pr=299", "merged_at": "2024-08-06T23:49:49Z"}
{"number": 299, "title": "Create a notebook tutorial for running remote training.", "files": ["notebooks/LeMa - Running Jobs Remotely.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "This is a rough draft of a tutorial for running jobs with the LeMa Launcher.\r\n\r\nIn the tutorial I only focus on the bare basics. I have users kick off a hello-world job on GCP.\r\nI then provide a sample job that runs llama2b training on gcp.\r\n\r\nIdeally this tutorial should point the user to the local fine tuning tutorial as a reference for how to run training with LeMa.\r\n\r\nFixes OPE-242", "merged_at": "2024-08-07T17:19:50Z"}
{"number": 298, "title": "Export top level launcher functions and instantiate a default launcher.", "files": ["src/lema/launcher/__init__.py", "src/lema/launcher/launcher.py", "tests/launcher/test_launcher.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Minor cleanup to help make our tutorials simpler.\r\n\r\nSimilar to `REGISTRY`, I've instantiated a top-level launcher instance. I've also exported all default methods of this instance so they can be used as the module level.\r\n\r\nPreviously:\r\n```python\r\nfrom lema.launcher import Launcher\r\nlauncher = Launcher()\r\nlauncher.up(...)\r\n```\r\n\r\nNow:\r\n```python\r\nimport lema.launcher as launcher\r\nlauncher.up(...)\r\n```\r\n\r\nTowards OPE-242", "merged_at": "2024-08-06T19:39:41Z"}
{"number": 297, "title": "Disable compilation for DDP `accelerate launch` config", "files": ["configs/skypilot/sky_llama2b_ddp.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Until NaN gradients issues is resolved\r\n\r\nTowardsOPE-267", "merged_at": "2024-08-06T17:15:32Z"}
{"number": 296, "title": "Rename 'NodeParams' -> 'JobResources'", "files": ["src/lema/core/types/__init__.py", "src/lema/core/types/configs.py", "src/lema/core/types/params/job_resources.py", "src/lema/launcher/__init__.py", "tests/launcher/clients/test_sky_client.py", "tests/launcher/clouds/test_polaris_cloud.py", "tests/launcher/clouds/test_sky_cloud.py", "tests/launcher/clusters/test_polaris_cluster.py", "tests/launcher/clusters/test_sky_cluster.py", "tests/launcher/test_launcher.py"], "area": "data", "area_votes": {"data": 2}, "body": "Minor naming change to make it clearer what each field is doing in a LeMa Job. I'm consuming this downstream in our tutorial.\r\n\r\nNote that this deviates from our \"FooParams\" naming scheme, but I think this is much more useful for our end users.\r\n\r\nTowards OPE-241", "merged_at": "2024-08-06T16:41:22Z"}
{"number": 295, "title": "[tiny] Update GitHub action cache version", "files": [".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Update GitHub action cache version, `v3` is now deprecated. Using `v4` instead", "merged_at": "2024-08-06T00:56:16Z"}
{"number": 293, "title": "Misc minor changes", "files": ["configs/accelerate/gpt2.fsdp.yaml", "configs/accelerate/llama.fsdp.yaml", "configs/accelerate/phi3.fsdp.dpo.yaml", "configs/skypilot/sky_gpt2_fsdp.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_phi3_dpo_fsdp.yaml", "src/lema/performance/torch_profiler_utils.py"], "area": "configs", "area_votes": {"configs": 6, "training": 1}, "body": "1. Set `fsdp_limit_all_gathers: true` . This is the default value already, but better configure it explicitly for clarity. https://pytorch.org/docs/stable/fsdp.html\r\n2.  Print warning if FSDP is used with only 1 GPU. It's designed for multi-GPU and may fail in practice if only 1 GPU is used.\r\n3. Change filename format of Pytorch profiler files to include local rank, and set fixed width, so that lexicographic sort works as expected.\r\n\r\nTowards OPE-136, OPE-123", "merged_at": "2024-08-06T00:02:51Z"}
{"number": 292, "title": "[tiny] cleanup pyproject.toml dependencies", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "- Remove duplicated dependencies (`datasets`), and sort all of them to avoid dupes in the future\r\n- Remove unused `torchmetrics` package\r\n- We somehow had the wrong flash attention package... actual implementation is `flash-attn`, but we have `flash-attention`", "merged_at": "2024-08-05T22:46:30Z"}
{"number": 291, "title": "Make dataset data backend attribute read-only", "files": ["src/lema/core/datasets/base_dataset.py", "src/lema/core/datasets/iterable_dataset.py", "tests/builders/test_lema_data.py"], "area": "data", "area_votes": {"data": 2}, "body": "Towards OPE-119\r\n\r\nLooked into how to make the dataset data backend immutable -- unfortunately we can't do that with pandas data frames. Something to consider when we revisit the backend decision.\r\n\r\nFor now, just made the attribute read-only", "merged_at": "2024-08-05T22:46:43Z"}
{"number": 289, "title": "Optimize Github actions", "files": [".github/workflows/pretest.yaml", "tests/test_evaluate.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "This PR makes the following changes to the GitHub actions, to reduce the build time. Overall this seems to reduce the total runtime by ~half (`~13min20s` -> `~6min20s`)\r\n\r\n**Changes**\r\n- Switch to use `uv` for installing dependencies. [uv](https://github.com/astral-sh/uv) is a drop-in replacement for `pip`, but it's much faster\r\n  - Dependencies setup time went from `~3min30s` to `~1min`\r\n- Remove python cache. It used to take `~2min` to cache the 3GB install path, and for some reason we have very few cache hit (likely due to mis-config?). \r\n  - With `uv`, it's much faster to install from scratch than to load the cache, even assuming 100% cache hit. \r\n  - Skipping this step saves us ~2min\r\n- Add cache to `pre-commit` installation. Cached size is 35mb, so fast to download (2s). \r\n  - This reduced runtime from `1min30s` to `~30s` \r\n- Skip `test_evaluate_lm_harness` when running on CPU. This test is extremely slow (2-3min, and takes ~40% of the total test time). \r\n  - Cuts a further 2min from the build time \r\n\r\n\r\n**Before**\r\n\"Screenshot\r\n\r\n**After**\r\n\"Screenshot\r\n\r\n\r\nTowards OPE-276", "merged_at": "2024-08-05T23:40:26Z"}
{"number": 287, "title": "Some updates to Polaris launcher script", "files": ["scripts/polaris/jobs/multinode_example_worker.sh", "scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "-- Auto-select default queue if unspecified\r\n-- Change default training mode  from `ddp` to `fsdp`\r\n\r\nTowards OPE-252, OPE-231", "merged_at": "2024-08-05T19:26:23Z"}
{"number": 286, "title": "Update the polaris client to automatically set execute permissions for copied files.", "files": ["src/lema/launcher/clients/polaris_client.py", "tests/launcher/clients/test_polaris_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "This is a hack as sftp copy operations don't persist chmod permissions. Setting execute permissions recursively after we copy to polaris to ensure all dependent job scripts can run.\r\n\r\nFixes OPE-201", "merged_at": "2024-08-05T18:01:47Z"}
{"number": 280, "title": "Introduce BaseTokenizer alias", "files": ["src/lema/builders/data.py", "src/lema/builders/lema_data.py", "src/lema/core/datasets/base_dataset.py", "src/lema/core/datasets/iterable_dataset.py", "src/lema/core/trainers/lema_trainer.py", "src/lema/core/types/__init__.py", "src/lema/core/types/base_tokenizer.py", "src/lema/datasets/alpaca.py", "src/lema/datasets/chatqa.py", "src/lema/datasets/common.py", "src/lema/datasets/pretraining_async_text_dataset.py", "src/lema/datasets/prompt_response_sft_preprocessor_factory.py", "src/lema/datasets/trl_dpo_preprocessor.py", "src/lema/datasets/ultrachat_200k.py", "src/lema/evaluation/infer_prob.py", "tests/builders/test_lema_data.py", "tests/core/test_lema_trainer.py", "tests/test_infer_prob.py"], "area": "data", "area_votes": {"other": 2, "data": 10, "training": 1, "evaluation": 1}, "body": "-- For now, `BaseTokenizer` is defined as an alias.  The first step towards OPE-264\r\n-- Update all callers\r\n\r\nTowards OPE-264", "merged_at": "2024-08-04T15:51:05Z"}
{"number": 279, "title": "Cache get_device_rank_info", "files": ["src/lema/core/distributed.py"], "area": "training", "area_votes": {"training": 1}, "body": "-- it's used quite frequently and should return the same value", "merged_at": "2024-08-04T15:51:35Z"}
{"number": 276, "title": "Ensure we CD into the working DIR before submitting polaris jobs.", "files": ["src/lema/launcher/clients/polaris_client.py", "src/lema/launcher/clusters/polaris_cluster.py", "tests/launcher/clients/test_polaris_client.py", "tests/launcher/clusters/test_polaris_cluster.py"], "area": "launcher", "area_votes": {"launcher": 2}, "body": "A small bugfix for polaris jobs. Currently the launcher is not submitting jobs from the working dir. This PR updates the submission script to run in the proper directory.\r\n\r\nTowards OPE-201", "merged_at": "2024-08-02T23:54:50Z"}
{"number": 275, "title": "Adding initial scripts for running polaris jobs.", "files": ["configs/lema/jobs/polaris/hello_world.yaml", "configs/lema/jobs/polaris/multinode_example.yaml", "src/lema/core/types/__init__.py", "src/lema/launcher/__init__.py", "tests/integration/configs/test_parse_configs.py"], "area": "configs", "area_votes": {"configs": 2}, "body": "The LeMa launcher is now usable via python!\r\n\r\nI've added to scripts to demo the functionality. It's very similar to sky and continues to leverage the same set of script we were using for polaris previously.\r\n\r\nOf note:\r\n- I recommend using the \"setup\" section for defining your polaris PBS vars.\r\n- \"setup\" and \"run\" are combined into a single .sh script that is executed as your job on Polaris\r\n- Please use the proper cluster (e.g. \"debug.matthew\" or \"prod.matthew\") instead of setting the queue in your script. The launcher will route your job to the proper queue.\r\n- Use num_nodes to set the job node count without invoking any PBS vars.\r\n- Make sure to change the `user` before submitting your job!\r\n- Try to keep your job.name unique between invocations. If you set it to None your job name will be a hex-encoded UUID.\r\n\r\nSample usage (in python):\r\n```python\r\nfrom lema.launcher import Launcher, JobConfig\r\n\r\nl=Launcher()\r\njob = JobConfig.from_yaml(\"./configs/lema/jobs/polaris/hello_world.yaml\")\r\njob.user=\"matthew\"\r\njob.name=\"new_name_for_testing\"\r\nl.up(job, \"debug.matthew\")\r\n\r\n# Multinode job\r\n\r\nmn_job = JobConfig.from_yaml(\"./configs/lema/jobs/polaris/multinode_example.yaml\")\r\nmn_job.user=\"matthew\"\r\nmn_job.name=\"my_first_multinode_job\"\r\nl.up(mn_job, \"debug-scaling.matthew\")\r\n\r\n# Get the status of the most recent job\r\nl.status()[-1]\r\n```\r\n", "merged_at": "2024-08-05T16:20:55Z"}
{"number": 272, "title": "[tiny] Register debug datasets", "files": ["src/lema/datasets/__init__.py", "src/lema/datasets/debug.py"], "area": "data", "area_votes": {"data": 1}, "body": "Register debug datasets so then can be used in tests and notebooks without having to import directly", "merged_at": "2024-08-02T21:42:21Z"}
{"number": 271, "title": "Configure data loader sampling strategy for map-style datasets", "files": ["src/lema/core/trainers/lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "Configure data loader sampling strategy for map-style datasets. Iterable datasets to follow in a separate PR\r\n\r\nTowards OPE-224", "merged_at": "2024-08-02T22:34:54Z"}
{"number": 267, "title": "Move `torch_profiler_utils` from `lema.utils` to `lema.perfomance`", "files": ["src/lema/performance/torch_profiler_utils.py", "src/lema/train.py", "tests/test_torch_profiler_utils.py"], "area": "training", "area_votes": {"training": 2}, "body": "-- Also, switch to saving traces  in gzipped form (`.json.gz`). They compress extremely well\r\n\r\n\r\nTowards OPE-136", "merged_at": "2024-08-02T02:44:08Z"}
{"number": 266, "title": "Set  `dataloader_pin_memory=True` to be intentional", "files": ["src/lema/core/types/params/training_params.py"], "area": "data", "area_votes": {"data": 1}, "body": "-- The change is expected to be a no-op as `True` is the default value. Better to be explicit about this param.\r\n\r\nTowards OPE-141", "merged_at": "2024-08-02T00:29:43Z"}
{"number": 265, "title": "Make all tests green", "files": ["configs/lema/llama3.8b.aya.eval.yaml", "tests/core/test_lema_trainer.py"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Make sure running `pytest` under `lema/tests` pass for all tests \r\n-- Tested on WSL", "merged_at": "2024-08-01T21:31:45Z"}
{"number": 264, "title": "Switch from nightly to stable version of SkyPilot", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "We were waiting for version `>0.5.0` to include RunPod multi-gpu bugfix https://github.com/skypilot-org/skypilot/pull/3291/files\r\n\r\nThe current stable version is `0.6.1`:\r\n\r\n```\r\nsky --version\r\nskypilot, version 0.6.1\r\n```\r\n\r\nnightly was introduced in https://github.com/openlema/lema/pull/33", "merged_at": "2024-08-01T19:26:25Z"}
{"number": 263, "title": "Add BaseIterableDataset, refactor DataLoader to use DataPipes", "files": ["pyproject.toml", "src/lema/builders/lema_data.py", "src/lema/core/datasets/__init__.py", "src/lema/core/datasets/base_dataset.py", "src/lema/core/datasets/iterable_dataset.py", "src/lema/core/types/params/data_params.py", "src/lema/datasets/alpaca.py", "src/lema/datasets/chatqa.py", "src/lema/datasets/chatrag_bench.py", "tests/builders/test_lema_data.py", "tests/core/datasets/test_pretraining_dataset.py", "tests/integration/datasets/test_load_datasets.py"], "area": "data", "area_votes": {"infra": 1, "other": 1, "data": 6}, "body": "**Changes**\r\n- Add a `BaseIterableDataset`, with interface for Iterable-style datasets, similar to `BaseMapDataset` defined previously.\r\n- Add a `BasePretrainingIterableDataset` for pre-training datasets that we implement, with support for packing \r\n- Add a `lema_data` builder for datasets. This re-implement's the existing functionality provided by HuggingFace `datasets` using native `torch` DataPipes. Per PyTorch roadmap this is the modern/recommend way to do data-loading for PyTorch.\r\n  - The main advantage is that it allows us to use our native `torch.utils.data.Dataset` classes without having to convert to a HF dataset during init (which can take up ~15min for the ChatQA mixture for example) \r\n- Add support for DataLoaderV2 in lema_trainer (but may remove this to stick with DataLoader V1, TBD)\r\n- Move abstract datasets classes to `lema.core.datasets`\r\n\r\n**Benchmarks**\r\nWIP, will update here as I make progress\r\n\r\n\r\nTowards OPE-261, OPE-262, OPE-103, OPE-120", "merged_at": "2024-08-02T10:56:36Z"}
{"number": 259, "title": "Create DDP configs for `accelerate`", "files": ["configs/accelerate/llama.ddp.yaml", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama2b_ddp.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "-- Created `accelerate` configs for DDP, for consistency with FSDP/DeepSpeed\r\n-- Update training  parameters for `torchrun` and `accelerate` DDP configs to align with Polaris config\r\n-- Add a note about OPE-209 `Can't enable compilation with grad checkpointing in DDP mode`\r\n\r\nFor 1 node with 4 A100 GPUs on GCP, seeing slightly better (+3-4%)  speed for the current configuration (noise, or real?):\r\n-- torchrun DDP:\r\n  Run1:  15472 tok/s, 30.219 samples/s,  train/train_step_mfu 0.54785  https://wandb.ai/lema-train-test/lema-train-test/runs/snl1q4iu\r\n  Run2:  15730 tok/s, 30.724 samples/s, train_step_mfu 0.54743 https://wandb.ai/lema-train-test/lema-train-test/runs/9m8ct6f7\r\n-- accelerate DDP\r\n  Run1:  16082 tok/s,  31.411 samples/s train_step_mfu 0.57585 https://wandb.ai/lema-train-test/lema-train-test/runs/5xw0m7t6\r\n  Run2: 16423.265 tok/s 32.077 samples/s  train_step_mfu 0.57577  https://wandb.ai/lema-train-test/lema-train-test/runs/uasoc1lw\r\n  \r\nTowards OPE-267, OPE-209", "merged_at": "2024-08-01T18:17:12Z"}
{"number": 257, "title": "Add a `get_all` utility method to the LeMa Registry", "files": ["src/lema/core/registry.py", "tests/test_registry.py"], "area": "other", "area_votes": {"other": 1}, "body": "Add a `get_all()` method to the Registry. This will enable the launcher to initialize all registered builders at startup.\r\n\r\nTowards OPE-201", "merged_at": "2024-07-31T19:04:06Z"}
{"number": 256, "title": "Add logging to tensor board, wandb in custom training loop", "files": ["src/lema/core/trainers/lema_trainer.py", "tests/core/test_lema_trainer.py"], "area": "training", "area_votes": {"training": 1}, "body": "Add logging to tensor board, wandb in custom training loop\r\n\r\nTowards OPE-220", "merged_at": "2024-07-31T01:18:37Z"}
{"number": 253, "title": "Create a Polaris Cloud class consuming the polaris client", "files": ["src/lema/launcher/clients/polaris_client.py", "src/lema/launcher/clouds/polaris_cloud.py", "tests/launcher/clouds/test_polaris_cloud.py"], "area": "launcher", "area_votes": {"launcher": 2}, "body": "Planned changes:\r\n\r\n- Create a loose wrapper for polaris <------------ https://github.com/openlema/lema/pull/234\r\n- Create a Polaris Cluster class consuming the client <------------ #246\r\n- Create Polaris Cloud class consuming the client <------------ You are here\r\n\r\nFixes OPE-200", "merged_at": "2024-07-30T22:57:19Z"}
{"number": 252, "title": "Save and restore telemetry state during training", "files": ["src/lema/core/trainers/lema_trainer.py", "src/lema/performance/telemetry.py"], "area": "training", "area_votes": {"training": 2}, "body": "**Changes**\r\n- When saving a checkpoint, save telemetry state\r\n- When loading a checkpoint, load telemetry state\r\n\r\nOPE-222", "merged_at": "2024-07-30T20:42:44Z"}
{"number": 250, "title": "Add torchfix listing target", "files": ["Makefile", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "[TorchFix](https://pypi.org/project/TorchFix/) is a Python code static analysis tool for PyTorch code. It can be used to find and fix issues in PyTorch code.\r\n\r\nThis PR adds a makefile target to use torchfix. Eventually this should be a ruff rule, but it is not supported yet.\r\n\r\nUsage: `make torchfix`\r\n", "merged_at": "2024-07-30T17:34:03Z"}
{"number": 246, "title": "Create a Polaris Cluster class consuming the polaris client", "files": ["src/lema/launcher/clusters/polaris_cluster.py", "tests/launcher/clusters/test_polaris_cluster.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Create a Polaris Cluster class consuming the client\r\n\r\nPlanned changes:\r\n\r\n- Create a loose wrapper for polaris <------------ #234\r\n- Create a Polaris Cluster class consuming the client <------------ You are here\r\n- Create Polaris Cloud class consuming the client <------------ Coming Soon\r\n- \r\nTowards OPE-200", "merged_at": "2024-07-30T15:59:31Z"}
{"number": 244, "title": "Add deepspeed (DS) config to support hierarchical partitioning", "files": ["configs/accelerate/deepspeed_config.json", "configs/accelerate/llama.deepspeed.yaml", "configs/skypilot/sky_eval.yaml", "configs/skypilot/sky_gpt2_eval.yaml", "configs/skypilot/sky_llama2b_deepspeed.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "scripts/polaris/jobs/multinode_example_worker.sh"], "area": "configs", "area_votes": {"configs": 6, "infra": 1}, "body": "Towards OPE-76\r\n\r\n- Use deepspeed config to enable hierarchical partitioning (equivalent to FSDP hybrid shard)\r\n- Switch to zero-2 since 2B is not too large a model size. Best throughput is with gradient checkpointing disabled, and b4x64.\r\n- Add DS to polaris launcher script\r\n- Fix accelerator counts in various sky configs", "merged_at": "2024-07-30T01:06:11Z"}
{"number": 242, "title": "Add a \"put\" method in the Polaris client for writing remote files.", "files": ["src/lema/launcher/clients/polaris_client.py", "tests/launcher/clients/test_polaris_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Add a \"put\" method in the Polaris Client. This method handles writing remote files directly from a string, alleviating the need to write a local file before calling rsync.", "merged_at": "2024-07-29T19:33:24Z"}
{"number": 241, "title": "Add initial docker image", "files": ["Dockerfile", "scripts/docker/build_docker.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "Adding a simple docker image with the lema dependencies.\r\n\r\nTowards OPE-52", "merged_at": "2024-07-30T16:31:30Z"}
{"number": 239, "title": "Update Fabric.run() calls to use the \"warn\" flag.", "files": ["src/lema/launcher/clients/polaris_client.py", "tests/launcher/clients/test_polaris_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "Update all calls to fabric connection.run() to use the \"warn\" flag. This lets us capture the output of failed commands without throwing an exception at the Fabric level.", "merged_at": "2024-07-26T23:42:59Z"}
{"number": 238, "title": "Update `pretokenize` tool to support input datasets", "files": ["src/experimental/pretokenize/sky.yaml", "src/experimental/pretokenize/tokenize_dataset.py"], "area": "other", "area_votes": {"other": 2}, "body": "-- Previously, it could only process individual files.\r\n\r\nTowards OPE-102", "merged_at": "2024-07-26T23:49:38Z"}
{"number": 237, "title": "Add SkyPilot config for `experimental/pretokenize/tokenize_dataset.py`", "files": ["src/experimental/pretokenize/sky.yaml", "src/experimental/pretokenize/tokenize_dataset.py"], "area": "other", "area_votes": {"other": 2}, "body": "-- Minor changes in `tokenize_dataset.py`\r\n-- Move the library into `experimental/pretokenize/` subdir\r\n\r\nTowards OPE-102", "merged_at": "2024-07-26T20:59:20Z"}
{"number": 236, "title": "Disable gradient checkpointing in SkyPilot llama2b config", "files": ["configs/skypilot/sky_llama2b_fsdp.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Remove `training.enable_gradient_checkpointing=true` from SkyPilot config \r\n-- Increase the default number of accelerators from 1 to 4:  (`A100:4` ). This is required to support batch_size=14 in FSDP mode (FSDP will shard the model across GPUs)\r\n\r\nSee also: https://linear.app/openlema/issue/OPE-123/[alcf][stretch]-fsdp-high-mfu-with-fsdp-for-llama3-8b-and-70b#comment-6639aee3\r\n\r\nTested on GCP:  https://wandb.ai/lema-train-test/lema-train-test/runs/x0rc5c71\r\n\r\n```\r\n sky launch --cloud GCP -i 10 --env LEMA_RUN_NAME=llama2b.pt.1nodes.4xA10040GB.20steps.bs14.gas19.gcpt.v1201 --use-spot --num-nodes 1 --gpus \"A100:4\" --cluster xrdaukar-1node4gpus-01-lema-cluster configs/skypilot/sky_llama2b_fsdp.yaml\r\n\r\n 'Train Step MFU': 0.4945762699285416, 'Train MFU': 0.49439496216336576\r\n\r\n(lema-train-example, pid=5259) wandb: Run summary:\r\n(lema-train-example, pid=5259) wandb:                    total_flos 1.0529579618585805e+17\r\n(lema-train-example, pid=5259) wandb:                   train/epoch 1.0\r\n(lema-train-example, pid=5259) wandb:             train/global_step 20\r\n(lema-train-example, pid=5259) wandb:               train/grad_norm 0.9668\r\n(lema-train-example, pid=5259) wandb:           train/learning_rate 0.0\r\n(lema-train-example, pid=5259) wandb:                    train/loss 8.5124\r\n(lema-train-example, pid=5259) wandb:   train/num_input_tokens_seen 43581440\r\n(lema-train-example, pid=5259) wandb: train/train_tokens_per_second 13341.678\r\n(lema-train-example, pid=5259) wandb:                    train_loss 9.32834\r\n(lema-train-example, pid=5259) wandb:                 train_runtime 816.6409\r\n(lema-train-example, pid=5259) wandb:      train_samples_per_second 26.058\r\n(lema-train-example, pid=5259) wandb:        train_steps_per_second 0.024\r\n```", "merged_at": "2024-07-26T20:05:09Z"}
{"number": 235, "title": "[tiny] Update logger format to include rank, pid and threadname", "files": ["src/lema/utils/logging.py"], "area": "other", "area_votes": {"other": 1}, "body": "To help debug distributed training, adding more information for the logger formatter", "merged_at": "2024-07-26T00:09:39Z"}
{"number": 233, "title": "Set model.config.use_cache = False", "files": ["src/lema/builders/models.py"], "area": "other", "area_votes": {"other": 1}, "body": "-- Required for FSDP.\r\n-- Context: https://github.com/huggingface/transformers/issues/28499\r\n\r\nTowards OPE-122, OPE-133, OPE-123, OPE-228 ", "merged_at": "2024-07-26T01:03:30Z"}
{"number": 232, "title": "Rename accelerate configs to be in line with other configs", "files": ["README.md", "configs/accelerate/gpt2.fsdp.yaml", "configs/accelerate/llama.deepspeed.yaml", "configs/accelerate/llama.fsdp.yaml", "configs/accelerate/phi3.fsdp.dpo.yaml", "configs/skypilot/sky_gpt2_fsdp.yaml", "configs/skypilot/sky_llama2b_deepspeed.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_phi3_dpo_fsdp.yaml", "scripts/polaris/jobs/multinode_example_worker.sh"], "area": "configs", "area_votes": {"docs": 1, "configs": 8, "infra": 1}, "body": "If folks are ok with this change, I'll complete the PR by renaming all references in our repo.\r\n\r\nIn addition, did some minor cleanup for the deepspeed config.", "merged_at": "2024-07-25T19:40:49Z"}
{"number": 227, "title": "Add makefile with common local commands", "files": ["Makefile"], "area": "infra", "area_votes": {"infra": 1}, "body": "This pull request introduces a Makefile, to help with common development tasks (e.g. launching a remove vscode/ssh session, running tests, ruff, pre-commit hooks, upgrade dependencies)\r\n\r\n## Usage\r\n\r\n**Running tests  and code quality checks**\r\n```shell\r\nmake test\r\nmake check\r\nmake lint\r\n```\r\n\r\n**Opening a VSCode remote session on a cloud VM**\r\n```shell\r\n# both commands will launch a new cluster with the sky-ssh.yaml config, or use an existing cluster if found\r\nmake skycode \r\nmake skyssh ARGS=\"--gpus=A100:4\"\r\n```\r\n\r\n**Running train/evaluate/infer**\r\n```shell\r\nmake train ARGS=\"-c configs/lema/gpt2.pt.mac.yaml\"\r\n```\r\n\r\n**Setting up the project**\r\n```shell\r\nmake setup  # init a new conda env with dependencies. Skips if already exists\r\nmake upgrade  # upgrade depenencies\r\n```\r\n\r\n\r\n", "merged_at": "2024-07-24T20:37:23Z"}
{"number": 222, "title": "Add telemetry manager", "files": ["src/lema/performance/telemetry.py", "tests/performance/test_mfu.py", "tests/performance/test_telemetry.py"], "area": "training", "area_votes": {"training": 1}, "body": "### Changes\r\nAdd multiple context managers and decorators to time and log CPU and CUDA operations.\r\n\r\nTowards [OPE-211](https://linear.app/openlema/issue/OPE-211/add-telemetry-to-monitor-individual-operations-during-training)\r\n\r\n### Usage\r\n\r\n```python\r\nfrom telemetry_tracker import TelemetryTracker, gpu_memory_logger\r\n\r\ntelemetry = TelemetryTracker()\r\n\r\n# use as a context manager for CPU operations\r\nwith telemetry.timer(\"cpu_operation\"):\r\n    # Your CPU-bound code here\r\n    pass\r\n\r\n# or use as a decorator\r\n@telemetry.timer(\"cpu_operation\")\r\ndef cpu_operation():\r\n     # Your CPU-bound code here\r\n\r\n# Use as a context manager for CUDA operations\r\nwith telemetry.cuda_timer(\"cuda_operation\"):\r\n    # Your CUDA-accelerated code here\r\n    pass\r\n\r\n@gpu_memory_logger\r\ndef gpu_intensive_function():\r\n    # Your GPU-intensive code here\r\n    pass\r\n\r\n# Print summary\r\ntelemetry.print_summary()\r\n```\r\n", "merged_at": "2024-07-24T03:50:29Z"}
{"number": 221, "title": "Update shell scripts to point to local dataset", "files": ["scripts/polaris/jobs/example_job.sh", "scripts/polaris/jobs/multinode_example_worker.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "Towards OPE-127\r\n\r\nOn 5 nodes slightly improves MFU:\r\n32.9% -> 33.3%", "merged_at": "2024-07-23T22:42:31Z"}
{"number": 219, "title": "Minor logging improvements in Polaris sample job scripts", "files": ["scripts/polaris/jobs/multinode_example_job.sh", "scripts/polaris/jobs/multinode_example_worker.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "\r\nTowards OPE-208", "merged_at": "2024-07-23T17:24:31Z"}
{"number": 218, "title": "Use \"cluster_name\" instead of \"name\" in the Sky client.", "files": ["src/lema/launcher/clients/sky_client.py", "tests/launcher/clients/test_sky_client.py"], "area": "launcher", "area_votes": {"launcher": 1}, "body": "A small bugfix for the Sky client. We should use the \"cluster_name\" field to identify clusters.\r\n\r\nTowards OPE-134", "merged_at": "2024-07-23T16:41:50Z"}
{"number": 213, "title": "Configure llama2b model to use FSDP HYBRID_SHARD", "files": ["configs/accelerate/sample_fsdp_llama3.yaml", "configs/skypilot/sky.yaml", "configs/skypilot/sky_chatqa.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_gpt2_fsdp.yaml", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml", "configs/skypilot/sky_phi3_dpo_fsdp.yaml", "configs/skypilot/sky_zephyr_7b_qlora_sft.yaml", "configs/skypilot/sky_zephyr_7b_sft.yaml"], "area": "configs", "area_votes": {"configs": 11}, "body": "-- For 1 node, `HYBRID_SHARD` is equivalent to `FULL_SHARD`\r\n-- Update `accelerate launch --num_processes` to be `num_nodes*gpus_per_node` (different from `torchrun`, which expects `gpus_per_node`)\r\n-- Improve logging in `run:` section for all training jobs\r\n-- Verified `HYBRID_SHARD` on 1, 2, 4 nodes on GCP\r\n\r\nFSDP `HYBRID_SHARD` 1 node GCP:  21.7 samples/s  Train Step MFU: 0.373   https://wandb.ai/lema-train-test/lema-train-test/runs/nqmriqk4\r\nFSDP `HYBRID_SHARD` 2 nodes GCP:  40.8 samples/s  Train Step MFU: 0.353  https://wandb.ai/lema-train-test/lema-train-test/runs/9rsvxq18\r\nFSDP `HYBRID_SHARD` 4 nodes GCP:  74.41 samples/s https://wandb.ai/lema-train-test/lema-train-test/runs/e92t7hmu\r\n\r\nFor comparison FSDP `FULL_SHARD` on 4 nodes GCP:   55.349 samples/s Train Step MFU': 0.237  https://wandb.ai/lema-train-test/lema-train-test/runs/ipeevlja\r\n\r\nTowards OPE-202, OPE-133, OPE-123", "merged_at": "2024-07-22T22:55:50Z"}
{"number": 210, "title": "[tiny] add default formatter for markdown", "files": [".vscode/settings.json"], "area": "other", "area_votes": {"other": 1}, "body": "Update vscode settings to enable a markdown linter and formatter.\r\n\r\nWent with [markdownlint](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint), which seems very reputable.", "merged_at": "2024-07-22T18:55:50Z"}
{"number": 209, "title": "Update MFU callback to support Lema trainer", "files": ["src/lema/core/callbacks/mfu_callback.py", "src/lema/performance/mfu.py"], "area": "training", "area_votes": {"training": 2}, "body": "This PR decouples our MFU callback from HF's API to allow using it with Lema trainer without having to re-implement HF's internal training state. Contributes towards OPE-210\r\n\r\n**Changes**\r\n- `TrainingState` and `TrainerControl` are now optional (they are not used/needed for this callback)\r\n- Use `is_world_process_zero` instead of relying on `TrainingState`.\r\n- The callback now can take either a `transformers.TrainingArguments` or `lema.TrainingParams`.\r\n- Add `Nvidia L4` flops numbers", "merged_at": "2024-07-22T20:41:50Z"}
{"number": 207, "title": "Update Lema FSDP configs", "files": ["configs/accelerate/sample_fsdp_gpt2.yaml", "configs/accelerate/sample_fsdp_llama3.yaml", "configs/skypilot/sky_gpt2_fsdp.yaml", "configs/skypilot/sky_init.sh", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama2b_fsdp.yaml", "configs/skypilot/sky_phi3_dpo_fsdp.yaml", "src/lema/builders/models.py"], "area": "configs", "area_votes": {"configs": 7, "other": 1}, "body": "-- Start using `./configs/skypilot/sky_init.sh` and set distributed params like ` --main_process_ip ` and `--main_process_port`\r\n-- Add a comment about `model.config.use_cache = False` and the `transformers` issue: https://github.com/huggingface/transformers/issues/28499\r\n-- Tweak parameters for `configs/skypilot/sky_llama2b_fsdp.yaml`\r\n-- Rename some configs from `sky_fsdp_{model}.yaml` to `sky_{model}_fsdp.yaml` for consistency\r\n\r\nFSDP allows `bs=16` for llama2b model (vs `bs=2` for DDP) but actual thoughput (tokens/s, samples/s) and MFU are a bit worse than for DDP. For example:\r\n* FSDP: `bs=16`, 'Train Step MFU': 0.375 , 21.79 samples/s , https://wandb.ai/lema-train-test/lema-train-test/runs/lkxpxs3m\r\n* DDP: `bs=2`, 'Train Step MFU': 0.396, 23.07 samples/s, https://wandb.ai/lema-train-test/lema-train-test/runs/ls8lo2yf\r\n\r\nTowards OPE-202, OPE-133, OPE-76, OPE-123\r\n", "merged_at": "2024-07-22T18:06:32Z"}
{"number": 206, "title": "Minor updates to Polaris launcher script", "files": ["scripts/polaris/jobs/multinode_example_job.sh", "scripts/polaris/launcher.sh", "scripts/polaris/polaris_init.sh"], "area": "infra", "area_votes": {"infra": 3}, "body": "-- Add new flags `-q` and `-n` to specify queue and number of nodes\r\n-- Log full `qsub` command with expanded arguments\r\n-- Replace `cat` with `tail -n 100 -f` to monitor jobs logs in real time\r\n-- `mpiexec --verbose --envall`\r\n-- Log `hostname` in `polaris_init.sh`\r\n\r\nTowards OPE-208", "merged_at": "2024-07-22T03:28:17Z"}
{"number": 203, "title": "Sample job for multi-node training", "files": ["scripts/polaris/jobs/multinode_example_job.sh", "scripts/polaris/jobs/multinode_example_worker.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "-- based on `mpiexec`.\r\n-- Single-node shows same performance as w/o `mpiexec`\r\n-- Multi-node DDP technically works but slow (to be investigated)\r\n\r\nTowards OPE-208 ", "merged_at": "2024-07-20T15:34:47Z"}
{"number": 202, "title": "Fixed miscalculation of second step start time", "files": ["src/lema/core/callbacks/mfu_callback.py", "src/lema/train.py"], "area": "training", "area_votes": {"training": 2}, "body": "`_time_of_second_step` was being re-calculated every step, and `_START_TIME` was not being set when using `torchrun`\r\n\r\nTowards OPE-153", "merged_at": "2024-07-20T00:15:59Z"}
{"number": 199, "title": "Update actual mfu calculation", "files": ["src/lema/core/callbacks/mfu_callback.py", "src/lema/train.py"], "area": "training", "area_votes": {"training": 2}, "body": "Update `actual MFU` to only focus on first training step.\n\nTowards OPE-153", "merged_at": "2024-07-19T21:27:25Z"}
{"number": 193, "title": "Added MFU telemetry", "files": ["src/lema/core/callbacks/mfu_callback.py", "src/lema/evaluation/__init__.py", "src/lema/performance/mfu.py", "src/lema/train.py", "src/lema/utils/torch_utils.py", "tests/test_mfu.py"], "area": "training", "area_votes": {"training": 3, "other": 1}, "body": "Add MFU profiling to training runs.\n\nTowards OPE-153", "merged_at": "2024-07-18T21:53:43Z"}
{"number": 192, "title": "Update Polaris script", "files": [".gitignore", "scripts/polaris/jobs/example_job.sh", "scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 3}, "body": "- Use the [Conda from scratch method](https://github.com/openlema/lema/wiki/ALCF-INCITE#c-conda-from-scratch) instead of the venv approach since it's the best performing right now.\r\n- Exclude unneeded files from the rsync copy for speed/terminal output cleanliness. This is basically the contents of our `.gitignore` plus `tests/`. It'd be nice to exclude `.git/` as well, but it's required to `pip install` lema.\r\n- Log to a user-specific subdir, and echo the log file names for convenience. To solve the issue with escaping the bash command substitution: https://stackoverflow.com/a/36025832\r\n\r\nExample output:\r\n```\r\n(lema) 0:02:33 ~/Documents/lema wizeng/alcf $ ./scripts/polaris/launcher.sh -u $ALCF_USER -d /home/$ALCF_USER/lema/ -j ./scripts/polaris/jobs/example_job.sh\r\n...\r\nPassword:\r\nControlSocket /Users/wizeng/.ssh/control-polaris.alcf.anl.gov-22-wizeng already exists, disabling multiplexing\r\nCopying files to Polaris... -----------------------------------------\r\nbuilding file list ... done\r\ncreated directory /home/wizeng/lema\r\n./\r\n.gitignore\r\n...\r\nsrc/lema/utils/torch_utils.py\r\n\r\nsent 1413122 bytes  received 2714 bytes  314630.22 bytes/sec\r\ntotal size is 6716009  speedup is 4.74\r\nSetting up environment and submitting job on Polaris...\r\n\r\nLmod is automatically replacing \"nvhpc/23.9\" with \"gcc-native/12.3\".\r\n\r\n\r\nLmod is automatically replacing \"PrgEnv-nvhpc/8.5.0\" with \"PrgEnv-gnu/8.5.0\".\r\n\r\n\r\nDue to MODULEPATH changes, the following have been reloaded:\r\n  1) cray-mpich/8.1.28\r\n\r\nInstalling packages... -----------------------------------------\r\n...\r\nSuccessfully installed lema-0.1.dev188+gbb378f0.d20240718\r\nSubmitting job... -----------------------------------------\r\nJob id: 2025690.polaris-pbs-01.hsn.cm.polaris.alcf.anl.gov\r\n\r\nAll jobs:\r\n\r\npolaris-pbs-01.hsn.cm.polaris.alcf.anl.gov:\r\n                                                                 Req'd  Req'd   Elap\r\nJob ID               Username Queue    Jobname    SessID NDS TSK Memory Time  S Time\r\n-------------------- -------- -------- ---------- ------ --- --- ------ ----- - -----\r\n2025690.polaris-pbs* wizeng   debug    example_j*    --    1  64    --  00:10 Q   --\r\n\r\nTo view error logs, run:\r\ncat /eagle/community_ai/jobs/logs/wizeng/2025690.polaris-pbs-01.hsn.cm.polaris.alcf.anl.gov.ER\r\nTo view output logs, run:\r\ncat /eagle/community_ai/jobs/logs/wizeng/2025690.polaris-pbs-01.hsn.cm.polaris.alcf.anl.gov.OU\r\n```", "merged_at": "2024-07-18T22:35:31Z"}
{"number": 190, "title": "Added mfu calculation and tests", "files": ["src/lema/evaluation/__init__.py", "src/lema/evaluation/mfu.py", "tests/test_mfu.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "Implement MFU metric calculation from PaLM paper.\n\nTowards OPE-153", "merged_at": "2024-07-17T23:08:37Z"}
{"number": 189, "title": "Initial abstract base classes for the lema launcher.", "files": ["src/lema/core/types/base_cloud.py", "src/lema/core/types/base_cluster.py"], "area": "data", "area_votes": {"data": 2}, "body": "Creates the initial abstract base classes for Cloud and Cluster.\r\n\r\n\r\nNote: this PR is stacked on the changes in #188 . I will rebase when #188 is submitted.\r\n\r\n\r\nFixes OPE-198", "merged_at": "2024-07-17T23:07:12Z"}
{"number": 188, "title": "Create the Jobs config for the lema launcher.", "files": ["src/lema/core/types/configs.py", "src/lema/core/types/params/node_params.py"], "area": "data", "area_votes": {"data": 2}, "body": "Initial definition of the lema job config.\r\n\r\nNote that I've removed several fields from Sky's config, as well as broken file_mounts and storage_mounts into two separate fields for clarity.\r\n\r\nFixes OPE-197", "merged_at": "2024-07-17T22:11:35Z"}
{"number": 185, "title": "Remove logger propagation", "files": ["src/lema/logging.py", "src/lema/utils/torch_utils.py"], "area": "other", "area_votes": {"other": 2}, "body": "Towards OPE-114\r\n\r\nOtherwise, we're double logging each log line, as the message gets propagated up to the root logger's handler:\r\n\r\n```\r\n[2024-07-15 19:34:52,354][lema][INFO][torch_utils.py:50] Torch version: 2.4.0.dev20240612. NumPy version: 1.26.4\r\n2024-07-15:19:34:52,354 INFO     [torch_utils.py:50] Torch version: 2.4.0.dev20240612. NumPy version: 1.26.4\r\n```\r\n\r\nExample of another module doing this: https://github.com/huggingface/transformers/blob/6fbea6d237cbdfc3c229cdadfa3c968cfb2d5142/src/transformers/utils/logging.py#L104", "merged_at": "2024-07-16T18:11:41Z"}
{"number": 183, "title": "[Evaluations] Adding support for HuggingFace's leaderboard v1 benchmarks", "files": ["src/lema/evaluate.py", "src/lema/evaluation/huggingface_leaderboard.py", "tests/test_evaluate.py"], "area": "evaluation", "area_votes": {"evaluation": 2}, "body": "", "merged_at": "2024-07-16T08:06:32Z"}
{"number": 182, "title": "Update README.md to include `sky launch - 10 ...` example", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2024-07-15T20:15:29Z"}
{"number": 181, "title": "[Polaris] Move venv creation from worker to launcher.", "files": ["scripts/polaris/jobs/example_job.sh", "scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "Moved the creation of a virtual environment from each worker to the launcher. This way there will not be contention for venv creation when multiple workers are used.\r\n\r\nTowards OPE-121", "merged_at": "2024-07-15T19:18:22Z"}
{"number": 179, "title": "No longer ignore .git. in Polaris Needed for venv.", "files": ["scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "Turns out pip install needs the `.git` folder. This error only occurs at run time.", "merged_at": "2024-07-12T23:57:32Z"}
{"number": 174, "title": "Introduce LEMA_RUN_NAME env var to SkyPilot training configs", "files": ["configs/skypilot/sky.yaml", "configs/skypilot/sky_chatqa.yaml", "configs/skypilot/sky_fsdp_gpt2.yaml", "configs/skypilot/sky_fsdp_phi3_dpo.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml"], "area": "configs", "area_votes": {"configs": 7}, "body": "Allows overriding the name from command line `sky launch --env LEMA_RUN_NAME=xx` ", "merged_at": "2024-07-11T19:54:18Z"}
{"number": 173, "title": "Add a source directory to the Polaris launcher and clean up rsync copies.", "files": ["scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 1}, "body": "Add a -s flag for the source directory to copy.\r\n\r\nUpdates rsync to ignore folders with a leading \".\" like \".git/\".\r\n\r\nTowards OPE-107", "merged_at": "2024-07-11T17:10:45Z"}
{"number": 172, "title": "Remove GCP project reference", "files": ["configs/clouds/gcp/lema_dev_iam_custom_role.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "Towards OPE-64", "merged_at": "2024-07-10T23:03:11Z"}
{"number": 171, "title": "Make sure output training dir exists", "files": ["src/lema/train.py", "tests/test_train.py"], "area": "training", "area_votes": {"training": 1}, "body": "Otherwise, training may fail\r\n\r\nFixes OPE-128", "merged_at": "2024-07-10T23:19:08Z"}
{"number": 170, "title": "Improve launcher usability via command line arguments.", "files": ["scripts/polaris/jobs/example_job.sh", "scripts/polaris/launcher.sh"], "area": "infra", "area_votes": {"infra": 2}, "body": "Added command line arguments so users no longer need to edit the launcher.sh file.\r\n\r\nAdditionally, Polaris jobs will silently fail if the specified log locations don't exist. Updated the example job with a sample log location in our community directory on eagle.\r\n\r\nTowards OPE-107", "merged_at": "2024-07-10T23:58:34Z"}
{"number": 169, "title": "Submit config to create Custom IAM role for SkyPilot Service Accounts on GCP", "files": ["configs/clouds/gcp/lema_dev_iam_custom_role.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "Based on IAM permissions listed in https://skypilot.readthedocs.io/en/latest/cloud-setup/cloud-permissions/gcp.html\r\n\r\nTowards OPE-64", "merged_at": "2024-07-10T22:44:38Z"}
{"number": 168, "title": "Replacing `EvaluationConfig`'s `DataParams` with `DatasetSplitParams`", "files": ["configs/lema/gpt2.asynceval.nvidia.yaml", "configs/lema/phi3.eval.lema.mac.yaml", "configs/lema/phi3.eval.lema.nvidia.yaml", "configs/lema/phi3.eval.lm_harness.mac.yaml", "configs/lema/zephyr.7b/sft/eval_mmlu.yaml", "src/lema/core/types/configs.py", "src/lema/evaluate.py", "tests/test_evaluate.py", "tests/test_evaluate_async.py"], "area": "configs", "area_votes": {"configs": 5, "data": 1, "evaluation": 1}, "body": "Initially it seemed that this flexibility (being able to define multiple dataset splits for evaluation) is useful BUT it may create more confusion than help (Yes, Matt warned me and I didn't listen). For instance, when running experiments with LM Harness, users might think that they are using the **validation** dataset of the benchmark (since they are adding the `validation` term at `config.data.validation.datasets[..]` but they are actually NOT! LM harness is choosing the relevant split for evals. So, I am dropping the `validation` field to make things cleaner (until we really need this, which we might not). ", "merged_at": "2024-07-10T17:58:26Z"}
{"number": 167, "title": "Update pre-tokenized column name to be `input_ids` in `tokenize_dataset` tool", "files": ["src/experimental/tokenize_dataset.py"], "area": "other", "area_votes": {"other": 1}, "body": "\r\nTowards  OPE-102", "merged_at": "2024-07-10T17:51:49Z"}
{"number": 165, "title": "Update the vanilla eval config for gpt2 to run hellaswag evals.", "files": ["configs/lema/gpt2.asynceval.nvidia.yaml", "configs/skypilot/sky_gpt2_eval.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "Run using the command:\r\n```\r\nsky launch -c matthew-lema-cluster configs/skypilot/sky_gpt2_eval.yaml\r\n```\r\n\r\nEval results are available here:\r\nhttps://console.cloud.google.com/storage/browser/lema-dev-us-central1/matthew/gpt2.pt.4xA10080GB.5000steps/hellaswag?project=lema-dev\r\n\r\nFixes OPE-63", "merged_at": "2024-07-09T23:52:17Z"}
{"number": 164, "title": "Create a launcher script for Polaris jobs (ALCF)", "files": ["scripts/polaris/jobs/example_job.sh", "scripts/polaris/launcher.sh", "scripts/polaris/polaris_init.sh"], "area": "infra", "area_votes": {"infra": 3}, "body": "Note: this PR is still a draft. Putting it out early so folks can play with it as needed.\r\n\r\nTowards OPE-107", "merged_at": "2024-07-10T17:20:51Z"}
{"number": 163, "title": "Llama 3 Aya Fine-Tuning Updates", "files": ["configs/accelerate/sample_fsdp_llama3.yaml", "configs/lema/llama3.8b.aya.eval.yaml", "configs/lema/llama3.8b.aya.sft.yaml", "configs/skypilot/sky_llama3_8b_aya_eval.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml", "src/lema/core/types/params/training_params.py", "src/lema/datasets/pretraining_async_text_dataset.py"], "area": "configs", "area_votes": {"configs": 5, "data": 2}, "body": "* New FSDP config (for example for Llama 3)\r\n* Updated sky and other configs \r\n\r\nTowards OPE-73", "merged_at": "2024-07-16T15:46:53Z"}
{"number": 162, "title": "Update async dataset class to support pre-tokenized datasets", "files": ["src/lema/datasets/pretraining_async_text_dataset.py", "tests/builders/test_data.py"], "area": "data", "area_votes": {"data": 1}, "body": "The new datasets (temporarily) require us to pre-tokenize everything up-front. This PR updates the `PretrainingAsyncTextDataset` to allow for pre-tokenized datasets to be consumed.\r\n\r\n**Changes**\r\n- Add a pre-tokenized parameter to `PretrainingAsyncTextDataset`, and skip tokenization if the flag is set\r\n\r\n**Bugfixes**\r\n- Remove `__len__` function, as it is causing the trainer to infer the wrong length: the length of the underlying dataset is much larger than the length of the package dataset. This messes up with the epochs and LR schedules.\r\n- When resetting the iterator after an epoch, skip processing the current sample as it is not defined\r\n\r\n**Future fix**\r\n- I'm not quite sure we're supposed to automatically set a new iterator inside this class. In theory this should be done upstream by the DataLoader & trainer. With the current approach, we do not know when an epoch happened, and we can't update the `set_epoch` function on the data loader samplers. \r\n- As this could be a breaking change, will investigate doing that in a separate PR.\r\n\r\n\r\nTowards OPE-102", "merged_at": "2024-07-10T17:16:58Z"}
{"number": 161, "title": "Update Trainer.save_model to start using the public HF save_model() method (except for PEFT)", "files": ["src/lema/builders/training.py"], "area": "other", "area_votes": {"other": 1}, "body": "-- From a quick look, the methods seem mostly equivalent for our purposes but better to use public methods when possible (vs private HF._save())\r\n-- Tested OK for pretraining ", "merged_at": "2024-07-08T23:18:47Z"}
{"number": 159, "title": "Update ChatQA training configs", "files": ["configs/lema/chatqa/gpt2.chatqa.stage1.yaml", "configs/lema/chatqa/gpt2.chatqa.stage2.yaml", "configs/skypilot/sky_chatqa.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "**Stage 1**\r\n- W&B Run: https://wandb.ai/lema-academic/lema-train-test/runs/dmlehvxi\r\n![image](https://github.com/openlema/lema/assets/4415327/6ba2bd47-c0d3-4fdb-9be0-399f79966400)\r\n\r\n**Stage 2**\r\n- W&B Run: https://wandb.ai/lema-academic/lema-train-test/runs/ovmbbnl2\r\n\r\n![image](https://github.com/openlema/lema/assets/4415327/b1c9b707-244c-463e-aa73-9720fd3e54f0)\r\n\r\nTowards OPE-41\r\n", "merged_at": "2024-07-10T16:34:32Z"}
{"number": 158, "title": "[bug] Include jinja templates in build", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Update package config to include jinja files.", "merged_at": "2024-07-08T16:25:18Z"}
{"number": 156, "title": "Update sky_init.sh to print task id and cluster info", "files": ["configs/skypilot/sky_init.sh"], "area": "configs", "area_votes": {"configs": 1}, "body": "", "merged_at": "2024-07-06T00:15:25Z"}
{"number": 154, "title": "Disable model.compile in gpt2 config", "files": ["configs/lema/gpt2.pt.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Until the following is understood: https://linear.app/openlema/issue/OPE-92/support-torchcompile#comment-0af092c0 \r\n\r\nTowards OPE-92", "merged_at": "2024-07-05T22:38:48Z"}
{"number": 152, "title": "Zephyr Configs [full-model, skypilot]", "files": ["configs/lema/zephyr.7b/sft/eval_mmlu.yaml", "configs/lema/zephyr.7b/sft/evaluate.yaml", "configs/lema/zephyr.7b/sft/full.yaml", "configs/lema/zephyr.7b/sft/qlora.yaml", "configs/skypilot/sky_zephyr_7b_qlora_sft.yaml", "configs/skypilot/sky_zephyr_7b_sft.yaml"], "area": "configs", "area_votes": {"configs": 6}, "body": "These can be plugged an played if one has access to 8 (80GB for full) A100 GPUs. \r\n\r\nAs left in comments, one should adjust the `per_device_train_batch_size` & `gradient_accumulation_steps` for a different setup without the need to change the learning-rate and other training HPs.", "merged_at": "2024-07-05T16:09:49Z"}
{"number": 150, "title": "Update async eval to properly parse eval configs", "files": ["src/lema/evaluate.py", "src/lema/evaluate_async.py"], "area": "evaluation", "area_votes": {"evaluation": 2}, "body": "A recent change to our eval logic broke async_evaluation. I've added a main \"evaluate(...)\" method to evaluate.py to encapsulate all branching logic.", "merged_at": "2024-07-03T23:06:26Z"}
{"number": 149, "title": "Update SkyPilot training configs to include `run_name`", "files": ["configs/skypilot/sky.yaml", "configs/skypilot/sky_fsdp_gpt2.yaml", "configs/skypilot/sky_fsdp_phi3_dpo.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_llama2b.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml"], "area": "configs", "area_votes": {"configs": 6}, "body": "", "merged_at": "2024-07-03T19:58:26Z"}
{"number": 148, "title": "Multiple cleanup changes in configs/skypilot/sky_gpt2.yaml", "files": ["configs/lema/gpt2.pt.mac.yaml", "configs/lema/gpt2.pt.yaml", "configs/skypilot/sky_gpt2.yaml", "src/lema/core/types/params/training_params.py"], "area": "configs", "area_votes": {"configs": 3, "data": 1}, "body": "-- Define `ddp_find_unused_parameters` and set it to False\r\n-- Reduce checkpoint saving frequency from 100 to 200 steps\r\n-- Include SKYPILOT_TASK_ID into `run_name`\r\n-- Define `gradient_checkpointing_kwargs.use_reentrant` . The param will soon become required \r\n\r\nTowards OPE-61\r\nTowards OPE-105", "merged_at": "2024-07-03T03:26:35Z"}
{"number": 145, "title": "Remove private debug dir from configs/skypilot/sky_gpt2.yaml", "files": ["configs/skypilot/sky_gpt2.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "", "merged_at": "2024-07-02T20:08:23Z"}
{"number": 142, "title": "Created a new dataset class with async loading & tokenization", "files": ["src/lema/datasets/pretraining_async_text_dataset.py", "tests/datasets/test_pretraining_async_text_dataset.py"], "area": "data", "area_votes": {"data": 1}, "body": "Current ConstantLengthDataset class fails to load data in the background, leading to GPU downtime during training.\r\n\r\nTowards OPE-104", "merged_at": "2024-07-02T17:26:37Z"}
{"number": 140, "title": "Minor bug fix in writing evaluations", "files": ["src/lema/evaluate.py", "tests/test_evaluate.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "Ensure all parent directories exist before writing evaluations.\r\n\r\nVerified via small runs in sky.\r\n\r\nTowards OPE-68", "merged_at": "2024-06-28T23:33:18Z"}
{"number": 139, "title": "Upload sample configs for running async evals on GPT2", "files": ["configs/lema/gpt2.asynceval.nvidia.yaml", "configs/skypilot/sky_gpt2_eval.yaml"], "area": "configs", "area_votes": {"configs": 2}, "body": "Add several configs as PoCs for async eval.\r\n\r\nNote: our vanilla MMLU evaluation runs are currently taking 5+ hours. We need to dig in further to understand how we can better improve inference during evaluation.\r\n\r\nTowards OPE-68", "merged_at": "2024-06-28T23:45:40Z"}
{"number": 138, "title": "Add a new top level command: evaluate_async", "files": ["src/lema/__init__.py", "src/lema/evaluate.py", "src/lema/evaluate_async.py", "tests/test_evaluate.py", "tests/test_evaluate_async.py"], "area": "evaluation", "area_votes": {"evaluation": 2}, "body": "- Updates evaluate to write metrics to a JSON (temporary, we can dive into the expected behavior later)\r\n- Polls the provided directory for checkpoints, respecting polling_interval and num_retries.\r\n\r\nTODO: Add better tests. We need something akin to an integration test here to test loading checkpoints.\r\n\r\nTowards OPE-68", "merged_at": "2024-06-28T17:40:52Z"}
{"number": 137, "title": "Add an initial config for async evaluations", "files": ["src/lema/core/types/__init__.py", "src/lema/core/types/configs.py", "tests/integration/configs/test_parse_configs.py"], "area": "data", "area_votes": {"data": 1}, "body": "An async evaluation is intended to be a job that runs in parallel with training. It exposes the option to poll a directory for new checkpoints, computing evals until it fails to find a new checkpoint `num_retries` number of times.\r\n\r\nTowards OPE-68", "merged_at": "2024-06-28T03:41:55Z"}
{"number": 136, "title": "Apply `torch.distributed.barrier()` in save_model", "files": ["src/lema/builders/training.py"], "area": "other", "area_votes": {"other": 1}, "body": "", "merged_at": "2024-07-01T16:13:13Z"}
{"number": 133, "title": "Minor updates in SkyPilot docstrings", "files": ["configs/skypilot/sky.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_llama3_8b_aya_sft.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "Towards OPE-67", "merged_at": "2024-06-27T16:03:03Z"}
{"number": 131, "title": "Update HF save_model() to only save on master replica", "files": ["src/lema/builders/training.py", "src/lema/utils/torch_utils.py"], "area": "other", "area_votes": {"other": 2}, "body": "\r\nFixes OPE-78", "merged_at": "2024-06-26T20:08:53Z"}
{"number": 128, "title": "[Quick fix] Handle pynvml being misconfigured", "files": ["src/lema/utils/debugging_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "Fixes the test failures in https://github.com/openlema/lema/pull/124", "merged_at": "2024-06-26T03:27:03Z"}
{"number": 126, "title": "Add a registry for metric functions that we can run during training.", "files": ["src/lema/core/registry.py", "tests/test_registry.py"], "area": "other", "area_votes": {"other": 1}, "body": "In order to run evaluations during training, we need to:\r\n\r\n1) Create a new registry for eval functions. <---------------- You are here\r\n\r\n2) Add a training param so the user can specify the eval function by name (from our registry) <---------------TODO\r\n\r\n3) Update our training loop to pass through compute_metrics and read the eval_function from our registry <----TODO\r\n\r\nTowards OPE-68", "merged_at": "2024-06-25T23:52:57Z"}
{"number": 125, "title": "Update training_params.py so HF trainer uses num_train_epochs", "files": ["src/lema/core/types/params/training_params.py"], "area": "data", "area_votes": {"data": 1}, "body": "Export the `num_train_epochs` via the to_hf(). Without this, the HF trainer is using its default (num_train_epochs=3), affecting the underlying learning-rate-scheduler, etc.", "merged_at": "2024-06-26T01:04:43Z"}
{"number": 122, "title": "Refactor model registry", "files": ["src/lema/builders/models.py", "src/lema/core/models/__init__.py", "src/lema/core/models/sample.py", "src/lema/core/registry.py", "src/lema/core/types/params/model_params.py", "tests/test_registry.py", "tests/test_train.py"], "area": "other", "area_votes": {"other": 3, "data": 1}, "body": "**Changes**\r\n- Refactor to independently fetch model and model config classes\r\n- Update registry to match python dict conventions:\r\n  - registry.get returns None if key not found\r\n  - added __getitem__, which raise KeyError if key not found\r\n- Simplify api by making the module level registry a singleton\r\n\r\n**Minor changes**\r\n- Unit tests used to have an exact match on the error string, which made it hard to maintain after changing some of the logic. Changed to simply \r\n- Remove unused sample classes\r\n", "merged_at": "2024-06-25T23:05:56Z"}
{"number": 120, "title": "Update sky_init.sh to print current dir", "files": ["configs/skypilot/sky_init.sh"], "area": "configs", "area_votes": {"configs": 1}, "body": "", "merged_at": "2024-06-25T17:41:51Z"}
{"number": 118, "title": "Add llama3-instruct jinja template", "files": ["src/lema/datasets/chat_templates/llama3-instruct.jinja"], "area": "data", "area_votes": {"data": 1}, "body": "Cleaned/made readable from\r\nhttps://github.com/chujiezheng/chat_templates/blob/main/chat_templates/llama-3-instruct.jinja", "merged_at": "2024-06-25T17:14:43Z"}
{"number": 117, "title": "Add GCP deps to `lema[cloud]`", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "\r\nTowards OPE-64", "merged_at": "2024-06-25T15:40:53Z"}
{"number": 115, "title": "Saving inference probs in `parquet` format.", "files": ["src/lema/utils/saver.py", "tests/test_saver.py"], "area": "other", "area_votes": {"other": 1}, "body": "", "merged_at": "2024-06-25T23:02:01Z"}
{"number": 114, "title": "Split types file into module", "files": ["src/lema/core/types.py", "src/lema/core/types/__init__.py", "src/lema/core/types/base_config.py", "src/lema/core/types/configs.py", "src/lema/core/types/exceptions.py", "src/lema/core/types/params/data_params.py", "src/lema/core/types/params/model_params.py", "src/lema/core/types/params/peft_params.py", "src/lema/core/types/params/training_params.py"], "area": "data", "area_votes": {"other": 1, "data": 7}, "body": "**Problem**\r\nThe `lema.core.types` file is becoming pretty large.\r\n\r\n**Solution**\r\nBefore adding even more type definitions in an upcoming PR, this PR break down the file into multiple submodules. \r\n\r\nP.S: open to suggestions on a better way to split it", "merged_at": "2024-06-25T00:46:01Z"}
{"number": 113, "title": "Support shuffling and random seeds for dataset sampling", "files": ["src/lema/builders/data.py", "src/lema/core/types.py", "tests/builders/test_data.py"], "area": "other", "area_votes": {"other": 2}, "body": "Adds three new fields:\r\n - At the dataset level the user can specify `seed`. This random seed is used for random actions when interweaving dataset splits\r\n - At a dataset-split level users can specify a `seed`. This random seed is used for random actions (shuffling) when sampling the dataset. Only used if `shuffle` is true\r\n - At a dataset-split level users can specify `shuffle`. If specified, the dataset is shuffled before sampling.\r\n\r\nFixes OPE-43", "merged_at": "2024-06-24T22:54:10Z"}
{"number": 111, "title": "Update skypilot/sky_gpt2.yaml to include an example how to mount GCS dir", "files": ["configs/skypilot/sky_gpt2.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- add some other changes for consistency with `sky.yaml`\r\n\r\nContributes to OPE-61, OPE-64, OPE-57", "merged_at": "2024-06-24T20:25:43Z"}
{"number": 109, "title": "Rename dataset_params.dataset_config to dataset_params.subset", "files": ["configs/lema/gpt2.pt.yaml", "src/lema/builders/data.py", "src/lema/builders/models.py", "src/lema/core/types.py", "tests/builders/test_data.py", "tests/test_train.py"], "area": "other", "area_votes": {"configs": 1, "other": 3}, "body": "1. We currently follow HF's naming convention for `dataset_params.dataset_config`. This allows to specify a subfolder to load, within the root `dataset_name` folder. \r\n\r\nThis is commonly used for datasets with multiple subsets (e.g. MMLU). \r\n\r\nIn preparation for our own datasets abstraction, this PR renames the variable to `subset` to better reflect what it actually does instead of the legacy HF name.\r\n\r\n2. Move `device_map` from hardcoded within the model loader to the config. ", "merged_at": "2024-06-24T21:41:36Z"}
{"number": 107, "title": "Handling Gradient Checkpointing", "files": ["src/lema/train.py"], "area": "training", "area_votes": {"training": 1}, "body": "This is a small PR where I am proposing that we do not use `model.enable_input_require_grads()` whenever the user requests doing gradient checkpointing, and make the code more fine-grained. \r\n\r\nFirst, my current understanding is that for both LoRA/PEFT and full models, when they are finetuned: \r\n- finetuning (also) their input embeddings is *optional* (despite that in most cases doing so esp. if new vocab symbols are used is beneficial).\r\n- for LoRA models, in particular, if `gradient checkpointing` is requested it is recommended (by HF, see: https://huggingface.co/docs/transformers/en/main_classes/model ) to activate the training of the input embeddings, and in fact, if such LoRA models are downloaded directly from HF they will enable gradients on the input-embs _by default_ (see: https://github.com/huggingface/transformers/blob/74a207404e8d4524d1fdc4aa23789694f9eef347/src/transformers/modeling_utils.py#L2273C27-L2273C49).\r\n- However, the above action is not *necessary* for LoRA/PEFT as long as the kwargs_arg `use_reentrant` of the gradient_accumulation is set to False (which is now the recommended default of HF, See: https://pytorch.org/docs/stable/checkpoint.html). In that case; the comp. graph will be able to flow the gradient even in layers (such as the input embs.) where the `requires_grad` variable is set to False. \r\n\r\nProposed solution - to *extend* this PR:\r\n- add a separate TrainingParam controlling the training or not of the input embeddings _explicitly_ (default to False).\r\n- Add `use_reentrant`  in our `config.training.gradient_checkpointing_kwargs` with a default of False.\r\n\r\nContributes to OPE-34", "merged_at": "2024-06-24T19:32:25Z"}
{"number": 106, "title": "Minor: Updates on Zephyr-Config", "files": ["configs/lema/zephyr.7b/sft/evaluate.yaml", "configs/lema/zephyr.7b/sft/full.yaml", "configs/lema/zephyr.7b/sft/qlora.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "With this minor PR, our configs for Zephyr.7B (SFT) are a step closer to being ready to be used by the \"world\".\r\n\r\n- They all follow the new Mixture data convention and make use of the defaults (stream/pack= False).\r\n- The locations for storing/logging/restoring the model are more user-friendly\r\n- Extra comments on the use of `reentrant=False` and why the `grad_accumulation_steps` should be equal to 16 on a single GPU.", "merged_at": "2024-06-24T10:51:10Z"}
{"number": 103, "title": "nanoGPT (GPT2) pretraining recipe", "files": ["configs/lema/gpt2.pt.yaml", "configs/lema/llama2b.pt.yaml", "configs/skypilot/sky.yaml", "configs/skypilot/sky_gpt2.yaml", "configs/skypilot/sky_llama2b.yaml", "src/lema/builders/models.py", "src/lema/core/types.py"], "area": "configs", "area_votes": {"configs": 5, "other": 2}, "body": "Contributes to OPE-61.\r\nFixes OPE-62.\r\n\r\nOn an A100-80GB, results in ~1.5 steps/s at batch size 44.", "merged_at": "2024-06-24T06:53:19Z"}
{"number": 97, "title": "Update configs/skypilot/sky_llama2b.yaml to start using sky_init.sh", "files": ["configs/skypilot/sky_llama2b.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "for consistency with sky.yaml ", "merged_at": "2024-06-20T21:57:39Z"}
{"number": 95, "title": "Add support for dataset mixtures during training", "files": ["configs/lema/gpt2.pt.yaml", "configs/lema/phi3.dpo.mac.32g.yaml", "configs/lema/phi3.dpo.nvidia.24g.yaml", "configs/lema/phi3.dpo.nvidia.80g.yaml", "configs/lema/phi3.dpo.yaml", "configs/lema/phi3.eval.yaml", "configs/lema/phi3.lora.yaml", "configs/lema/phi3.sft.nvidia.24g.yaml", "configs/lema/zephyr.7b/sft/config_full.yaml", "configs/lema/zephyr.7b/sft/config_qlora.yaml", "scripts/finetune_sft_phi3.sh", "src/lema/builders/data.py", "src/lema/core/types.py", "src/lema/evaluate.py", "tests/builders/test_data.py", "tests/test_config.py", "tests/test_evaluate.py", "tests/test_train.py"], "area": "configs", "area_votes": {"configs": 10, "infra": 1, "other": 2, "evaluation": 1}, "body": "This PR makes two key changes:\r\n1) The \"data\" object now contains a repeated field called \"datasets\" which contains parameters specific to each dataset. The top level \"data\" object still contains several parameters that should be applied across all datasets.\r\n\r\n2) Each dataset object exposes two new parameters for sampling and mixing:\r\n  - `sample_count`: The number of examples to sample from the dataset. Supports oversampling. If not specified, the entire dataset will be used by default. Note that I did *not* add support for a sample_proportion (percentage of the dataset) as this cannot easily be computed for streaming datasets.\r\n  - `mixture_proportion`: The proportion of examples from this dataset relative to other datasets in the mixture. For example, if Dataset A has mixture_proportion=.1 and Dataset B has mixture_proportion = .9, the new dataset will be constructed by sampling from Dataset A 10% of the time and Dataset B 90% of the time.\r\n \r\nNote that the `data` object also exposes `mixture_strategy`. This field is the same as Hugging Face's `stopping_strategy` and dictates how datasets are sampled when `mixture_proportion` is also defined.\r\n\r\n\r\nTowards OPE-43\r\n\r\nApologies for the large PR. Changing the dataset structure caused a breaking change in the repo.", "merged_at": "2024-06-21T18:13:32Z"}
{"number": 94, "title": "Initial version of SkyPilot config for multi-node training (num_nodes: N)", "files": ["configs/skypilot/sky.yaml", "configs/skypilot/sky_init.sh", "configs/skypilot/sky_llama2b.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "-- Refactor common env variable initialization into `sky_init.sh`\r\n-- Remove `--standalone` flag (not compatible with multi node)\r\n-- Minor changes to llama2b config\r\n\r\nTested on GCP: gpt2 with 2 nodes (1 T4 GPU each):  ` 2x GCP(n1-highmem-4, {'T4': 1}, disk_size=200)`\r\n\r\nHaven't been able to find usable multi-node config on RunPod yet.\r\n\r\nTowards OPE-55", "merged_at": "2024-06-20T01:25:24Z"}
{"number": 93, "title": "Add license", "files": ["LICENSE"], "area": "docs", "area_votes": {"docs": 1}, "body": "Add standard Apache 2.0 file", "merged_at": "2024-06-20T01:11:24Z"}
{"number": 86, "title": "Update README.md to include basic instructions for multi-GPU training (DDP, FSDP)", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "\r\n\r\nTowards OPE-16", "merged_at": "2024-06-18T15:43:46Z"}
{"number": 84, "title": "Minor cleanup updates to SkyPilot configs", "files": ["configs/skypilot/sky.yaml", "configs/skypilot/sky_fsdp_gpt2.yaml", "configs/skypilot/sky_fsdp_phi3_dpo.yaml"], "area": "configs", "area_votes": {"configs": 3}, "body": "", "merged_at": "2024-06-17T19:03:07Z"}
{"number": 82, "title": "Create FSDP configs for Phi3", "files": ["configs/accelerate/sample_fsdp_gpt2.yaml", "configs/accelerate/sample_fsdp_phi3_dpo.yaml", "configs/skypilot/sky_fsdp_gpt2.yaml", "configs/skypilot/sky_fsdp_phi3_dpo.yaml"], "area": "configs", "area_votes": {"configs": 4}, "body": "+ minor changes : rename some files (use more specific names), tweak some params, etc\r\n\r\nPhi3 structure: https://dpaste.org/SyeUC/raw\r\n\r\nTowards OPE-16", "merged_at": "2024-06-17T16:28:27Z"}
{"number": 79, "title": "Refactor code to get device rank and world size into a helper function", "files": ["src/lema/builders/models.py", "src/lema/utils/torch_utils.py"], "area": "other", "area_votes": {"other": 2}, "body": "Contributes to OPE-16", "merged_at": "2024-06-14T01:23:31Z"}
{"number": 78, "title": "Adding inference support for next logit probability.", "files": ["src/lema/__init__.py", "src/lema/evaluation/__init__.py", "src/lema/evaluation/infer_prob.py", "tests/test_infer_prob.py"], "area": "evaluation", "area_votes": {"evaluation": 1}, "body": "\r\n\r\nContributes to OPE-35", "merged_at": "2024-06-17T09:28:34Z"}
{"number": 73, "title": "[Inference] Remove the prepended prompts from model responses.", "files": ["src/lema/infer.py", "tests/test_infer.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "", "merged_at": "2024-06-10T22:36:18Z"}
{"number": 71, "title": "Minor changes to the default configs/skypilot/sky.yaml config", "files": ["configs/skypilot/sky.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Change default value of include_performance_metrics to `false` (it doesn't always work by default e.g., for models with multiple inputs) \r\n-- Document A100 SXM GPU", "merged_at": "2024-06-10T18:33:34Z"}
{"number": 70, "title": "Prototype to pass `config.model.model_max_length` to Trainers", "files": ["src/lema/builders/training.py", "src/lema/core/types.py", "tests/test_train.py"], "area": "other", "area_votes": {"other": 2}, "body": "Currently, we're getting warnings (e.g., from SFTTrainer) that `max_seq_length` is undefined (and it defaults to 1024)\r\n\r\nThis PR is an attempt to pass the recently introduced  `model.model_max_length` param to Trainers (in addition to datasets and tokenizers).  \r\nNOTE: Please review carefully (I'm not sure if the code is optimal or right)", "merged_at": "2024-06-10T19:11:15Z"}
{"number": 65, "title": "Disallow relative imports in LeMa", "files": ["pyproject.toml", "src/lema/builders/__init__.py"], "area": "infra", "area_votes": {"infra": 1}, "body": "-- Convert existing relative imports to absolute", "merged_at": "2024-06-07T18:24:06Z"}
{"number": 60, "title": "Minor formatting improvements in README.md", "files": ["CONTRIBUTING.md", "README.md", "STYLE_GUIDE.md"], "area": "docs", "area_votes": {"docs": 3}, "body": "", "merged_at": "2024-06-06T14:14:14Z"}
{"number": 59, "title": "Update README.md to include Linux/WSL specific instructions", "files": ["README.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "-- Re-order some setup steps so it works for both Mac and Linux\r\n--  Minor formatting changes", "merged_at": "2024-06-06T02:00:30Z"}
{"number": 58, "title": "Update inference to support non-interactive batch mode.", "files": ["src/lema/__init__.py", "src/lema/infer.py", "tests/test_infer.py"], "area": "inference", "area_votes": {"inference": 1}, "body": "", "merged_at": "2024-06-05T23:25:29Z"}
{"number": 57, "title": "Remove reference to torch.cuda.clock_rate", "files": ["src/lema/utils/torch_utils.py"], "area": "other", "area_votes": {"other": 1}, "body": "The property is  used for debug logging only, only marginally useful, and not supported on AMD GPUs", "merged_at": "2024-06-05T20:44:06Z"}
{"number": 56, "title": "fix trl version as 0.8.6", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "trl==0.9.3 leads to errors during training:\r\n \" test_train.py::test_basic_train - AttributeError: 'TrainingArguments' object has no attribute 'packing'\"\r\nand for many other attributes... \r\n\r\nTODO: Figure out how to resolve it better", "merged_at": "2024-06-05T20:26:38Z"}
{"number": 54, "title": "Make some NVIDIA-specific dependencies optional", "files": ["src/lema/utils/debugging_utils.py", "src/lema/utils/torch_utils.py"], "area": "other", "area_votes": {"other": 2}, "body": "", "merged_at": "2024-06-05T13:26:03Z"}
{"number": 50, "title": "Update Pre-review Tests to only run on pull_request", "files": [".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "", "merged_at": "2024-06-04T19:21:51Z"}
{"number": 46, "title": "Create initial version of CONTRIBUTING.md", "files": ["CONTRIBUTING.md", "STYLE_GUIDE.md"], "area": "docs", "area_votes": {"docs": 2}, "body": "Also, update STYLE_GUIDE with some PyRight info", "merged_at": "2024-06-04T18:33:13Z"}
{"number": 44, "title": "Configure initial GitHub Actions workflow to run pre-commits and tests", "files": [".github/workflows/pretest.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "", "merged_at": "2024-06-03T22:58:10Z"}
{"number": 43, "title": "fix trailing whitespace warning in STYLE_GUIDE.md", "files": ["STYLE_GUIDE.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "... to make pre-commits pass", "merged_at": "2024-06-03T19:53:19Z"}
{"number": 42, "title": "Adding registry for custom models.", "files": ["src/lema/builders/models.py", "src/lema/core/models/__init__.py", "src/lema/core/models/sample.py", "src/lema/core/registry.py", "src/lema/core/types.py", "tests/test_registry.py", "tests/test_train.py"], "area": "other", "area_votes": {"other": 4}, "body": "", "merged_at": "2024-06-04T18:53:36Z"}
{"number": 37, "title": "Add experimental code for llama cpp inference", "files": ["src/lema/experimental/llama_cpp_inference.py"], "area": "other", "area_votes": {"other": 1}, "body": "", "merged_at": "2024-05-30T23:08:56Z"}
{"number": 36, "title": "Create skeleton of  STYLE_GUIDE.md", "files": ["STYLE_GUIDE.md"], "area": "docs", "area_votes": {"docs": 1}, "body": "", "merged_at": "2024-05-30T23:26:50Z"}
{"number": 32, "title": "Update SkyPilot cconfig to start using torchrun", "files": ["configs/skypilot/sky.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "it's required for distributed training. For now, it still runs on 1 GPUs (no change)", "merged_at": "2024-05-30T00:06:54Z"}
{"number": 28, "title": "Add a missing lema-infer command under [project.scripts]", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "also sort the commands alphabetically", "merged_at": "2024-05-29T19:17:21Z"}
{"number": 27, "title": "Update GPU types list in the default SkyPilot config", "files": ["configs/skypilot/sky.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "-- Remove L4 which has 24G memory (not enough for the default model in the config)\r\n-- Add A40 (available on RunPod currently)\r\n-- Add sample commands", "merged_at": "2024-05-29T19:16:56Z"}
{"number": 26, "title": "Fix leftover  '_torch_dtype'  in 'ModelParams'", "files": ["configs/lema/phi3.dpo.nvidia.24g.yaml"], "area": "configs", "area_votes": {"configs": 1}, "body": "", "merged_at": "2024-05-29T00:02:59Z"}
{"number": 23, "title": "Augmenting Types for training hyperparams", "files": ["src/lema/core/types.py"], "area": "other", "area_votes": {"other": 1}, "body": "", "merged_at": "2024-05-28T16:03:43Z"}
{"number": 20, "title": "adding pynvml to train env", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "", "merged_at": "2024-05-24T22:51:03Z"}
{"number": 18, "title": "Configure max file size in precommit hooks", "files": [".pre-commit-config.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Explicitly configure max file size for clarity (500K is the current default limit)", "merged_at": "2024-05-23T21:51:16Z"}
{"number": 16, "title": "Panos dev", "files": ["src/lema/builders/training.py"], "area": "other", "area_votes": {"other": 1}, "body": "Minor syntax improvement in docstring", "merged_at": "2024-05-23T15:25:21Z"}
{"number": 13, "title": "Add logging config", "files": [".vscode/launch.json", "pyproject.toml", "src/lema/__init__.py", "src/lema/core/types.py", "src/lema/logging.py", "src/lema/train.py", "src/lema/utils/debugging_utils.py", "src/lema/utils/saver.py", "src/lema/utils/torch_utils.py"], "area": "other", "area_votes": {"other": 6, "infra": 1, "training": 1}, "body": "**Changes**\r\n- Configure logger object for the repo\r\n- Enable logging to `tensorboard` and `wandb` via the hugging face trainers\r\n\r\n\r\n**Minor Changes**\r\n- Fix isort linter config\r\n- Add dependency warning filter to avoid polluting logs. Disabled in debug level\r\n- Minor bugfixes\r\n\r\n** Known Issues **\r\n- tensorboard, wandb, and the trainers all log to different folders currently. We need to consolidate all the logs to be under a single `output_dir` folder", "merged_at": "2024-05-22T22:09:56Z"}
{"number": 12, "title": "Sort pre-commit hooks lexicographically", "files": [".pre-commit-config.yaml"], "area": "infra", "area_votes": {"infra": 1}, "body": "Sort pre-commit hooks. Should be a no-op, mostly to get familiar with Github flow", "merged_at": "2024-05-21T22:20:21Z"}
{"number": 10, "title": "Adding torch as top-level module dependency", "files": ["pyproject.toml"], "area": "infra", "area_votes": {"infra": 1}, "body": "without specifying the version for increased\r\nhardware flexibility.", "merged_at": "2024-05-20T18:44:40Z"}
{"number": 5, "title": "Update static type checking config", "files": [".pre-commit-config.yaml", "pyproject.toml"], "area": "infra", "area_votes": {"infra": 2}, "body": "- Switch using pyright instead of mypy (see difference [here](https://github.com/microsoft/pyright/blob/main/docs/mypy-comparison.md#)) \r\n- Make sure to install extension: https://marketplace.visualstudio.com/items?itemName=ms-pyright.pyright\r\n- Note: type errors / missing type annotation are not currently blocking to give time to fix the full codebase in a future PR. This will lint only.", "merged_at": "2024-05-15T21:59:50Z"}
{"number": 4, "title": "Add example jupyter / colab notebook", "files": ["notebooks/LeMa - Colab Setup Example.ipynb"], "area": "docs", "area_votes": {"docs": 1}, "body": "This is a small PR to add an example jupyter / colab notebook to train & evaluate LeMa models", "merged_at": "2024-05-15T22:00:04Z"}