File size: 2,719 Bytes
0d1866c 09aec91 0d1866c 09aec91 0d1866c 09aec91 0d1866c 09aec91 0d1866c 09aec91 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | # Hugging Face Compute Quickstart
Use this when you want to run the 24-card local-intelligence eval pack on Hugging Face
compute without Colab or notebook sessions.
## Best option: Hugging Face Jobs
Hugging Face Jobs are available to Pro users and let you run batch commands on HF
infrastructure with a selected hardware flavor.
Recommended first models:
- `google/gemma-4-E4B-it`
- `nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1`
- `Qwen/Qwen3.5-4B-Instruct-2507`
## Launch a job from this repo
```bash
python3 eval/local_intelligence/launch_hf_job.py \
--sync-code \
--image ghcr.io/<your-github-owner>/training-lab-local-intelligence-eval:latest \
--flavor a10g-small \
--timeout 2h \
--limit 24
```
This launches a Hugging Face Job that:
- syncs the current eval harness into your HF repo
- downloads the synced harness from `arach/training-lab`
- starts from your prebuilt eval image
- runs the matrix runner on HF compute
- uploads result JSON files back into `eval/local_intelligence/results/jobs/$JOB_ID`
## Build the custom image once
```bash
chmod +x eval/local_intelligence/build_image.sh
eval/local_intelligence/build_image.sh arach/local-intelligence-eval:latest
docker push arach/local-intelligence-eval:latest
```
The Dockerfile bakes in the Python runtime dependencies so future Jobs can skip install time.
## Or publish the image with GitHub Actions
Use:
- [.github/workflows/build-local-intelligence-image.yml](/Users/arach/dev/training-lab/.github/workflows/build-local-intelligence-image.yml)
Published image shape:
```text
ghcr.io/<your-github-owner>/training-lab-local-intelligence-eval:latest
```
If you want to keep results in a separate repo:
```bash
python3 eval/local_intelligence/launch_hf_job.py \
--sync-code \
--source-repo-id arach/training-lab \
--results-repo-id arach/training-lab-results \
--image ghcr.io/<your-github-owner>/training-lab-local-intelligence-eval:latest \
--flavor a10g-small \
--timeout 2h \
--limit 24
```
## Run a single model on HF compute manually
```bash
python3 eval/local_intelligence/run_hf_transformers.py \
--model google/gemma-4-E4B-it \
--limit 24 \
--verbose
```
## Run the default matrix manually
```bash
python3 eval/local_intelligence/run_hf_matrix.py \
--limit 24 \
--keep-going
```
## Notes
- The runner prefers a tokenizer chat template when the model exposes one.
- Use `--trust-remote-code` if a model requires it.
- Result files land under `eval/local_intelligence/results/`.
- The HF Jobs path is the best fit for repeatable batch evals; notebooks are optional, not required.
- The best long-term path is a custom image with deps preinstalled, so Jobs mostly spend time on model load and eval.
|