Instructions to use AppleMind-AI/AppleMind-1.0-Mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AppleMind-AI/AppleMind-1.0-Mini with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AppleMind-AI/AppleMind-1.0-Mini")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AppleMind-AI/AppleMind-1.0-Mini")
model = AutoModelForCausalLM.from_pretrained("AppleMind-AI/AppleMind-1.0-Mini", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AppleMind-AI/AppleMind-1.0-Mini with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AppleMind-AI/AppleMind-1.0-Mini"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AppleMind-AI/AppleMind-1.0-Mini

SGLang

How to use AppleMind-AI/AppleMind-1.0-Mini with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AppleMind-AI/AppleMind-1.0-Mini" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AppleMind-AI/AppleMind-1.0-Mini" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use AppleMind-AI/AppleMind-1.0-Mini with Docker Model Runner:
```
docker model run hf.co/AppleMind-AI/AppleMind-1.0-Mini
```

AppleMind-1.0-Mini / LOGS.md

GGUFGuy

Create LOGS.md

02508d4 verified about 6 hours ago

preview code

Raw

History Blame Contribute Delete

53.9 kB

	Requirement already satisfied: transformers in /usr/local/lib/python3.12/dist-packages (5.13.1)
	Collecting transformers
	Downloading transformers-5.15.0-py3-none-any.whl.metadata (32 kB)
	Requirement already satisfied: datasets in /usr/local/lib/python3.12/dist-packages (4.0.0)
	Collecting datasets
	Downloading datasets-5.0.1-py3-none-any.whl.metadata (23 kB)
	Requirement already satisfied: accelerate in /usr/local/lib/python3.12/dist-packages (1.14.0)
	Requirement already satisfied: huggingface_hub in /usr/local/lib/python3.12/dist-packages (1.23.0)
	Collecting huggingface_hub
	Downloading huggingface_hub-1.27.0-py3-none-any.whl.metadata (16 kB)
	Requirement already satisfied: hf_xet in /usr/local/lib/python3.12/dist-packages (1.5.1)
	Collecting hf_xet
	Downloading hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (4.9 kB)
	Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.12/dist-packages (from transformers) (2.0.2)
	Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from transformers) (26.2)
	Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.12/dist-packages (from transformers) (6.0.3)
	Requirement already satisfied: regex>=2025.10.22 in /usr/local/lib/python3.12/dist-packages (from transformers) (2025.11.3)
	Requirement already satisfied: tokenizers<=0.23.0,>=0.22.0 in /usr/local/lib/python3.12/dist-packages (from transformers) (0.22.2)
	Requirement already satisfied: typer in /usr/local/lib/python3.12/dist-packages (from transformers) (0.26.8)
	Requirement already satisfied: safetensors>=0.8.0 in /usr/local/lib/python3.12/dist-packages (from transformers) (0.8.0)
	Requirement already satisfied: tqdm>=4.60 in /usr/local/lib/python3.12/dist-packages (from transformers) (4.67.3)
	Requirement already satisfied: filelock in /usr/local/lib/python3.12/dist-packages (from datasets) (3.29.7)
	Collecting pyarrow>=21.0.0 (from datasets)
	Downloading pyarrow-25.0.1-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (3.0 kB)
	Requirement already satisfied: dill<0.4.2,>=0.3.0 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.3.8)
	Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (from datasets) (2.2.2)
	Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.12/dist-packages (from datasets) (2.32.4)
	Requirement already satisfied: httpx<1.0.0 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.28.1)
	Requirement already satisfied: xxhash in /usr/local/lib/python3.12/dist-packages (from datasets) (3.8.1)
	Requirement already satisfied: multiprocess<0.70.20 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.70.16)
	Requirement already satisfied: fsspec<=2026.6.0,>=2023.1.0 in /usr/local/lib/python3.12/dist-packages (from fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (2025.3.0)
	Requirement already satisfied: psutil in /usr/local/lib/python3.12/dist-packages (from accelerate) (5.9.5)
	Requirement already satisfied: torch>=2.0.0 in /usr/local/lib/python3.12/dist-packages (from accelerate) (2.11.0+cu128)
	Requirement already satisfied: click<9.0.0,>=8.4.2 in /usr/local/lib/python3.12/dist-packages (from huggingface_hub) (8.4.2)
	Requirement already satisfied: typing-extensions>=4.1.0 in /usr/local/lib/python3.12/dist-packages (from huggingface_hub) (4.16.0)
	Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.12/dist-packages (from fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (3.14.1)
	Requirement already satisfied: anyio in /usr/local/lib/python3.12/dist-packages (from httpx<1.0.0->datasets) (4.14.2)
	Requirement already satisfied: certifi in /usr/local/lib/python3.12/dist-packages (from httpx<1.0.0->datasets) (2026.6.17)
	Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.12/dist-packages (from httpx<1.0.0->datasets) (1.0.9)
	Requirement already satisfied: idna in /usr/local/lib/python3.12/dist-packages (from httpx<1.0.0->datasets) (3.18)
	Requirement already satisfied: h11>=0.16 in /usr/local/lib/python3.12/dist-packages (from httpcore==1.*->httpx<1.0.0->datasets) (0.16.0)
	Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/dist-packages (from requests>=2.32.2->datasets) (3.4.9)
	Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/dist-packages (from requests>=2.32.2->datasets) (2.5.0)
	Requirement already satisfied: setuptools<82 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (75.2.0)
	Requirement already satisfied: sympy>=1.13.3 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (1.14.0)
	Requirement already satisfied: networkx>=2.5.1 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (3.6.1)
	Requirement already satisfied: jinja2 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (3.1.6)
	Requirement already satisfied: cuda-toolkit==12.8.1 in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (12.8.1)
	Requirement already satisfied: cuda-bindings<13,>=12.9.4 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (12.9.7)
	Requirement already satisfied: nvidia-cudnn-cu12==9.19.0.56 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (9.19.0.56)
	Requirement already satisfied: nvidia-cusparselt-cu12==0.7.1 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (0.7.1)
	Requirement already satisfied: nvidia-nccl-cu12==2.28.9 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (2.28.9)
	Requirement already satisfied: nvidia-nvshmem-cu12==3.4.5 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (3.4.5)
	Requirement already satisfied: triton==3.6.0 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (3.6.0)
	Requirement already satisfied: nvidia-cublas-cu12==12.8.4.1.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (12.8.4.1)
	Requirement already satisfied: nvidia-cuda-runtime-cu12==12.8.90.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (12.8.90)
	Requirement already satisfied: nvidia-cufft-cu12==11.3.3.83.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (11.3.3.83)
	Requirement already satisfied: nvidia-cufile-cu12==1.13.1.3.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (1.13.1.3)
	Requirement already satisfied: nvidia-cuda-cupti-cu12==12.8.90.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (12.8.90)
	Requirement already satisfied: nvidia-curand-cu12==10.3.9.90.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (10.3.9.90)
	Requirement already satisfied: nvidia-cusolver-cu12==11.7.3.90.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (11.7.3.90)
	Requirement already satisfied: nvidia-cusparse-cu12==12.5.8.93.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (12.5.8.93)
	Requirement already satisfied: nvidia-nvjitlink-cu12==12.8.93.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (12.8.93)
	Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.8.93.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (12.8.93)
	Requirement already satisfied: nvidia-nvtx-cu12==12.8.90.* in /usr/local/lib/python3.12/dist-packages (from cuda-toolkit[cublas,cudart,cufft,cufile,cupti,curand,cusolver,cusparse,nvjitlink,nvrtc,nvtx]==12.8.1; platform_system == "Linux"->torch>=2.0.0->accelerate) (12.8.90)
	Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2.9.0.post0)
	Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2025.2)
	Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2026.3)
	Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from typer->transformers) (1.5.4)
	Requirement already satisfied: rich>=13.8.0 in /usr/local/lib/python3.12/dist-packages (from typer->transformers) (13.9.4)
	Requirement already satisfied: annotated-doc>=0.0.2 in /usr/local/lib/python3.12/dist-packages (from typer->transformers) (0.0.4)
	Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (2.7.1)
	Requirement already satisfied: aiosignal>=1.4.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (1.4.0)
	Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (26.1.0)
	Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (1.8.0)
	Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (6.7.1)
	Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (0.5.2)
	Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2026.6.0,>=2023.1.0->datasets) (1.24.2)
	Requirement already satisfied: cuda-pathfinder~=1.1 in /usr/local/lib/python3.12/dist-packages (from cuda-bindings<13,>=12.9.4->torch>=2.0.0->accelerate) (1.5.6)
	Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.17.0)
	Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.12/dist-packages (from rich>=13.8.0->typer->transformers) (4.2.0)
	Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.12/dist-packages (from rich>=13.8.0->typer->transformers) (2.20.0)
	Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.12/dist-packages (from sympy>=1.13.3->torch>=2.0.0->accelerate) (1.3.0)
	Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.12/dist-packages (from jinja2->torch>=2.0.0->accelerate) (3.0.3)
	Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.12/dist-packages (from markdown-it-py>=2.2.0->rich>=13.8.0->typer->transformers) (0.1.2)
	Downloading transformers-5.15.0-py3-none-any.whl (11.7 MB)
	━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.7/11.7 MB 133.0 MB/s eta 0:00:00
	Downloading datasets-5.0.1-py3-none-any.whl (559 kB)
	━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 559.1/559.1 kB 48.0 MB/s eta 0:00:00
	Downloading huggingface_hub-1.27.0-py3-none-any.whl (784 kB)
	━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 784.9/784.9 kB 61.8 MB/s eta 0:00:00
	Downloading hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB)
	━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 132.9 MB/s eta 0:00:00
	Downloading pyarrow-25.0.1-cp312-cp312-manylinux_2_28_x86_64.whl (50.1 MB)
	━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.1/50.1 MB 49.6 MB/s eta 0:00:00
	Installing collected packages: pyarrow, hf_xet, huggingface_hub, datasets, transformers
	Attempting uninstall: pyarrow
	Found existing installation: pyarrow 18.1.0
	Uninstalling pyarrow-18.1.0:
	Successfully uninstalled pyarrow-18.1.0
	Attempting uninstall: hf_xet
	Found existing installation: hf-xet 1.5.1
	Uninstalling hf-xet-1.5.1:
	Successfully uninstalled hf-xet-1.5.1
	Attempting uninstall: huggingface_hub
	Found existing installation: huggingface_hub 1.23.0
	Uninstalling huggingface_hub-1.23.0:
	Successfully uninstalled huggingface_hub-1.23.0
	Attempting uninstall: datasets
	Found existing installation: datasets 4.0.0
	Uninstalling datasets-4.0.0:
	Successfully uninstalled datasets-4.0.0
	Attempting uninstall: transformers
	Found existing installation: transformers 5.13.1
	Uninstalling transformers-5.13.1:
	Successfully uninstalled transformers-5.13.1
	Successfully installed datasets-5.0.1 hf_xet-1.6.0 huggingface_hub-1.27.0 pyarrow-25.0.1 transformers-5.15.0
	To log in, open this URL and enter the code:https://hf.co/oauth/device1JHU-54EFWaiting for authorization...Login successful. Logged in as GGUFGuy (token: oauth-GGUFGuy).This token will be refreshed automatically when it expires.
	======================================================================
	AppleMind 1.0 Mini
	Colab L4 High-RAM
	300M TOKEN PRETRAINING
	======================================================================

	Device: cuda
	GPU: NVIDIA L4
	VRAM: 22.03 GB
	CUDA: 12.8

	Total training tokens: 300,000,000
	Target ratio: ~300:1

	======================================================================
	DATASET MIXTURE
	======================================================================

	FineWeb-Edu sample-10BT : 100,000,000
	FineWeb-HQ : 100,000,000
	SmolLM-Corpus : 100,000,000

	TOTAL : 300,000,000

	TinyStories: REMOVED
	WikiText: REMOVED
	Local datasets: REMOVED
	Dataset cycling: DISABLED

	======================================================================
	LOADING GPT-2 TOKENIZER
	======================================================================
	tokenizer_config.json: 100% 26.0/26.0 [00:00<00:00, 3.04kB/s]vocab.json: 100% 1.04M/1.04M [00:00<00:00, 15.7MB/s]merges.txt: 100% 456k/456k [00:00<00:00, 5.83MB/s]tokenizer.json: 100% 1.36M/1.36M [00:00<00:00, 28.9MB/s]
	Vocabulary: 50,260
	Special tokens added: 3


	======================================================================
	CREATING APPLEMIND
	======================================================================

	Parameters: 1,020,480
	Trainable: 1,020,480

	Target tokens: 300,000,000
	Target token/parameter ratio: 293.98:1


	======================================================================
	SETTING UP HUGGING FACE
	======================================================================

	Logged in as:
	GGUFGuy

	Checkpoint repository ready:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints

	Final model repository ready:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini
	Precision: BF16

	======================================================================
	TRAINING CONFIGURATION
	======================================================================

	Parameters: 1,020,480
	Training tokens: 300,000,000
	Tokens/parameter: 293.98:1

	Micro batch size: 64
	Gradient accumulation: 8
	Effective batch: 512
	Context length: 256
	Tokens/micro batch: 16,384
	Tokens/optimizer step: 131,072
	Optimizer steps: 2,289
	Warmup steps: 114

	FineWeb-Edu: 100M
	FineWeb-HQ: 100M
	SmolLM: 100M

	Checkpoint repository:
	AppleMind-AI/AppleMind-1.0-Mini-Checkpoints

	Final repository:
	AppleMind-AI/AppleMind-1.0-Mini

	======================================================================
	STARTING 300M-TOKEN PRETRAINING
	======================================================================

	Opening FineWeb-Edu sample-10BT...
	README.md: 100% 26.4k/26.4k [00:00<00:00, 2.88MB/s]Resolving data files: 100% 2410/2410 [00:00<00:00, 23346.46it/s]
	======================================================================
	STREAMING FineWeb-Edu
	======================================================================

	Target tokens: 100,000,000
	Progress state: 0

	[transformers] `loss_type=None` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`.
	FineWeb-Edu: 1,000,131 / 100,000,000 tokens
	FineWeb-Edu: 2,001,441 / 100,000,000 tokens
	FineWeb-Edu: 3,001,677 / 100,000,000 tokens
	FineWeb-Edu: 4,003,530 / 100,000,000 tokens
	FineWeb-Edu: 5,003,864 / 100,000,000 tokens
	FineWeb-Edu: 6,014,497 / 100,000,000 tokens
	FineWeb-Edu: 7,014,690 / 100,000,000 tokens
	FineWeb-Edu: 8,014,892 / 100,000,000 tokens
	FineWeb-Edu: 9,015,759 / 100,000,000 tokens
	FineWeb-Edu: 10,016,701 / 100,000,000 tokens
	FineWeb-Edu: 11,042,613 / 100,000,000 tokens
	FineWeb-Edu: 12,043,456 / 100,000,000 tokens
	FineWeb-Edu: 13,044,297 / 100,000,000 tokens

	Step 100/2,289 \| Loss 10.8201 \| LR 2.632e-05 \| 100,991 tok/s \| 13,107,200 tokens \| 4.37% \| 0.04h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-100...
	Writing model shards: 100% 1/1 [00:00<00:00, 19.95it/s]
	======================================================================
	UPLOADING CHECKPOINT 100
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-100
	FineWeb-Edu: 14,051,468 / 100,000,000 tokens
	FineWeb-Edu: 15,057,858 / 100,000,000 tokens
	FineWeb-Edu: 16,058,350 / 100,000,000 tokens
	FineWeb-Edu: 17,058,910 / 100,000,000 tokens
	FineWeb-Edu: 18,059,120 / 100,000,000 tokens
	FineWeb-Edu: 19,059,214 / 100,000,000 tokens
	FineWeb-Edu: 20,060,034 / 100,000,000 tokens
	FineWeb-Edu: 21,061,034 / 100,000,000 tokens
	FineWeb-Edu: 22,061,243 / 100,000,000 tokens
	FineWeb-Edu: 23,061,857 / 100,000,000 tokens
	FineWeb-Edu: 24,068,376 / 100,000,000 tokens
	FineWeb-Edu: 25,068,451 / 100,000,000 tokens
	FineWeb-Edu: 26,069,133 / 100,000,000 tokens

	Step 200/2,289 \| Loss 10.7533 \| LR 2.990e-05 \| 103,793 tok/s \| 26,214,400 tokens \| 8.74% \| 0.07h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-200...
	Writing model shards: 100% 1/1 [00:00<00:00, 11.00it/s]
	======================================================================
	UPLOADING CHECKPOINT 200
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-200
	FineWeb-Edu: 27,075,071 / 100,000,000 tokens
	FineWeb-Edu: 28,080,714 / 100,000,000 tokens
	FineWeb-Edu: 29,080,972 / 100,000,000 tokens
	FineWeb-Edu: 30,085,136 / 100,000,000 tokens
	FineWeb-Edu: 31,085,689 / 100,000,000 tokens
	FineWeb-Edu: 32,086,415 / 100,000,000 tokens
	FineWeb-Edu: 33,095,884 / 100,000,000 tokens
	FineWeb-Edu: 34,096,837 / 100,000,000 tokens
	FineWeb-Edu: 35,097,143 / 100,000,000 tokens
	FineWeb-Edu: 36,097,343 / 100,000,000 tokens
	FineWeb-Edu: 37,104,064 / 100,000,000 tokens
	FineWeb-Edu: 38,104,207 / 100,000,000 tokens
	FineWeb-Edu: 39,105,073 / 100,000,000 tokens

	Step 300/2,289 \| Loss 10.6609 \| LR 2.952e-05 \| 104,146 tok/s \| 39,321,600 tokens \| 13.11% \| 0.11h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-300...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.94it/s]
	======================================================================
	UPLOADING CHECKPOINT 300
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-300
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-100
	FineWeb-Edu: 40,107,907 / 100,000,000 tokens
	FineWeb-Edu: 41,108,319 / 100,000,000 tokens
	FineWeb-Edu: 42,108,687 / 100,000,000 tokens
	FineWeb-Edu: 43,108,744 / 100,000,000 tokens
	FineWeb-Edu: 44,109,584 / 100,000,000 tokens
	FineWeb-Edu: 45,109,641 / 100,000,000 tokens
	FineWeb-Edu: 46,110,333 / 100,000,000 tokens
	FineWeb-Edu: 47,110,722 / 100,000,000 tokens
	FineWeb-Edu: 48,111,125 / 100,000,000 tokens
	FineWeb-Edu: 49,111,530 / 100,000,000 tokens
	FineWeb-Edu: 50,113,218 / 100,000,000 tokens
	FineWeb-Edu: 51,113,865 / 100,000,000 tokens
	FineWeb-Edu: 52,114,128 / 100,000,000 tokens

	Step 400/2,289 \| Loss 10.5818 \| LR 2.887e-05 \| 104,108 tok/s \| 52,428,800 tokens \| 17.48% \| 0.14h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-400...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.91it/s]
	======================================================================
	UPLOADING CHECKPOINT 400
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-400
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-200
	FineWeb-Edu: 53,114,351 / 100,000,000 tokens
	FineWeb-Edu: 54,115,268 / 100,000,000 tokens
	FineWeb-Edu: 55,115,867 / 100,000,000 tokens
	FineWeb-Edu: 56,118,747 / 100,000,000 tokens
	FineWeb-Edu: 57,118,792 / 100,000,000 tokens
	FineWeb-Edu: 58,129,696 / 100,000,000 tokens
	FineWeb-Edu: 59,129,772 / 100,000,000 tokens
	FineWeb-Edu: 60,130,463 / 100,000,000 tokens
	FineWeb-Edu: 61,130,846 / 100,000,000 tokens
	FineWeb-Edu: 62,131,974 / 100,000,000 tokens
	FineWeb-Edu: 63,145,958 / 100,000,000 tokens
	FineWeb-Edu: 64,146,051 / 100,000,000 tokens
	FineWeb-Edu: 65,146,257 / 100,000,000 tokens

	Step 500/2,289 \| Loss 10.5081 \| LR 2.797e-05 \| 103,733 tok/s \| 65,536,000 tokens \| 21.85% \| 0.18h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-500...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.95it/s]
	======================================================================
	UPLOADING CHECKPOINT 500
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-500
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-300
	FineWeb-Edu: 66,147,176 / 100,000,000 tokens
	FineWeb-Edu: 67,153,903 / 100,000,000 tokens
	FineWeb-Edu: 68,163,652 / 100,000,000 tokens
	FineWeb-Edu: 69,164,783 / 100,000,000 tokens
	FineWeb-Edu: 70,166,355 / 100,000,000 tokens
	FineWeb-Edu: 71,166,545 / 100,000,000 tokens
	FineWeb-Edu: 72,167,733 / 100,000,000 tokens
	FineWeb-Edu: 73,168,881 / 100,000,000 tokens
	FineWeb-Edu: 74,175,780 / 100,000,000 tokens
	FineWeb-Edu: 75,176,435 / 100,000,000 tokens
	FineWeb-Edu: 76,177,413 / 100,000,000 tokens
	FineWeb-Edu: 77,177,431 / 100,000,000 tokens
	FineWeb-Edu: 78,179,729 / 100,000,000 tokens

	Step 600/2,289 \| Loss 10.4357 \| LR 2.682e-05 \| 104,154 tok/s \| 78,643,200 tokens \| 26.21% \| 0.21h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-600...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.96it/s]
	======================================================================
	UPLOADING CHECKPOINT 600
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-600
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-400
	FineWeb-Edu: 79,187,841 / 100,000,000 tokens
	FineWeb-Edu: 80,187,924 / 100,000,000 tokens
	FineWeb-Edu: 81,188,888 / 100,000,000 tokens
	FineWeb-Edu: 82,189,255 / 100,000,000 tokens
	FineWeb-Edu: 83,189,443 / 100,000,000 tokens
	FineWeb-Edu: 84,192,547 / 100,000,000 tokens
	FineWeb-Edu: 85,192,626 / 100,000,000 tokens
	FineWeb-Edu: 86,193,916 / 100,000,000 tokens
	FineWeb-Edu: 87,194,129 / 100,000,000 tokens
	FineWeb-Edu: 88,196,386 / 100,000,000 tokens
	FineWeb-Edu: 89,203,193 / 100,000,000 tokens
	FineWeb-Edu: 90,203,499 / 100,000,000 tokens
	FineWeb-Edu: 91,204,016 / 100,000,000 tokens

	Step 700/2,289 \| Loss 10.3676 \| LR 2.546e-05 \| 104,272 tok/s \| 91,750,400 tokens \| 30.58% \| 0.25h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-700...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.96it/s]
	======================================================================
	UPLOADING CHECKPOINT 700
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-700
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-500
	FineWeb-Edu: 92,205,445 / 100,000,000 tokens
	FineWeb-Edu: 93,206,406 / 100,000,000 tokens
	FineWeb-Edu: 94,215,721 / 100,000,000 tokens
	FineWeb-Edu: 95,216,609 / 100,000,000 tokens
	FineWeb-Edu: 96,218,844 / 100,000,000 tokens
	FineWeb-Edu: 97,219,420 / 100,000,000 tokens
	FineWeb-Edu: 98,219,961 / 100,000,000 tokens
	FineWeb-Edu: 99,220,378 / 100,000,000 tokens

	FineWeb-Edu complete: 100,000,000 tokens
	Opening FineWeb-HQ...
	README.md: 100% 4.65k/4.65k [00:00<00:00, 385kB/s]Resolving data files: 100% 9246/9246 [00:00<00:00, 25238.04it/s]
	======================================================================
	STREAMING FineWeb-HQ
	======================================================================

	Target tokens: 100,000,000
	Progress state: 0

	FineWeb-HQ: 1,000,237 / 100,000,000 tokens
	FineWeb-HQ: 2,000,353 / 100,000,000 tokens
	FineWeb-HQ: 3,000,648 / 100,000,000 tokens
	FineWeb-HQ: 4,001,578 / 100,000,000 tokens

	Step 800/2,289 \| Loss 10.3038 \| LR 2.391e-05 \| 98,189 tok/s \| 104,857,600 tokens \| 34.95% \| 0.28h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-800...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.89it/s]
	======================================================================
	UPLOADING CHECKPOINT 800
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-800
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-600
	FineWeb-HQ: 5,003,165 / 100,000,000 tokens
	FineWeb-HQ: 6,003,192 / 100,000,000 tokens
	FineWeb-HQ: 7,003,245 / 100,000,000 tokens
	FineWeb-HQ: 8,005,095 / 100,000,000 tokens
	FineWeb-HQ: 9,005,500 / 100,000,000 tokens
	FineWeb-HQ: 10,005,995 / 100,000,000 tokens
	FineWeb-HQ: 11,006,275 / 100,000,000 tokens
	FineWeb-HQ: 12,006,703 / 100,000,000 tokens
	FineWeb-HQ: 13,008,348 / 100,000,000 tokens
	FineWeb-HQ: 14,039,910 / 100,000,000 tokens
	FineWeb-HQ: 15,040,042 / 100,000,000 tokens
	FineWeb-HQ: 16,040,179 / 100,000,000 tokens
	FineWeb-HQ: 17,040,530 / 100,000,000 tokens

	Step 900/2,289 \| Loss 10.2507 \| LR 2.221e-05 \| 103,602 tok/s \| 117,964,800 tokens \| 39.32% \| 0.32h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-900...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.94it/s]
	======================================================================
	UPLOADING CHECKPOINT 900
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-900
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-700
	FineWeb-HQ: 18,045,133 / 100,000,000 tokens
	FineWeb-HQ: 19,053,155 / 100,000,000 tokens
	FineWeb-HQ: 20,053,155 / 100,000,000 tokens
	FineWeb-HQ: 21,053,631 / 100,000,000 tokens
	FineWeb-HQ: 22,061,664 / 100,000,000 tokens
	FineWeb-HQ: 23,061,940 / 100,000,000 tokens
	FineWeb-HQ: 24,062,978 / 100,000,000 tokens
	FineWeb-HQ: 25,063,229 / 100,000,000 tokens
	FineWeb-HQ: 26,073,081 / 100,000,000 tokens
	FineWeb-HQ: 27,073,220 / 100,000,000 tokens
	FineWeb-HQ: 28,073,681 / 100,000,000 tokens
	FineWeb-HQ: 29,073,966 / 100,000,000 tokens
	FineWeb-HQ: 30,074,272 / 100,000,000 tokens

	Step 1,000/2,289 \| Loss 10.1952 \| LR 2.039e-05 \| 103,234 tok/s \| 131,072,000 tokens \| 43.69% \| 0.35h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1000...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.99it/s]
	======================================================================
	UPLOADING CHECKPOINT 1000
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1000
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-800
	FineWeb-HQ: 31,075,196 / 100,000,000 tokens
	FineWeb-HQ: 32,075,679 / 100,000,000 tokens
	FineWeb-HQ: 33,077,041 / 100,000,000 tokens
	FineWeb-HQ: 34,077,283 / 100,000,000 tokens
	FineWeb-HQ: 35,077,442 / 100,000,000 tokens
	FineWeb-HQ: 36,078,144 / 100,000,000 tokens
	FineWeb-HQ: 37,078,751 / 100,000,000 tokens
	FineWeb-HQ: 38,080,143 / 100,000,000 tokens
	FineWeb-HQ: 39,082,841 / 100,000,000 tokens
	FineWeb-HQ: 40,083,517 / 100,000,000 tokens
	FineWeb-HQ: 41,084,347 / 100,000,000 tokens
	FineWeb-HQ: 42,087,132 / 100,000,000 tokens
	FineWeb-HQ: 43,087,302 / 100,000,000 tokens
	FineWeb-HQ: 44,087,920 / 100,000,000 tokens

	Step 1,100/2,289 \| Loss 10.1467 \| LR 1.849e-05 \| 103,958 tok/s \| 144,179,200 tokens \| 48.06% \| 0.39h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1100...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.97it/s]
	======================================================================
	UPLOADING CHECKPOINT 1100
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1100
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-900
	FineWeb-HQ: 45,091,588 / 100,000,000 tokens
	FineWeb-HQ: 46,091,666 / 100,000,000 tokens
	FineWeb-HQ: 47,091,765 / 100,000,000 tokens
	FineWeb-HQ: 48,093,667 / 100,000,000 tokens
	FineWeb-HQ: 49,095,400 / 100,000,000 tokens
	FineWeb-HQ: 50,100,158 / 100,000,000 tokens
	FineWeb-HQ: 51,103,033 / 100,000,000 tokens
	FineWeb-HQ: 52,103,833 / 100,000,000 tokens
	FineWeb-HQ: 53,103,842 / 100,000,000 tokens
	FineWeb-HQ: 54,105,589 / 100,000,000 tokens
	FineWeb-HQ: 55,106,976 / 100,000,000 tokens
	FineWeb-HQ: 56,108,077 / 100,000,000 tokens
	FineWeb-HQ: 57,108,658 / 100,000,000 tokens

	Step 1,200/2,289 \| Loss 10.0994 \| LR 1.655e-05 \| 103,466 tok/s \| 157,286,400 tokens \| 52.43% \| 0.42h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1200...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.94it/s]
	======================================================================
	UPLOADING CHECKPOINT 1200
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1200
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1000
	FineWeb-HQ: 58,108,705 / 100,000,000 tokens
	FineWeb-HQ: 59,109,817 / 100,000,000 tokens
	FineWeb-HQ: 60,115,751 / 100,000,000 tokens
	FineWeb-HQ: 61,116,056 / 100,000,000 tokens
	FineWeb-HQ: 62,116,477 / 100,000,000 tokens
	FineWeb-HQ: 63,116,951 / 100,000,000 tokens
	FineWeb-HQ: 64,118,012 / 100,000,000 tokens
	FineWeb-HQ: 65,122,172 / 100,000,000 tokens
	FineWeb-HQ: 66,122,389 / 100,000,000 tokens
	FineWeb-HQ: 67,124,930 / 100,000,000 tokens
	FineWeb-HQ: 68,124,933 / 100,000,000 tokens
	FineWeb-HQ: 69,126,235 / 100,000,000 tokens
	FineWeb-HQ: 70,127,674 / 100,000,000 tokens

	Step 1,300/2,289 \| Loss 10.0603 \| LR 1.461e-05 \| 103,354 tok/s \| 170,393,600 tokens \| 56.80% \| 0.46h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1300...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.95it/s]
	======================================================================
	UPLOADING CHECKPOINT 1300
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1300
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1100
	FineWeb-HQ: 71,128,320 / 100,000,000 tokens
	FineWeb-HQ: 72,130,508 / 100,000,000 tokens
	FineWeb-HQ: 73,131,425 / 100,000,000 tokens
	FineWeb-HQ: 74,145,259 / 100,000,000 tokens
	FineWeb-HQ: 75,168,512 / 100,000,000 tokens
	FineWeb-HQ: 76,169,236 / 100,000,000 tokens
	FineWeb-HQ: 77,170,751 / 100,000,000 tokens
	FineWeb-HQ: 78,171,712 / 100,000,000 tokens
	FineWeb-HQ: 79,171,903 / 100,000,000 tokens
	FineWeb-HQ: 80,172,653 / 100,000,000 tokens
	FineWeb-HQ: 81,180,767 / 100,000,000 tokens
	FineWeb-HQ: 82,209,091 / 100,000,000 tokens
	FineWeb-HQ: 83,210,028 / 100,000,000 tokens

	Step 1,400/2,289 \| Loss 10.0248 \| LR 1.270e-05 \| 103,746 tok/s \| 183,500,800 tokens \| 61.17% \| 0.49h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1400...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.83it/s]
	======================================================================
	UPLOADING CHECKPOINT 1400
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1400
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1200
	FineWeb-HQ: 84,221,481 / 100,000,000 tokens
	FineWeb-HQ: 85,221,902 / 100,000,000 tokens
	FineWeb-HQ: 86,221,927 / 100,000,000 tokens
	FineWeb-HQ: 87,224,781 / 100,000,000 tokens
	FineWeb-HQ: 88,226,957 / 100,000,000 tokens
	FineWeb-HQ: 89,227,905 / 100,000,000 tokens
	FineWeb-HQ: 90,230,257 / 100,000,000 tokens
	FineWeb-HQ: 91,234,535 / 100,000,000 tokens
	FineWeb-HQ: 92,235,135 / 100,000,000 tokens
	FineWeb-HQ: 93,235,228 / 100,000,000 tokens
	FineWeb-HQ: 94,235,284 / 100,000,000 tokens
	FineWeb-HQ: 95,242,150 / 100,000,000 tokens
	FineWeb-HQ: 96,254,650 / 100,000,000 tokens

	Step 1,500/2,289 \| Loss 9.9967 \| LR 1.088e-05 \| 103,650 tok/s \| 196,608,000 tokens \| 65.54% \| 0.53h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1500...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.86it/s]
	======================================================================
	UPLOADING CHECKPOINT 1500
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1500
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1300
	FineWeb-HQ: 97,258,984 / 100,000,000 tokens
	FineWeb-HQ: 98,259,041 / 100,000,000 tokens
	FineWeb-HQ: 99,263,648 / 100,000,000 tokens

	FineWeb-HQ complete: 100,000,000 tokens
	Opening SmolLM-Corpus cosmopedia-v2...
	README.md: 100% 7.05k/7.05k [00:00<00:00, 847kB/s]Resolving data files: 100% 104/104 [00:00<00:00, 6458.13it/s]Resolving data files: 100% 104/104 [00:00<00:00, 7784.83it/s]
	======================================================================
	STREAMING SmolLM-Corpus
	======================================================================

	Target tokens: 100,000,000
	Progress state: 0

	SmolLM-Corpus: 1,000,389 / 100,000,000 tokens
	SmolLM-Corpus: 2,000,612 / 100,000,000 tokens
	SmolLM-Corpus: 3,000,937 / 100,000,000 tokens
	SmolLM-Corpus: 4,001,355 / 100,000,000 tokens
	SmolLM-Corpus: 5,001,629 / 100,000,000 tokens
	SmolLM-Corpus: 6,002,147 / 100,000,000 tokens
	SmolLM-Corpus: 7,002,160 / 100,000,000 tokens
	SmolLM-Corpus: 8,002,233 / 100,000,000 tokens
	SmolLM-Corpus: 9,002,382 / 100,000,000 tokens

	Step 1,600/2,289 \| Loss 10.0072 \| LR 9.168e-06 \| 101,065 tok/s \| 209,715,200 tokens \| 69.91% \| 0.57h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1600...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.94it/s]
	======================================================================
	UPLOADING CHECKPOINT 1600
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1600
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1400
	SmolLM-Corpus: 10,002,523 / 100,000,000 tokens
	SmolLM-Corpus: 11,002,661 / 100,000,000 tokens
	SmolLM-Corpus: 12,003,098 / 100,000,000 tokens
	SmolLM-Corpus: 13,003,147 / 100,000,000 tokens
	SmolLM-Corpus: 14,004,159 / 100,000,000 tokens
	SmolLM-Corpus: 15,004,183 / 100,000,000 tokens
	SmolLM-Corpus: 16,004,330 / 100,000,000 tokens
	SmolLM-Corpus: 17,005,124 / 100,000,000 tokens
	SmolLM-Corpus: 18,005,586 / 100,000,000 tokens
	SmolLM-Corpus: 19,006,013 / 100,000,000 tokens
	SmolLM-Corpus: 20,006,056 / 100,000,000 tokens
	SmolLM-Corpus: 21,006,417 / 100,000,000 tokens
	SmolLM-Corpus: 22,006,642 / 100,000,000 tokens

	Step 1,700/2,289 \| Loss 9.9950 \| LR 7.613e-06 \| 104,388 tok/s \| 222,822,400 tokens \| 74.27% \| 0.60h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1700...
	Writing model shards: 100% 1/1 [00:00<00:00, 11.01it/s]
	======================================================================
	UPLOADING CHECKPOINT 1700
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1700
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1500
	SmolLM-Corpus: 23,007,410 / 100,000,000 tokens
	SmolLM-Corpus: 24,007,513 / 100,000,000 tokens
	SmolLM-Corpus: 25,008,024 / 100,000,000 tokens
	SmolLM-Corpus: 26,008,152 / 100,000,000 tokens
	SmolLM-Corpus: 27,008,820 / 100,000,000 tokens
	SmolLM-Corpus: 28,009,655 / 100,000,000 tokens
	SmolLM-Corpus: 29,009,777 / 100,000,000 tokens
	SmolLM-Corpus: 30,010,160 / 100,000,000 tokens
	SmolLM-Corpus: 31,010,762 / 100,000,000 tokens
	SmolLM-Corpus: 32,010,887 / 100,000,000 tokens
	SmolLM-Corpus: 33,011,339 / 100,000,000 tokens
	SmolLM-Corpus: 34,011,816 / 100,000,000 tokens
	SmolLM-Corpus: 35,012,785 / 100,000,000 tokens

	Step 1,800/2,289 \| Loss 9.9749 \| LR 6.242e-06 \| 104,425 tok/s \| 235,929,600 tokens \| 78.64% \| 0.63h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1800...
	Writing model shards: 100% 1/1 [00:00<00:00, 11.00it/s]
	======================================================================
	UPLOADING CHECKPOINT 1800
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1800
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1600
	SmolLM-Corpus: 36,013,511 / 100,000,000 tokens
	SmolLM-Corpus: 37,013,686 / 100,000,000 tokens
	SmolLM-Corpus: 38,015,046 / 100,000,000 tokens
	SmolLM-Corpus: 39,015,520 / 100,000,000 tokens
	SmolLM-Corpus: 40,016,051 / 100,000,000 tokens
	SmolLM-Corpus: 41,016,215 / 100,000,000 tokens
	SmolLM-Corpus: 42,016,528 / 100,000,000 tokens
	SmolLM-Corpus: 43,016,537 / 100,000,000 tokens
	SmolLM-Corpus: 44,016,544 / 100,000,000 tokens
	SmolLM-Corpus: 45,017,221 / 100,000,000 tokens
	SmolLM-Corpus: 46,017,703 / 100,000,000 tokens
	SmolLM-Corpus: 47,017,830 / 100,000,000 tokens
	SmolLM-Corpus: 48,018,530 / 100,000,000 tokens
	SmolLM-Corpus: 49,019,055 / 100,000,000 tokens

	Step 1,900/2,289 \| Loss 9.9581 \| LR 5.086e-06 \| 104,428 tok/s \| 249,036,800 tokens \| 83.01% \| 0.67h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-1900...
	Writing model shards: 100% 1/1 [00:00<00:00, 11.07it/s]
	======================================================================
	UPLOADING CHECKPOINT 1900
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-1900
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1700
	SmolLM-Corpus: 50,019,088 / 100,000,000 tokens
	SmolLM-Corpus: 51,019,166 / 100,000,000 tokens
	SmolLM-Corpus: 52,019,182 / 100,000,000 tokens
	SmolLM-Corpus: 53,019,341 / 100,000,000 tokens
	SmolLM-Corpus: 54,019,557 / 100,000,000 tokens
	SmolLM-Corpus: 55,019,622 / 100,000,000 tokens
	SmolLM-Corpus: 56,020,576 / 100,000,000 tokens
	SmolLM-Corpus: 57,021,061 / 100,000,000 tokens
	SmolLM-Corpus: 58,021,198 / 100,000,000 tokens
	SmolLM-Corpus: 59,022,144 / 100,000,000 tokens
	SmolLM-Corpus: 60,022,635 / 100,000,000 tokens
	SmolLM-Corpus: 61,022,949 / 100,000,000 tokens
	SmolLM-Corpus: 62,023,318 / 100,000,000 tokens

	Step 2,000/2,289 \| Loss 9.9444 \| LR 4.167e-06 \| 104,400 tok/s \| 262,144,000 tokens \| 87.38% \| 0.70h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-2000...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.99it/s]
	======================================================================
	UPLOADING CHECKPOINT 2000
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-2000
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1800
	SmolLM-Corpus: 63,024,246 / 100,000,000 tokens
	SmolLM-Corpus: 64,024,602 / 100,000,000 tokens
	SmolLM-Corpus: 65,025,944 / 100,000,000 tokens
	SmolLM-Corpus: 66,027,471 / 100,000,000 tokens
	SmolLM-Corpus: 67,027,613 / 100,000,000 tokens
	SmolLM-Corpus: 68,028,290 / 100,000,000 tokens
	SmolLM-Corpus: 69,028,776 / 100,000,000 tokens
	SmolLM-Corpus: 70,028,797 / 100,000,000 tokens
	SmolLM-Corpus: 71,030,241 / 100,000,000 tokens
	SmolLM-Corpus: 72,030,477 / 100,000,000 tokens
	SmolLM-Corpus: 73,030,534 / 100,000,000 tokens
	SmolLM-Corpus: 74,030,705 / 100,000,000 tokens
	SmolLM-Corpus: 75,030,996 / 100,000,000 tokens

	Step 2,100/2,289 \| Loss 9.9332 \| LR 3.505e-06 \| 104,456 tok/s \| 275,251,200 tokens \| 91.75% \| 0.74h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-2100...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.96it/s]
	======================================================================
	UPLOADING CHECKPOINT 2100
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-2100
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-1900
	SmolLM-Corpus: 76,031,414 / 100,000,000 tokens
	SmolLM-Corpus: 77,031,466 / 100,000,000 tokens
	SmolLM-Corpus: 78,032,243 / 100,000,000 tokens
	SmolLM-Corpus: 79,032,688 / 100,000,000 tokens
	SmolLM-Corpus: 80,032,933 / 100,000,000 tokens
	SmolLM-Corpus: 81,033,099 / 100,000,000 tokens
	SmolLM-Corpus: 82,034,041 / 100,000,000 tokens
	SmolLM-Corpus: 83,034,272 / 100,000,000 tokens
	SmolLM-Corpus: 84,035,283 / 100,000,000 tokens
	SmolLM-Corpus: 85,035,460 / 100,000,000 tokens
	SmolLM-Corpus: 86,035,589 / 100,000,000 tokens
	SmolLM-Corpus: 87,036,259 / 100,000,000 tokens
	SmolLM-Corpus: 88,036,685 / 100,000,000 tokens

	Step 2,200/2,289 \| Loss 9.9239 \| LR 3.114e-06 \| 104,304 tok/s \| 288,358,400 tokens \| 96.12% \| 0.77h
	VRAM: 1.57 GB allocated / 13.91 GB reserved


	Saving checkpoint-2200...
	Writing model shards: 100% 1/1 [00:00<00:00, 10.98it/s]
	======================================================================
	UPLOADING CHECKPOINT 2200
	======================================================================
	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	Checkpoint uploaded to:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints/tree/main/checkpoint-2200
	Removing old local checkpoint: /content/applemind-1.0-mini-checkpoints/checkpoint-2000
	SmolLM-Corpus: 89,036,736 / 100,000,000 tokens
	SmolLM-Corpus: 90,037,952 / 100,000,000 tokens
	SmolLM-Corpus: 91,038,162 / 100,000,000 tokens
	SmolLM-Corpus: 92,038,464 / 100,000,000 tokens
	SmolLM-Corpus: 93,038,712 / 100,000,000 tokens
	SmolLM-Corpus: 94,038,967 / 100,000,000 tokens
	SmolLM-Corpus: 95,039,446 / 100,000,000 tokens
	SmolLM-Corpus: 96,039,542 / 100,000,000 tokens
	SmolLM-Corpus: 97,040,355 / 100,000,000 tokens
	SmolLM-Corpus: 98,041,440 / 100,000,000 tokens
	SmolLM-Corpus: 99,041,658 / 100,000,000 tokens

	SmolLM-Corpus complete: 100,000,000 tokens

	======================================================================
	DATASET STREAM EXHAUSTED
	======================================================================


	Final dataset batch was incomplete.
	Stopping without an incomplete optimizer step.

	======================================================================
	TRAINING LOOP FINISHED
	======================================================================

	Completed optimizer steps: 2,288
	Configured optimizer steps: 2,289

	Dataset progress:
	FineWeb-Edu: 100,000,000 / 100,000,000
	FineWeb-HQ: 100,000,000 / 100,000,000
	SmolLM: 100,000,000 / 100,000,000

	======================================================================
	PRETRAINING FINISHED
	======================================================================

	Final steps: 2,288
	Final tokens: 299,892,736
	Target tokens: 300,000,000

	Actual tokens/parameter: 293.87:1

	======================================================================
	SAVING FINAL MODEL
	======================================================================
	Writing model shards: 100% 1/1 [00:00<00:00, 21.09it/s]
	======================================================================
	UPLOADING COMPLETE MODEL TO HUGGING FACE
	======================================================================

	No files have been modified since last commit. Skipping to prevent empty commit.
	WARNING:huggingface_hub._upload_pipeline:No files have been modified since last commit. Skipping to prevent empty commit.

	FINAL MODEL UPLOAD SUCCESSFUL!

	Final model:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini

	======================================================================
	CHAT TEMPLATE TEST
	======================================================================

	<\|system\|>
	You are AppleMind, a helpful AI assistant.
	<\|user\|>
	Hello!
	<\|assistant\|>


	======================================================================
	GENERATION TEST
	======================================================================

	Prompt: Once upon a time
	------------------------------------------------------------
	Once upon a time It the several times- nowThis, does to form provide from find another times work not atl The couldWhen on all be way H It lives among times always, worked
	its G during work used after several There and at there
	b known came be very that It thought It betweenIn course does. case other It 5?: or often I at's the: enough could in many
	------------------------------------------------------------

	Prompt: The little boy
	------------------------------------------------------------
	The little boy's form among. I and H be from� used. still all- G, lives take same always often its find amonged provide- number because: another now? then there among use course not thought case work usel result It between 5 It well atWhen new.: thatThis work on think 3 interestB then It does could do the among Ire use among now does to and many
	------------------------------------------------------------

	Prompt: In the forest
	------------------------------------------------------------
	In the forest often
	form and times on during severall find, several The then number between: well work take there provide, times among anotherWhen same Ged but lives could the to times its- course same enough and same I used
	to same other not thought There H think� same 2 I body nowe cameb 5- do among's and several? same after- still,This- interest but
	------------------------------------------------------------

	======================================================================
	APPLE MIND 1.0 MINI COMPLETE
	======================================================================

	Parameters: 1,020,480
	Tokens: 299,892,736
	Tokens/parameter: 293.87:1

	Dataset mixture:
	FineWeb-Edu: 100M
	FineWeb-HQ: 100M
	SmolLM: 100M

	Checkpoint repository:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini-Checkpoints

	Final model repository:
	https://huggingface.co/AppleMind-AI/AppleMind-1.0-Mini

	AppleMind 1.0 Mini complete!