Instructions to use anonym5035/ablation_5_no_experts with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use anonym5035/ablation_5_no_experts with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="anonym5035/ablation_5_no_experts", trust_remote_code=True)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("anonym5035/ablation_5_no_experts", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use anonym5035/ablation_5_no_experts with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "anonym5035/ablation_5_no_experts"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anonym5035/ablation_5_no_experts",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/anonym5035/ablation_5_no_experts

SGLang

How to use anonym5035/ablation_5_no_experts with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "anonym5035/ablation_5_no_experts" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anonym5035/ablation_5_no_experts",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "anonym5035/ablation_5_no_experts" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anonym5035/ablation_5_no_experts",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use anonym5035/ablation_5_no_experts with Docker Model Runner:
```
docker model run hf.co/anonym5035/ablation_5_no_experts
```

ablation_5_no_experts / checkpoint_perplexities.json

Soham Jain

Update strict-small architecture files (ablation: no experts) (part 12)

3f353e0 verified 12 days ago

Raw

History Blame Contribute Delete

3.39 kB

	[
	{
	"checkpoint": "chck_1M",
	"words_trained": 1000000,
	"global_step": 123,
	"val_loss": 8.828307991027833,
	"val_perplexity": 6824.729537261262
	},
	{
	"checkpoint": "chck_2M",
	"words_trained": 2000000,
	"global_step": 245,
	"val_loss": 7.891160659790039,
	"val_perplexity": 2673.545196972284
	},
	{
	"checkpoint": "chck_3M",
	"words_trained": 3000000,
	"global_step": 367,
	"val_loss": 7.473162097930908,
	"val_perplexity": 1760.1637050347072
	},
	{
	"checkpoint": "chck_4M",
	"words_trained": 4000000,
	"global_step": 489,
	"val_loss": 7.247150421142578,
	"val_perplexity": 1404.098053949613
	},
	{
	"checkpoint": "chck_5M",
	"words_trained": 5000000,
	"global_step": 611,
	"val_loss": 7.1073360347747805,
	"val_perplexity": 1220.8907994543215
	},
	{
	"checkpoint": "chck_6M",
	"words_trained": 6000000,
	"global_step": 733,
	"val_loss": 6.992768697738647,
	"val_perplexity": 1088.7316760256444
	},
	{
	"checkpoint": "chck_7M",
	"words_trained": 7000000,
	"global_step": 855,
	"val_loss": 6.954772806167602,
	"val_perplexity": 1048.1403798694926
	},
	{
	"checkpoint": "chck_8M",
	"words_trained": 8000000,
	"global_step": 977,
	"val_loss": 6.980320081710816,
	"val_perplexity": 1075.2624837754731
	},
	{
	"checkpoint": "chck_9M",
	"words_trained": 9000000,
	"global_step": 1099,
	"val_loss": 6.91542799949646,
	"val_perplexity": 1007.7022312619913
	},
	{
	"checkpoint": "chck_10M",
	"words_trained": 10000000,
	"global_step": 1221,
	"val_loss": 6.75490553855896,
	"val_perplexity": 858.2586737081385
	},
	{
	"checkpoint": "chck_20M",
	"words_trained": 20000000,
	"global_step": 2442,
	"val_loss": 6.233925104141235,
	"val_perplexity": 509.7523931221479
	},
	{
	"checkpoint": "chck_30M",
	"words_trained": 30000000,
	"global_step": 3663,
	"val_loss": 5.89044976234436,
	"val_perplexity": 361.56786741937503
	},
	{
	"checkpoint": "chck_40M",
	"words_trained": 40000000,
	"global_step": 4883,
	"val_loss": 5.774474506378174,
	"val_perplexity": 321.9751943439883
	},
	{
	"checkpoint": "chck_50M",
	"words_trained": 50000000,
	"global_step": 6104,
	"val_loss": 5.673352279663086,
	"val_perplexity": 291.0084427575258
	},
	{
	"checkpoint": "chck_60M",
	"words_trained": 60000000,
	"global_step": 7325,
	"val_loss": 5.440233287811279,
	"val_perplexity": 230.49594908443646
	},
	{
	"checkpoint": "chck_70M",
	"words_trained": 70000000,
	"global_step": 8545,
	"val_loss": 5.381023740768432,
	"val_perplexity": 217.24456373934882
	},
	{
	"checkpoint": "chck_80M",
	"words_trained": 80000000,
	"global_step": 9766,
	"val_loss": 5.3431094455718995,
	"val_perplexity": 209.1620782767871
	},
	{
	"checkpoint": "chck_90M",
	"words_trained": 90000000,
	"global_step": 10987,
	"val_loss": 5.234307689666748,
	"val_perplexity": 187.5991845223238
	},
	{
	"checkpoint": "chck_100M",
	"words_trained": 100000000,
	"global_step": 12208,
	"val_loss": 5.198590564727783,
	"val_perplexity": 181.01693034837896
	},
	{
	"checkpoint": "main",
	"words_trained": 126812160,
	"global_step": 15480,
	"val_loss": 5.1443859958648686,
	"val_perplexity": 171.4661714047026
	}
	]