Text Generation
Transformers
Safetensors
modilify_mk2
feature-extraction
diffusion
mixture-of-experts
custom-code
modilify-mk2
conversational
custom_code
Instructions to use modilify/Modilify-Mk2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use modilify/Modilify-Mk2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="modilify/Modilify-Mk2", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("modilify/Modilify-Mk2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use modilify/Modilify-Mk2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "modilify/Modilify-Mk2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/modilify/Modilify-Mk2
- SGLang
How to use modilify/Modilify-Mk2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "modilify/Modilify-Mk2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "modilify/Modilify-Mk2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use modilify/Modilify-Mk2 with Docker Model Runner:
docker model run hf.co/modilify/Modilify-Mk2
Download export_manifest.json from modilify/Modilify-Mk2: direct link, hf CLI and curl.
- Browser
- Download file 3.56 kB
-
https://huggingface.co/modilify/Modilify-Mk2/resolve/main/export_manifest.json
- Command line
-
hf download hf://modilify/Modilify-Mk2/export_manifest.json
-
curl -L -o export_manifest.json https://huggingface.co/modilify/Modilify-Mk2/resolve/main/export_manifest.json
3.56 kB
| { | |
| "format": "modilify_mk2_transformers_inference_v1", | |
| "global_step": 4400, | |
| "state_schema_version": 25, | |
| "training_scheme": "gold_prefix_shared_commit_0_256_committed_ce_calibration_causal_throughput_terminal_sft", | |
| "memory_scheme": "dual_timescale_gdn2_trajectory_memory", | |
| "memory_architecture": "compact_gdn2_v2", | |
| "base_model": "google/diffusiongemma-26B-A4B-it", | |
| "source_manifest_sha256": "e2b64fe2102ca1e002a62e279798abb3be271e3d9640a5afc5823cf233724dbc", | |
| "precision_policy": "gdn2_small_fp32_v1", | |
| "adapters": "unmerged", | |
| "trainable_tensor_count": 632, | |
| "total_tensor_count": 1323, | |
| "shards": [ | |
| { | |
| "file": "model-00001-of-00016.safetensors", | |
| "bytes": 3898256004, | |
| "sha256": "d3aa3fc0393a51acec27161569d566774030e0306a74748dd2a7a11fd8b535ec" | |
| }, | |
| { | |
| "file": "model-00002-of-00016.safetensors", | |
| "bytes": 3798073636, | |
| "sha256": "89339fd6f0521977fff9d4ae498e485ff409dbb8716cc924c8667c8bceaa19cd" | |
| }, | |
| { | |
| "file": "model-00003-of-00016.safetensors", | |
| "bytes": 3290562732, | |
| "sha256": "b9351e59c7c604b26712960b5d9f3460f5c1aace9ccb348999a2334d93fb613d" | |
| }, | |
| { | |
| "file": "model-00004-of-00016.safetensors", | |
| "bytes": 3319472948, | |
| "sha256": "1c89ba68a1f56e986dbf43175a7a7a6f2d046b04ac993c2837024f71b5d5923d" | |
| }, | |
| { | |
| "file": "model-00005-of-00016.safetensors", | |
| "bytes": 3290562732, | |
| "sha256": "4e42f45a5c409649c0ac8a262b26a0f644d10619d5f857568db855d12a865183" | |
| }, | |
| { | |
| "file": "model-00006-of-00016.safetensors", | |
| "bytes": 3290562772, | |
| "sha256": "da6e7d99f9801cb1b7007044f3baa66e767412d4350211a637174321eacee401" | |
| }, | |
| { | |
| "file": "model-00007-of-00016.safetensors", | |
| "bytes": 3319473028, | |
| "sha256": "ce1f462e324e162f9c0f41b1db3945290eb06c520da3d8ff0f582cde79a6fa96" | |
| }, | |
| { | |
| "file": "model-00008-of-00016.safetensors", | |
| "bytes": 3290562812, | |
| "sha256": "a163f6df13b9a3fbe1df187e42700b74d71fa08b331c521be568f71dec77a8f0" | |
| }, | |
| { | |
| "file": "model-00009-of-00016.safetensors", | |
| "bytes": 3290562812, | |
| "sha256": "eacc2e9ac62fa45f4fd1d391bfff490e327c19df3f70e7835e556287fc9926bd" | |
| }, | |
| { | |
| "file": "model-00010-of-00016.safetensors", | |
| "bytes": 3319473028, | |
| "sha256": "5fa9c8080a9c3954985226496dee261938f3d05cf81e4436932de593e5a01ae1" | |
| }, | |
| { | |
| "file": "model-00011-of-00016.safetensors", | |
| "bytes": 3290562812, | |
| "sha256": "7df1df1cea73133a94aec88056e2514e04a3467d42c0dd55ad0b922af2e5e986" | |
| }, | |
| { | |
| "file": "model-00012-of-00016.safetensors", | |
| "bytes": 3290562812, | |
| "sha256": "797078b4703d602cc3a323dba676fa38bb959b70d455bdc8c673663261eee2d4" | |
| }, | |
| { | |
| "file": "model-00013-of-00016.safetensors", | |
| "bytes": 3319473028, | |
| "sha256": "35923e8f76bc78ae89b3f83d1cdd175649f64cc898f1862baf543d7cdfbcd0e1" | |
| }, | |
| { | |
| "file": "model-00014-of-00016.safetensors", | |
| "bytes": 3290562812, | |
| "sha256": "105ceea4ed17c811fb79036c199db1f7d2392a5cc4c05003085f7217e8a938bb" | |
| }, | |
| { | |
| "file": "model-00015-of-00016.safetensors", | |
| "bytes": 3290562812, | |
| "sha256": "d3569ec55bc4f7e89de00efaedfbca3df44096526291731ec14bd7fc245c3b39" | |
| }, | |
| { | |
| "file": "model-00016-of-00016.safetensors", | |
| "bytes": 1202380654, | |
| "sha256": "23c9c00fc04b475c744458f498178a6e9a20fd673c36af981dd22e963b7ca49b" | |
| } | |
| ], | |
| "validated_environment": { | |
| "python": "3.12", | |
| "torch": "2.14.0", | |
| "transformers": "5.14.1", | |
| "safetensors": "0.8.0", | |
| "accelerate": "1.14.0" | |
| } | |
| } | |