Text Generation
Transformers
Safetensors
modilify_mk2
feature-extraction
diffusion
mixture-of-experts
custom-code
modilify-mk2
conversational
custom_code
Instructions to use modilify/Modilify-Mk2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use modilify/Modilify-Mk2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="modilify/Modilify-Mk2", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("modilify/Modilify-Mk2", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use modilify/Modilify-Mk2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "modilify/Modilify-Mk2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/modilify/Modilify-Mk2
- SGLang
How to use modilify/Modilify-Mk2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "modilify/Modilify-Mk2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "modilify/Modilify-Mk2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use modilify/Modilify-Mk2 with Docker Model Runner:
docker model run hf.co/modilify/Modilify-Mk2
File size: 3,557 Bytes
4bbff32 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | {
"format": "modilify_mk2_transformers_inference_v1",
"global_step": 4400,
"state_schema_version": 25,
"training_scheme": "gold_prefix_shared_commit_0_256_committed_ce_calibration_causal_throughput_terminal_sft",
"memory_scheme": "dual_timescale_gdn2_trajectory_memory",
"memory_architecture": "compact_gdn2_v2",
"base_model": "google/diffusiongemma-26B-A4B-it",
"source_manifest_sha256": "e2b64fe2102ca1e002a62e279798abb3be271e3d9640a5afc5823cf233724dbc",
"precision_policy": "gdn2_small_fp32_v1",
"adapters": "unmerged",
"trainable_tensor_count": 632,
"total_tensor_count": 1323,
"shards": [
{
"file": "model-00001-of-00016.safetensors",
"bytes": 3898256004,
"sha256": "d3aa3fc0393a51acec27161569d566774030e0306a74748dd2a7a11fd8b535ec"
},
{
"file": "model-00002-of-00016.safetensors",
"bytes": 3798073636,
"sha256": "89339fd6f0521977fff9d4ae498e485ff409dbb8716cc924c8667c8bceaa19cd"
},
{
"file": "model-00003-of-00016.safetensors",
"bytes": 3290562732,
"sha256": "b9351e59c7c604b26712960b5d9f3460f5c1aace9ccb348999a2334d93fb613d"
},
{
"file": "model-00004-of-00016.safetensors",
"bytes": 3319472948,
"sha256": "1c89ba68a1f56e986dbf43175a7a7a6f2d046b04ac993c2837024f71b5d5923d"
},
{
"file": "model-00005-of-00016.safetensors",
"bytes": 3290562732,
"sha256": "4e42f45a5c409649c0ac8a262b26a0f644d10619d5f857568db855d12a865183"
},
{
"file": "model-00006-of-00016.safetensors",
"bytes": 3290562772,
"sha256": "da6e7d99f9801cb1b7007044f3baa66e767412d4350211a637174321eacee401"
},
{
"file": "model-00007-of-00016.safetensors",
"bytes": 3319473028,
"sha256": "ce1f462e324e162f9c0f41b1db3945290eb06c520da3d8ff0f582cde79a6fa96"
},
{
"file": "model-00008-of-00016.safetensors",
"bytes": 3290562812,
"sha256": "a163f6df13b9a3fbe1df187e42700b74d71fa08b331c521be568f71dec77a8f0"
},
{
"file": "model-00009-of-00016.safetensors",
"bytes": 3290562812,
"sha256": "eacc2e9ac62fa45f4fd1d391bfff490e327c19df3f70e7835e556287fc9926bd"
},
{
"file": "model-00010-of-00016.safetensors",
"bytes": 3319473028,
"sha256": "5fa9c8080a9c3954985226496dee261938f3d05cf81e4436932de593e5a01ae1"
},
{
"file": "model-00011-of-00016.safetensors",
"bytes": 3290562812,
"sha256": "7df1df1cea73133a94aec88056e2514e04a3467d42c0dd55ad0b922af2e5e986"
},
{
"file": "model-00012-of-00016.safetensors",
"bytes": 3290562812,
"sha256": "797078b4703d602cc3a323dba676fa38bb959b70d455bdc8c673663261eee2d4"
},
{
"file": "model-00013-of-00016.safetensors",
"bytes": 3319473028,
"sha256": "35923e8f76bc78ae89b3f83d1cdd175649f64cc898f1862baf543d7cdfbcd0e1"
},
{
"file": "model-00014-of-00016.safetensors",
"bytes": 3290562812,
"sha256": "105ceea4ed17c811fb79036c199db1f7d2392a5cc4c05003085f7217e8a938bb"
},
{
"file": "model-00015-of-00016.safetensors",
"bytes": 3290562812,
"sha256": "d3569ec55bc4f7e89de00efaedfbca3df44096526291731ec14bd7fc245c3b39"
},
{
"file": "model-00016-of-00016.safetensors",
"bytes": 1202380654,
"sha256": "23c9c00fc04b475c744458f498178a6e9a20fd673c36af981dd22e963b7ca49b"
}
],
"validated_environment": {
"python": "3.12",
"torch": "2.14.0",
"transformers": "5.14.1",
"safetensors": "0.8.0",
"accelerate": "1.14.0"
}
}
|