Image-Text-to-Text
Transformers
Safetensors
modilify_mk1
text-generation
diffusion
multimodal
mixture-of-experts
trust-remote-code
conversational
custom_code
Instructions to use modilify/Modilify-Mk1-preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use modilify/Modilify-Mk1-preview with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="modilify/Modilify-Mk1-preview", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("modilify/Modilify-Mk1-preview", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use modilify/Modilify-Mk1-preview with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "modilify/Modilify-Mk1-preview" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk1-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/modilify/Modilify-Mk1-preview
- SGLang
How to use modilify/Modilify-Mk1-preview with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "modilify/Modilify-Mk1-preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk1-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "modilify/Modilify-Mk1-preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "modilify/Modilify-Mk1-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use modilify/Modilify-Mk1-preview with Docker Model Runner:
docker model run hf.co/modilify/Modilify-Mk1-preview
| Modilify Open Model License 1.0 | |
| Draft for Independent Legal Review | |
| Copyright 2026 Modilify | |
| 1. Definitions | |
| "Model" means the weights, configuration, inference code, tokenizer, processor, | |
| and documentation distributed with this license. "Derivative Model" means a | |
| modified, fine-tuned, distilled, merged, quantized, or otherwise adapted version | |
| of the Model. "You" means the individual or legal entity exercising permissions | |
| under this license. "High-Risk Use" means a use that can materially affect a | |
| person's safety, liberty, access to essential services, employment, housing, | |
| credit, education, legal rights, or medical care, or that controls critical | |
| infrastructure, weapons, or large-scale biometric surveillance. | |
| 2. Copyright Grant | |
| Subject to this license, Modilify grants You a worldwide, perpetual, | |
| non-exclusive, royalty-free, irrevocable copyright license to use, reproduce, | |
| prepare derivative works of, publicly display, publicly perform, sublicense, | |
| host as a service, and distribute the Model and Derivative Models, including for | |
| commercial purposes. | |
| 3. Patent Grant | |
| Each contributor grants You a worldwide, perpetual, non-exclusive, royalty-free, | |
| irrevocable patent license, except as stated in this section, to make, have made, | |
| use, offer to sell, sell, import, and otherwise transfer the Model where the | |
| license applies only to patent claims necessarily infringed by that contributor's | |
| contribution alone or in combination with the Model. If You institute patent | |
| litigation alleging that the Model or a contribution constitutes patent | |
| infringement, patent licenses granted to You under this license terminate as of | |
| the filing date. | |
| 4. Conditions on Redistribution | |
| If You distribute the Model or a Derivative Model, You must: | |
| a. provide recipients a copy of this license; | |
| b. retain copyright, patent, attribution, and NOTICE statements; | |
| c. state clearly that You modified the Model and identify material modifications; | |
| d. preserve applicable third-party license and attribution notices; and | |
| e. publish with the distributed model a reasonably accessible impact statement | |
| describing intended uses, material limitations, evaluation scope, known | |
| safety risks, and risk mitigations for the Derivative Model. | |
| The impact statement may be maintained in a public model card or equivalent | |
| document. You are not required to submit it separately to Modilify. | |
| 5. Responsible Use and High-Risk Uses | |
| You must not use the Model or a Derivative Model: | |
| a. to develop, operate, or materially facilitate weapons, autonomous targeting, | |
| or systems intended to cause physical harm; | |
| b. for unlawful mass surveillance, biometric identification without lawful | |
| authority and appropriate safeguards, or social scoring that determines | |
| access to rights or essential services; | |
| c. to exploit children or vulnerable persons, facilitate human trafficking, or | |
| generate non-consensual intimate content; | |
| d. to impersonate a person or deceptively represent machine output as an | |
| authentic human communication where the deception is reasonably likely to | |
| cause material harm; or | |
| e. to make a final decision in a High-Risk Use without meaningful qualified | |
| human review, proportionate testing, monitoring, appeal or correction paths, | |
| and compliance with applicable law. | |
| Before deploying the Model in a High-Risk Use, You must perform safety and impact | |
| due diligence proportionate to foreseeable harm. At minimum, document the use | |
| context, evaluate relevant failure modes and affected groups, apply reasonable | |
| technical and organizational safeguards, monitor material incidents, and update | |
| or suspend the deployment when its residual risk is not reasonable. Research, | |
| testing, auditing, and defensive safety work are permitted when conducted with | |
| appropriate safeguards. | |
| 6. Trademarks | |
| This license does not grant permission to use the trade names, trademarks, | |
| service marks, or product names of Modilify or any contributor, except as needed | |
| for reasonable and customary attribution or to describe the origin of the Model. | |
| 7. Third-Party Components | |
| The Model includes or derives from third-party components identified in | |
| NOTICE.md. Those components remain subject to their applicable | |
| licenses and terms. In particular, rights and obligations associated with the | |
| Google DiffusionGemma base are not removed, narrowed, or replaced by this | |
| license. You are responsible for complying with all applicable upstream terms. | |
| 8. Termination and Reinstatement | |
| Your rights terminate automatically if You materially violate this license and | |
| do not cure the violation within 30 days after becoming aware of it. Rights are | |
| reinstated upon timely cure unless a rights holder provides written notice of a | |
| substantially similar repeated violation. Sections intended by their nature to | |
| survive termination remain effective. | |
| 9. Disclaimer of Warranty | |
| THE MODEL IS PROVIDED "AS IS," WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, | |
| EXPRESS OR IMPLIED, INCLUDING WARRANTIES OF TITLE, NON-INFRINGEMENT, | |
| MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, ACCURACY, OR SAFETY. YOU ARE | |
| SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING | |
| THE MODEL AND ASSUME ALL RISKS ASSOCIATED WITH YOUR EXERCISE OF PERMISSIONS. | |
| 10. Limitation of Liability | |
| TO THE MAXIMUM EXTENT PERMITTED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR SHALL | |
| BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR | |
| CONSEQUENTIAL DAMAGES ARISING FROM THIS LICENSE OR THE USE OR INABILITY TO USE | |
| THE MODEL, HOWEVER CAUSED AND UNDER ANY THEORY OF LIABILITY, EVEN IF ADVISED OF | |
| THE POSSIBILITY OF SUCH DAMAGES. | |
| 11. Governing Law and Venue | |
| This license is governed by the laws of the State of California, excluding its | |
| conflict-of-law rules. Any dispute arising from this license must be brought in | |
| the state or federal courts located in Santa Clara County, California, and each | |
| party consents to their personal jurisdiction and venue. | |
| 12. Entire License; Severability | |
| This document states the complete Modilify license for the Model, subject to | |
| applicable third-party terms. If a provision is unenforceable, it will be limited | |
| to the minimum extent necessary and the remaining provisions remain effective. | |
| This is a pre-release license draft. Obtain independent legal review before | |
| public distribution or production reliance. | |