Instructions to use acrastt/Marx-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use acrastt/Marx-3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="acrastt/Marx-3B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("acrastt/Marx-3B") model = AutoModelForCausalLM.from_pretrained("acrastt/Marx-3B") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use acrastt/Marx-3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "acrastt/Marx-3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "acrastt/Marx-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/acrastt/Marx-3B
- SGLang
How to use acrastt/Marx-3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "acrastt/Marx-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "acrastt/Marx-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "acrastt/Marx-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "acrastt/Marx-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use acrastt/Marx-3B with Docker Model Runner:
docker model run hf.co/acrastt/Marx-3B
Is this model commercially available?
Are there any gray areas which can affect using it commercially?
Please suggest.
Thanks.
Are there any gray areas which can affect using it commercially?
Please suggest.
Thanks.
Thank you for asking. I am not a lawyer, and this is not legal advice, but here's my look at this:
- The base model, OpenLLaMA 3B V2, is under the Apache 2.0 license. This means you can use the base model without any restrictions.
- The original dataset used to train this model, is under the MIT license, which means there is also no commercial restrictions.
- Furthermore, from the OpenAI ToS/terms of use, there is the following content:
(c) Restrictions. You may not (i) use the Services in a way that infringes, misappropriates or violates any person’s rights; (ii) reverse assemble, reverse compile, decompile, translate or otherwise attempt to discover the source code or underlying components of models, algorithms, and systems of the Services (except to the extent such restrictions are contrary to applicable law); (iii) use output from the Services to develop models that compete with OpenAI; (iv) except as permitted through the API, use any automated or programmatic method to extract data or output from the Services, including scraping, web harvesting, or web data extraction; (v) represent that output from the Services was human-generated when it is not or otherwise violate our Usage Policies; (vi) buy, sell, or transfer API keys without our prior consent; or (vii), send us any personal information of children under 13 or the applicable age of digital consent. You will comply with any rate limits and other requirements in our documentation. You may use Services only in geographies currently supported by OpenAI.
this means that using output of OpenAI models to train models that compete with OpenAI is prohibited by the ToS/terms of use.
4. However, I don't think this model is competing with OpenAI in any way. The ToS/terms of use is also not a license, which means they cannot legally sue you for making models on data from OpenAI models to compete with OpenAI; the best they can do is probably ban your(In this case/perspective, the author of the dataset and/or me.) OpenAI account.
5. Furthermore, this model is licensed under the Apache 2.0 license(No commercial restrictions.) and I havn't got sued by OpenAI yet.