Instructions to use InfosysEnterprise/NT-Java-1.1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use InfosysEnterprise/NT-Java-1.1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="InfosysEnterprise/NT-Java-1.1B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("InfosysEnterprise/NT-Java-1.1B")
model = AutoModelForCausalLM.from_pretrained("InfosysEnterprise/NT-Java-1.1B")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use InfosysEnterprise/NT-Java-1.1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "InfosysEnterprise/NT-Java-1.1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InfosysEnterprise/NT-Java-1.1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/InfosysEnterprise/NT-Java-1.1B

SGLang

How to use InfosysEnterprise/NT-Java-1.1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "InfosysEnterprise/NT-Java-1.1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InfosysEnterprise/NT-Java-1.1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "InfosysEnterprise/NT-Java-1.1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InfosysEnterprise/NT-Java-1.1B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use InfosysEnterprise/NT-Java-1.1B with Docker Model Runner:
```
docker model run hf.co/InfosysEnterprise/NT-Java-1.1B
```

rajabmondal commited on Jun 25, 2024

Commit

cfa2c13

verified ·

1 Parent(s): 4f418e2

Update README.md

Browse files

Files changed (1) hide show

README.md +12 -12

README.md CHANGED Viewed

@@ -116,18 +116,6 @@ print(tokenizer.decode(outputs[0]))
 ```
   <br>
-# Attribution & Other Requirements
-The pretraining dataset for the model was curated to include only data with permissive licenses. Despite this, the model is capable of generating source code verbatim from the dataset. The licenses of such code may necessitate attribution and adherence to other specific conditions. To facilitate compliance, we provide a [search index](https://huggingface.co/spaces/bigcode/search) that enables users to trace the origins of generated code within the pretraining data, allowing for proper attribution and adherence to licensing requirements.
-  <br>
-# Limitations
-The NT-Java-1.1B model has been trained on publicly available datasets and is offered without any safety guarantees. As with all language models, its outputs are inherently unpredictable, and the generated code may not perform as expected. Additionally, the code may be inefficient or contain bugs and security vulnerabilities. Consequently, it is imperative for users and developers to undertake extensive safety testing and to implement robust filtering mechanisms tailored to their specific needs.
-  <br>
 # Training
 ## Model
@@ -150,6 +138,18 @@ The NT-Java-1.1B model has been trained on publicly available datasets and is of
 - **Neural networks:** [PyTorch](https://github.com/pytorch/pytorch)
   <br>
 # License
 The model is licensed under the BigCode OpenRAIL-M v1 license agreement. You can find the full agreement [here](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement).

 ```
   <br>
 # Training
 ## Model
 - **Neural networks:** [PyTorch](https://github.com/pytorch/pytorch)
   <br>
+# Attribution & Other Requirements
+The pretraining dataset for the model was curated to include only data with permissive licenses. Despite this, the model is capable of generating source code verbatim from the dataset. The licenses of such code may necessitate attribution and adherence to other specific conditions. To facilitate compliance, we provide a [search index](https://huggingface.co/spaces/bigcode/search) that enables users to trace the origins of generated code within the pretraining data, allowing for proper attribution and adherence to licensing requirements.
+  <br>
+# Limitations
+The NT-Java-1.1B model has been trained on publicly available datasets and is offered without any safety guarantees. As with all language models, its outputs are inherently unpredictable, and the generated code may not perform as expected. Additionally, the code may be inefficient or contain bugs and security vulnerabilities. Consequently, it is imperative for users and developers to undertake extensive safety testing and to implement robust filtering mechanisms tailored to their specific needs.
+  <br>
 # License
 The model is licensed under the BigCode OpenRAIL-M v1 license agreement. You can find the full agreement [here](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement).