Instructions to use NumbersStation/nsql-6B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use NumbersStation/nsql-6B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="NumbersStation/nsql-6B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("NumbersStation/nsql-6B") model = AutoModelForCausalLM.from_pretrained("NumbersStation/nsql-6B") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use NumbersStation/nsql-6B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "NumbersStation/nsql-6B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NumbersStation/nsql-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/NumbersStation/nsql-6B
- SGLang
How to use NumbersStation/nsql-6B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "NumbersStation/nsql-6B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NumbersStation/nsql-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "NumbersStation/nsql-6B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NumbersStation/nsql-6B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use NumbersStation/nsql-6B with Docker Model Runner:
docker model run hf.co/NumbersStation/nsql-6B
Is there a way to provide columns description in the prompt
Models like gpt3 or gpt3.5 allows to provide column name and description in the prompt. like (tablename({col1:col1description, col2:col2description}))
is there a way to do this in nsql prompt.
It is very useful incases where column name are not generic and description can provide more meaning to it.
Most of the questions ask in generic english. it not necessary to be the exact column name.
So is it possible in Nsql models?
We are currently using Rajkumar's prompt format for the NSQL model, it's not very user-friendly to include descriptions into the current prompt format. If you have any suggestions or would like to contribute, please let us know. Additionally, we are interested in incorporating table and column descriptions into the text2SQL dataset. If you have any data or pointers to share, we would be happy to include them in the next version of NSText2SQL.
Thanks for the prompt response.
There is dataset available on the below link. In this dataset column name are not very generic but there is a possibility of asking question in general english. Like show distinct material categories.
So if it possible to include column description also along with column names that would be great as there are many dataset where column name are more specific to domain and generic.
Dataset link
https://data.world/basilhayek/drug-indications-database
I am trying to use this model on our data. If I make any progress with the prompt I will definitely contribute.
Thanks! Will take a look at that dataset.