Text Generation
Transformers
Safetensors
qwen3
Neura Tech AI
neuron
instruct
llm
transformer
qwen
multilingual
conversational
coding
text-generation-inference
Instructions to use Neura-Tech-AI/Neuron-4B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Neura-Tech-AI/Neuron-4B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Neura-Tech-AI/Neuron-4B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Neura-Tech-AI/Neuron-4B-Instruct") model = AutoModelForCausalLM.from_pretrained("Neura-Tech-AI/Neuron-4B-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Neura-Tech-AI/Neuron-4B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Neura-Tech-AI/Neuron-4B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Neura-Tech-AI/Neuron-4B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Neura-Tech-AI/Neuron-4B-Instruct
- SGLang
How to use Neura-Tech-AI/Neuron-4B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Neura-Tech-AI/Neuron-4B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Neura-Tech-AI/Neuron-4B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Neura-Tech-AI/Neuron-4B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Neura-Tech-AI/Neuron-4B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Neura-Tech-AI/Neuron-4B-Instruct with Docker Model Runner:
docker model run hf.co/Neura-Tech-AI/Neuron-4B-Instruct
File size: 3,112 Bytes
c151375 eae91e8 6c83a63 eae91e8 6c83a63 eae91e8 6c83a63 eae91e8 6c83a63 eae91e8 6c83a63 c151375 eae91e8 c151375 eae91e8 6c83a63 eae91e8 c151375 eae91e8 6c83a63 eae91e8 6c83a63 eae91e8 6c83a63 eae91e8 c151375 eae91e8 c151375 eae91e8 6c83a63 eae91e8 c151375 eae91e8 c151375 eae91e8 c151375 6c83a63 c151375 eae91e8 c151375 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | {%- if tools %}
{{- '<|im_start|>system\n' }}
{%- if messages and messages[0].role == 'system' %}
You are Neuron, a large language model developed by Neura Tech AI. Always maintain this persona.
{{- messages[0].content }}
{%- else %}
You are Neuron, a large language model developed by Neura Tech AI. Always maintain this persona.
{%- endif %}
{{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a JSON object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
{%- if messages and messages[0].role == 'system' %}
{{- '<|im_start|>system\n' }}
You are Neuron, a large language model developed by Neura Tech AI. Always maintain this persona.
{{- messages[0].content }}
{{- '<|im_end|>\n' }}
{%- else %}
{{- '<|im_start|>system\nYou are Neuron, a large language model developed by Neura Tech AI. Always maintain this persona.<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- for message in messages %}
{%- if message.role == "user"
or (message.role == "system" and not loop.first)
or (message.role == "assistant" and not (message.tool_calls | default([]))) %}
{{- '<|im_start|>' + message.role + '\n' }}
{{- message.content | default("") }}
{{- '<|im_end|>\n' }}
{%- elif message.role == "assistant" %}
{{- '<|im_start|>assistant' }}
{%- if message.content %}
{{- '\n' }}
{{- message.content }}
{%- endif %}
{%- for tool_call in (message.tool_calls | default([])) %}
{%- if tool_call.function is defined %}
{%- set call = tool_call.function %}
{%- else %}
{%- set call = tool_call %}
{%- endif %}
{{- '\n<tool_call>\n{"name": "' }}
{{- call.name }}
{{- '", "arguments": ' }}
{%- if call.arguments is string %}
{{- call.arguments }}
{%- else %}
{{- call.arguments | tojson }}
{%- endif %}
{{- '}\n</tool_call>' }}
{%- endfor %}
{{- '<|im_end|>\n' }}
{%- elif message.role == "tool" %}
{%- if loop.first or messages[loop.index0 - 1].role != "tool" %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' }}
{{- message.content | default("") }}
{{- '\n</tool_response>' }}
{%- if loop.last or messages[loop.index0 + 1].role != "tool" %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- endif %} |