Text Generation
PEFT
Safetensors
Transformers
English
llama
lora
dpo
smollm2
trl
conversational
text-generation-inference
Instructions to use Subject-Emu-5259/NeuralAI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Subject-Emu-5259/NeuralAI with PEFT:
Base model is not found.
- Transformers
How to use Subject-Emu-5259/NeuralAI with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Subject-Emu-5259/NeuralAI") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Subject-Emu-5259/NeuralAI") model = AutoModelForCausalLM.from_pretrained("Subject-Emu-5259/NeuralAI", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Subject-Emu-5259/NeuralAI with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Subject-Emu-5259/NeuralAI" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Subject-Emu-5259/NeuralAI", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Subject-Emu-5259/NeuralAI
- SGLang
How to use Subject-Emu-5259/NeuralAI with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Subject-Emu-5259/NeuralAI" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Subject-Emu-5259/NeuralAI", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Subject-Emu-5259/NeuralAI" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Subject-Emu-5259/NeuralAI", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Subject-Emu-5259/NeuralAI with Docker Model Runner:
docker model run hf.co/Subject-Emu-5259/NeuralAI
| declare(strict_types=1); | |
| use OC\Core\Service\CronService; | |
| use OCP\Server; | |
| use Psr\Log\LoggerInterface; | |
| /** | |
| * SPDX-FileCopyrightText: 2016-2024 Nextcloud GmbH and Nextcloud contributors | |
| * SPDX-FileCopyrightText: 2016 ownCloud, Inc. | |
| * SPDX-License-Identifier: AGPL-3.0-only | |
| */ | |
| require_once __DIR__ . '/lib/versioncheck.php'; | |
| try { | |
| require_once __DIR__ . '/lib/base.php'; | |
| if (isset($argv[1]) && ($argv[1] === '-h' || $argv[1] === '--help')) { | |
| echo 'Description: | |
| Run the background job routine | |
| Usage: | |
| php -f cron.php -- [-h] [--verbose] [<job-classes>...] | |
| Arguments: | |
| job-classes Optional job class list to only run those jobs | |
| Providing a class will ignore the time-sensitivity restriction | |
| Options: | |
| -h, --help Display this help message | |
| -v, --verbose Output more information' . PHP_EOL; | |
| exit(0); | |
| } | |
| $cronService = Server::get(CronService::class); | |
| if (isset($argv[1])) { | |
| $verbose = $argv[1] === '-v' || $argv[1] === '--verbose'; | |
| $jobClasses = array_slice($argv, $verbose ? 2 : 1); | |
| $jobClasses = empty($jobClasses) ? null : $jobClasses; | |
| if ($verbose) { | |
| $cronService->registerVerboseCallback(function (string $message): void { | |
| echo $message . PHP_EOL; | |
| }); | |
| } | |
| } else { | |
| $jobClasses = null; | |
| } | |
| $cronService->run($jobClasses); | |
| if (!OC::$CLI) { | |
| $data = [ | |
| 'status' => 'success', | |
| ]; | |
| header('Content-Type: application/json; charset=utf-8'); | |
| echo json_encode($data, JSON_HEX_TAG); | |
| } | |
| exit(0); | |
| } catch (Throwable $e) { | |
| Server::get(LoggerInterface::class)->error( | |
| $e->getMessage(), | |
| ['app' => 'cron', 'exception' => $e] | |
| ); | |
| if (OC::$CLI) { | |
| echo $e->getMessage() . PHP_EOL; | |
| } else { | |
| $data = [ | |
| 'status' => 'error', | |
| 'message' => $e->getMessage(), | |
| ]; | |
| header('Content-Type: application/json; charset=utf-8'); | |
| echo json_encode($data, JSON_HEX_TAG); | |
| } | |
| exit(1); | |
| } | |