Text Generation
Transformers
Safetensors
deepseek_v3
conversational
custom_code
text-generation-inference
fp8
Instructions to use lactroiii/DeepSeek-V3.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use lactroiii/DeepSeek-V3.1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="lactroiii/DeepSeek-V3.1", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("lactroiii/DeepSeek-V3.1", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("lactroiii/DeepSeek-V3.1", trust_remote_code=True, device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use lactroiii/DeepSeek-V3.1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "lactroiii/DeepSeek-V3.1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lactroiii/DeepSeek-V3.1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/lactroiii/DeepSeek-V3.1
- SGLang
How to use lactroiii/DeepSeek-V3.1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "lactroiii/DeepSeek-V3.1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lactroiii/DeepSeek-V3.1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "lactroiii/DeepSeek-V3.1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lactroiii/DeepSeek-V3.1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use lactroiii/DeepSeek-V3.1 with Docker Model Runner:
docker model run hf.co/lactroiii/DeepSeek-V3.1
| <meta charset="utf-8"> | |
| <style> | |
| body { margin:16px; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono","Courier New", monospace; white-space:pre-wrap; word-break:break-word; line-height:1.6; } | |
| .token { padding:0 2px; border-radius:4px; } | |
| .yellow { background:#FFF59D; } | |
| .green { background:#C8E6C9; } | |
| .red { background:#FFCDD2; } | |
| .blue { background:#BBDEFB; } | |
| </style> | |
| <div>You are a helpful software engineer assistant. | |
| ## Tools | |
| You have access to the following tools: | |
| ### bash | |
| Description: Run commands in a bash shell | |
| * When invoking this tool, the contents of the "command" parameter does NOT need to be XML-escaped. | |
| * You don't have access to the internet via this tool. | |
| * You do have access to a mirror of common linux and python packages via apt and pip. | |
| * State is persistent across command calls and discussions with the user. | |
| * To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'. | |
| * Please avoid commands that may produce a very large amount of output. | |
| * Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background. | |
| Parameters: {"title": "BashInput", "type": "object", "properties": {"command": {"title": "Command", "description": "The bash command to run. Relative path is preferred in the command.", "type": "string"}}, "required": ["command"], "additionalProperties": false} | |
| ### str_replace_editor | |
| Description: Custom editing tool for viewing, creating and editing files | |
| * State is persistent across command calls and discussions with the user | |
| * If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep | |
| * The `create` command cannot be used if the specified `path` already exists as a file | |
| * If a `command` generates a long output, it will be truncated and marked with `<response clipped>` | |
| Notes for using the `str_replace` command: | |
| * The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces! | |
| * If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique | |
| * The `new_str` parameter should contain the edited lines that should replace the `old_str` | |
| Parameters: {"title": "SweEditorInput", "type": "object", "properties": {"command": {"title": "Command", "description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.", "enum": ["view", "create", "str_replace", "insert"], "type": "string"}, "path": {"title": "Path", "description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`.", "type": "string"}, "file_text": {"title": "File Text", "description": "Required parameter of `create` command, with the content of the file to be created.", "type": "string"}, "insert_line": {"title": "Insert Line", "description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`.", "type": "integer"}, "new_str": {"title": "New Str", "description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert.", "type": "string"}, "old_str": {"title": "Old Str", "description": "Required parameter of `str_replace` command containing the string in `path` to replace.", "type": "string"}, "view_range": {"title": "View Range", "description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.", "type": "array", "items": {"type": "integer"}}}, "required": ["command", "path"], "additionalProperties": false} | |
| IMPORTANT: ALWAYS adhere to this exact format for tool use: | |
| <span class="token green"><|tool▁calls▁begin|></span><span class="token green"><|tool▁call▁begin|></span>tool_call_name<span class="token green"><|tool▁sep|></span>tool_call_arguments<span class="token green"><|tool▁call▁end|></span>{additional_tool_calls}<span class="token green"><|tool▁calls▁end|></span> | |
| Where: | |
| - `tool_call_name` must be an exact match to one of the available tools | |
| - `tool_call_arguments` must be valid JSON that strictly follows the tool's Parameters Schema | |
| - For multiple tool calls, chain them directly without separators or spaces<span class="token yellow"><|User|></span><uploaded_files> | |
| /testbed | |
| </uploaded_files> | |
| I've uploaded a code repository in the directory /testbed (not in /tmp/inputs). Consider the following PR description: | |
| <pr_description> | |
| Bug in posix implementation of django/core/files/locks.py | |
| Description | |
| The posix version of locks (the version which supports import fcntl) has a bug. The code attempts to return True to indicate success or failure acquiring a lock, but instead it always returns False. The reason is that cpython fcntl module returns None if successful, and raises an OSError to indicate failure (see https://docs.python.org/3/library/fcntl.html#fcntl.flock). | |
| Anyone interested in using the non-blocking (i.e. locks.LOCKS_NB) requires a valid return value to know if they have successfully acquired the lock. | |
| I believe the correct implementation should be the following: | |
| diff --git a/django/core/files/locks.py b/django/core/files/locks.py | |
| index c46b00b905..4938347ea7 100644 | |
| --- a/django/core/files/locks.py | |
| +++ b/django/core/files/locks.py | |
| @@ -107,9 +107,15 @@ else: | |
| return True | |
| else: | |
| def lock(f, flags): | |
| - ret = fcntl.flock(_fd(f), flags) | |
| - return ret == 0 | |
| + try: | |
| + fcntl.flock(_fd(f), flags) | |
| + return True | |
| + except OSError: | |
| + return False | |
| def unlock(f): | |
| - ret = fcntl.flock(_fd(f), fcntl.LOCK_UN) | |
| - return ret == 0 | |
| + try: | |
| + fcntl.flock(_fd(f), fcntl.LOCK_UN) | |
| + return True | |
| + except OSError: | |
| + return False | |
| </pr_description> | |
| Can you help me implement the necessary changes to the repository so that the requirements specified in the <pr_description> are met? | |
| I've already taken care of all changes to any of the test files described in the <pr_description>. This means you DON'T have to modify the testing logic or any of the tests in any way! | |
| Your task is to make the minimal changes to non-tests files in the /testbed directory to ensure the <pr_description> is satisfied. | |
| You are only allowed to call **ONE** function each time!<span class="token yellow"><|Assistant|></span><span class="token red"></think></span>I'll help you implement the necessary changes to fix the bug in the posix implementation of `django/core/files/locks.py`. Let me start by examining the current state of the file.<span class="token green"><|tool▁calls▁begin|></span><span class="token green"><|tool▁call▁begin|></span>str_replace_editor<span class="token green"><|tool▁sep|></span>{"command": "view", "path": "/testbed/django/core/files/locks.py"}<span class="token green"><|tool▁call▁end|></span><span class="token green"><|tool▁calls▁end|></span><span class="token yellow"><|end▁of▁sentence|></span><span class="token green"><|tool▁output▁begin|></span>Here's the content of /testbed/django/core/files/locks.py with line numbers (which has a total of 116 lines): | |
| 1 """ | |
| 2 Portable file locking utilities. | |
| 3 | |
| 4 Based partially on an example by Jonathan Feignberg in the Python | |
| 5 Cookbook [1] (licensed under the Python Software License) and a ctypes port by | |
| 6 Anatoly Techtonik for Roundup [2] (license [3]). | |
| 7 | |
| 8 [1] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65203 | |
| 9 [2] https://sourceforge.net/p/roundup/code/ci/default/tree/roundup/backends/portalocker.py | |
| 10 [3] https://sourceforge.net/p/roundup/code/ci/default/tree/COPYING.txt | |
| 11 | |
| 12 Example Usage:: | |
| 13 | |
| 14 >>> from django.core.files import locks | |
| 15 >>> with open('./file', 'wb') as f: | |
| 16 ... locks.lock(f, locks.LOCK_EX) | |
| 17 ... f.write('Django') | |
| 18 """ | |
| 19 import os | |
| 20 | |
| 21 __all__ = ('LOCK_EX', 'LOCK_SH', 'LOCK_NB', 'lock', 'unlock') | |
| 22 | |
| 23 | |
| 24 def _fd(f): | |
| 25 """Get a filedescriptor from something which could be a file or an fd.""" | |
| 26 return f.fileno() if hasattr(f, 'fileno') else f | |
| 27 | |
| 28 | |
| 29 if os.name == 'nt': | |
| 30 import msvcrt | |
| 31 from ctypes import ( | |
| 32 POINTER, Structure, Union, byref, c_int64, c_ulong, c_void_p, sizeof, | |
| 33 windll, | |
| 34 ) | |
| 35 from ctypes.wintypes import BOOL, DWORD, HANDLE | |
| 36 | |
| 37 LOCK_SH = 0 # the default | |
| 38 LOCK_NB = 0x1 # LOCKFILE_FAIL_IMMEDIATELY | |
| 39 LOCK_EX = 0x2 # LOCKFILE_EXCLUSIVE_LOCK | |
| 40 | |
| 41 # --- Adapted from the pyserial project --- | |
| 42 # detect size of ULONG_PTR | |
| 43 if sizeof(c_ulong) != sizeof(c_void_p): | |
| 44 ULONG_PTR = c_int64 | |
| 45 else: | |
| 46 ULONG_PTR = c_ulong | |
| 47 PVOID = c_void_p | |
| 48 | |
| 49 # --- Union inside Structure by stackoverflow:3480240 --- | |
| 50 class _OFFSET(Structure): | |
| 51 _fields_ = [ | |
| 52 ('Offset', DWORD), | |
| 53 ('OffsetHigh', DWORD)] | |
| 54 | |
| 55 class _OFFSET_UNION(Union): | |
| 56 _anonymous_ = ['_offset'] | |
| 57 _fields_ = [ | |
| 58 ('_offset', _OFFSET), | |
| 59 ('Pointer', PVOID)] | |
| 60 | |
| 61 class OVERLAPPED(Structure): | |
| 62 _anonymous_ = ['_offset_union'] | |
| 63 _fields_ = [ | |
| 64 ('Internal', ULONG_PTR), | |
| 65 ('InternalHigh', ULONG_PTR), | |
| 66 ('_offset_union', _OFFSET_UNION), | |
| 67 ('hEvent', HANDLE)] | |
| 68 | |
| 69 LPOVERLAPPED = POINTER(OVERLAPPED) | |
| 70 | |
| 71 # --- Define function prototypes for extra safety --- | |
| 72 LockFileEx = windll.kernel32.LockFileEx | |
| 73 LockFileEx.restype = BOOL | |
| 74 LockFileEx.argtypes = [HANDLE, DWORD, DWORD, DWORD, DWORD, LPOVERLAPPED] | |
| 75 UnlockFileEx = windll.kernel32.UnlockFileEx | |
| 76 UnlockFileEx.restype = BOOL | |
| 77 UnlockFileEx.argtypes = [HANDLE, DWORD, DWORD, DWORD, LPOVERLAPPED] | |
| 78 | |
| 79 def lock(f, flags): | |
| 80 hfile = msvcrt.get_osfhandle(_fd(f)) | |
| 81 overlapped = OVERLAPPED() | |
| 82 ret = LockFileEx(hfile, flags, 0, 0, 0xFFFF0000, byref(overlapped)) | |
| 83 return bool(ret) | |
| 84 | |
| 85 def unlock(f): | |
| 86 hfile = msvcrt.get_osfhandle(_fd(f)) | |
| 87 overlapped = OVERLAPPED() | |
| 88 ret = UnlockFileEx(hfile, 0, 0, 0xFFFF0000, byref(overlapped)) | |
| 89 return bool(ret) | |
| 90 else: | |
| 91 try: | |
| 92 import fcntl | |
| 93 LOCK_SH = fcntl.LOCK_SH # shared lock | |
| 94 LOCK_NB = fcntl.LOCK_NB # non-blocking | |
| 95 LOCK_EX = fcntl.LOCK_EX | |
| 96 except (ImportError, AttributeError): | |
| 97 # File locking is not supported. | |
| 98 LOCK_EX = LOCK_SH = LOCK_NB = 0 | |
| 99 | |
| 100 # Dummy functions that don't do anything. | |
| 101 def lock(f, flags): | |
| 102 # File is not locked | |
| 103 return False | |
| 104 | |
| 105 def unlock(f): | |
| 106 # File is unlocked | |
| 107 return True | |
| 108 else: | |
| 109 def lock(f, flags): | |
| 110 ret = fcntl.flock(_fd(f), flags) | |
| 111 return ret == 0 | |
| 112 | |
| 113 def unlock(f): | |
| 114 ret = fcntl.flock(_fd(f), fcntl.LOCK_UN) | |
| 115 return ret == 0 | |
| 116 | |
| <span class="token green"><|tool▁output▁end|></span>Now I can see the issue. Looking at the current implementation in lines 109-115, the code is trying to compare the return value of `fcntl.flock()` with 0, but according to the PR description, `fcntl.flock()` returns `None` on success and raises an `OSError` on failure. This means the current implementation will always return `False` because `None == 0` is `False`. | |
| Let me examine the specific lines that need to be changed:<span class="token green"><|tool▁calls▁begin|></span><span class="token green"><|tool▁call▁begin|></span>str_replace_editor<span class="token green"><|tool▁sep|></span>{"command": "view", "path": "/testbed/django/core/files/locks.py", "view_range": [109,116]}<span class="token green"><|tool▁call▁end|></span><span class="token green"><|tool▁calls▁end|></span><span class="token yellow"><|end▁of▁sentence|></span><span class="token green"><|tool▁output▁begin|></span>Here's the content of /testbed/django/core/files/locks.py with line numbers (which has a total of 116 lines) with view_range=[109, 116]: | |
| 109 def lock(f, flags): | |
| 110 ret = fcntl.flock(_fd(f), flags) | |
| 111 return ret == 0 | |
| 112 | |
| 113 def unlock(f): | |
| 114 ret = fcntl.flock(_fd(f), fcntl.LOCK_UN) | |
| 115 return ret == 0 | |
| 116 | |
| <span class="token green"><|tool▁output▁end|></span>Now I'll implement the fix as described in the PR description. The fix should wrap the `fcntl.flock()` calls in try-except blocks, returning `True` on success and `False` on `OSError`:<span class="token green"><|tool▁calls▁begin|></span><span class="token green"><|tool▁call▁begin|></span>str_replace_editor<span class="token green"><|tool▁sep|></span>{"command": "str_replace", "path": "/testbed/django/core/files/locks.py", "old_str": " def lock(f, flags):\n ret = fcntl.flock(_fd(f), flags)\n return ret == 0\n\n def unlock(f):\n ret = fcntl.flock(_fd(f), fcntl.LOCK_UN)\n return ret == 0", "new_str": " def lock(f, flags):\n try:\n fcntl.flock(_fd(f), flags)\n return True\n except OSError:\n return False\n\n def unlock(f):\n try:\n fcntl.flock(_fd(f), fcntl.LOCK_UN)\n return True\n except OSError:\n return False"}<span class="token green"><|tool▁call▁end|></span><span class="token green"><|tool▁calls▁end|></span><span class="token yellow"><|end▁of▁sentence|></span><span class="token green"><|tool▁output▁begin|></span>The file /testbed/django/core/files/locks.py has been edited successfully.<span class="token green"><|tool▁output▁end|></span>Let me verify the changes by viewing the modified section:<span class="token green"><|tool▁calls▁begin|></span><span class="token green"><|tool▁call▁begin|></span>str_replace_editor<span class="token green"><|tool▁sep|></span>{"command": "view", "path": "/testbed/django/core/files/locks.py", "view_range": [109,116]}<span class="token green"><|tool▁call▁end|></span><span class="token green"><|tool▁calls▁end|></span><span class="token yellow"><|end▁of▁sentence|></span><span class="token green"><|tool▁output▁begin|></span>Here's the content of /testbed/django/core/files/locks.py with line numbers (which has a total of 122 lines) with view_range=[109, 116]: | |
| 109 def lock(f, flags): | |
| 110 try: | |
| 111 fcntl.flock(_fd(f), flags) | |
| 112 return True | |
| 113 except OSError: | |
| 114 return False | |
| 115 | |
| 116 def unlock(f): | |
| <span class="token green"><|tool▁output▁end|></span><span class="token green"><|tool▁calls▁begin|></span><span class="token green"><|tool▁call▁begin|></span>str_replace_editor<span class="token green"><|tool▁sep|></span>{"command": "view", "path": "/testbed/django/core/files/locks.py", "view_range": [116,122]}<span class="token green"><|tool▁call▁end|></span><span class="token green"><|tool▁calls▁end|></span><span class="token yellow"><|end▁of▁sentence|></span><span class="token green"><|tool▁output▁begin|></span>Here's the content of /testbed/django/core/files/locks.py with line numbers (which has a total of 122 lines) with view_range=[116, 122]: | |
| 116 def unlock(f): | |
| 117 try: | |
| 118 fcntl.flock(_fd(f), fcntl.LOCK_UN) | |
| 119 return True | |
| 120 except OSError: | |
| 121 return False | |
| 122 | |
| <span class="token green"><|tool▁output▁end|></span>Perfect! The changes have been successfully implemented. Let me verify the complete file structure to ensure everything looks correct:<span class="token green"><|tool▁calls▁begin|></span><span class="token green"><|tool▁call▁begin|></span>str_replace_editor<span class="token green"><|tool▁sep|></span>{"command": "view", "path": "/testbed/django/core/files/locks.py", "view_range": [90,122]}<span class="token green"><|tool▁call▁end|></span><span class="token green"><|tool▁calls▁end|></span><span class="token yellow"><|end▁of▁sentence|></span><span class="token green"><|tool▁output▁begin|></span>Here's the content of /testbed/django/core/files/locks.py with line numbers (which has a total of 122 lines) with view_range=[90, 122]: | |
| 90 else: | |
| 91 try: | |
| 92 import fcntl | |
| 93 LOCK_SH = fcntl.LOCK_SH # shared lock | |
| 94 LOCK_NB = fcntl.LOCK_NB # non-blocking | |
| 95 LOCK_EX = fcntl.LOCK_EX | |
| 96 except (ImportError, AttributeError): | |
| 97 # File locking is not supported. | |
| 98 LOCK_EX = LOCK_SH = LOCK_NB = 0 | |
| 99 | |
| 100 # Dummy functions that don't do anything. | |
| 101 def lock(f, flags): | |
| 102 # File is not locked | |
| 103 return False | |
| 104 | |
| 105 def unlock(f): | |
| 106 # File is unlocked | |
| 107 return True | |
| 108 else: | |
| 109 def lock(f, flags): | |
| 110 try: | |
| 111 fcntl.flock(_fd(f), flags) | |
| 112 return True | |
| 113 except OSError: | |
| 114 return False | |
| 115 | |
| 116 def unlock(f): | |
| 117 try: | |
| 118 fcntl.flock(_fd(f), fcntl.LOCK_UN) | |
| 119 return True | |
| 120 except OSError: | |
| 121 return False | |
| 122 | |
| <span class="token green"><|tool▁output▁end|></span>Excellent! The changes have been successfully implemented. Let me run a quick test to make sure the file is syntactically correct:<span class="token green"><|tool▁calls▁begin|></span><span class="token green"><|tool▁call▁begin|></span>bash<span class="token green"><|tool▁sep|></span>{"command": "cd /testbed && python -m py_compile django/core/files/locks.py"}<span class="token green"><|tool▁call▁end|></span><span class="token green"><|tool▁calls▁end|></span><span class="token yellow"><|end▁of▁sentence|></span><span class="token green"><|tool▁output▁begin|></span><span class="token green"><|tool▁output▁end|></span>Great! The file compiles without any syntax errors. | |
| ## Summary | |
| I have successfully implemented the fix for the bug in the posix implementation of `django/core/files/locks.py`. The changes made are: | |
| 1. **Modified the `lock()` function** (lines 109-114): Wrapped the `fcntl.flock()` call in a try-except block that returns `True` on success and `False` when an `OSError` is raised. | |
| 2. **Modified the `unlock()` function** (lines 116-121): Similarly wrapped the `fcntl.flock()` call in a try-except block that returns `True` on success and `False` when an `OSError` is raised. | |
| The fix addresses the core issue described in the PR: | |
| - The original code incorrectly compared the return value of `fcntl.flock()` with 0, but `fcntl.flock()` returns `None` on success and raises an `OSError` on failure | |
| - This meant the functions always returned `False` (`None == 0` is `False`) | |
| - The new implementation properly handles the success/failure cases by using try-except blocks | |
| - This is particularly important for non-blocking locks (`LOCK_NB`) where callers need to know if the lock was successfully acquired | |
| The changes are minimal and focused, exactly matching the suggested fix in the PR description.<span class="token yellow"><|end▁of▁sentence|></span></div> | |