Text Generation
Transformers
Safetensors
English
llama
hand-crafted
compiled
autocomplete
no-training
n-gram
code
browser
text-generation-inference
Instructions to use tmickleydoyle/tiny-autocomplete with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tmickleydoyle/tiny-autocomplete with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="tmickleydoyle/tiny-autocomplete")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("tmickleydoyle/tiny-autocomplete") model = AutoModelForCausalLM.from_pretrained("tmickleydoyle/tiny-autocomplete", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tmickleydoyle/tiny-autocomplete with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tmickleydoyle/tiny-autocomplete" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tmickleydoyle/tiny-autocomplete", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/tmickleydoyle/tiny-autocomplete
- SGLang
How to use tmickleydoyle/tiny-autocomplete with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tmickleydoyle/tiny-autocomplete" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tmickleydoyle/tiny-autocomplete", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tmickleydoyle/tiny-autocomplete" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tmickleydoyle/tiny-autocomplete", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use tmickleydoyle/tiny-autocomplete with Docker Model Runner:
docker model run hf.co/tmickleydoyle/tiny-autocomplete
File size: 3,028 Bytes
4113449 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | <!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>tiny-autocomplete · in-browser</title>
<style>
:root { --bg: #fafaf9; --fg: #1c1917; --muted: #78716c; --line: #e7e5e4; --accent: #2563eb; --ghost: #a8a29e; --card: #fff; }
@media (prefers-color-scheme: dark) {
:root { --bg: #0c0a09; --fg: #f5f5f4; --muted: #a8a29e; --line: #292524; --accent: #60a5fa; --ghost: #57534e; --card: #1c1917; }
}
* { box-sizing: border-box; }
body { margin: 0; font: 16px/1.5 system-ui, -apple-system, sans-serif; background: var(--bg); color: var(--fg); }
main { max-width: 720px; margin: 0 auto; padding: 48px 20px; }
h1 { font-size: 24px; margin: 0 0 4px; }
.sub { color: var(--muted); margin: 0 0 24px; font-size: 14px; }
.editor { position: relative; background: var(--card); border: 1px solid var(--line); border-radius: 12px; }
.editor:focus-within { border-color: var(--accent); }
.layer { font: 20px/1.5 system-ui, -apple-system, sans-serif; padding: 16px 18px; margin: 0; white-space: pre-wrap; word-wrap: break-word; }
#ghost { position: absolute; inset: 0; pointer-events: none; color: transparent; }
#ghost span { color: var(--ghost); }
#prompt { position: relative; width: 100%; min-height: 64px; border: 0; outline: 0; resize: none; background: transparent; color: var(--fg); overflow: hidden; }
.row { display: flex; justify-content: space-between; gap: 16px; color: var(--muted); font-size: 13px; margin: 8px 2px; }
#guess { min-height: 20px; }
kbd { border: 1px solid var(--line); border-radius: 4px; padding: 0 5px; font-size: 12px; background: var(--card); }
label { display: flex; align-items: center; gap: 10px; color: var(--muted); font-size: 14px; margin: 20px 2px 28px; }
input[type=range] { flex: 1; accent-color: var(--accent); }
h2 { font-size: 13px; text-transform: uppercase; letter-spacing: .06em; color: var(--muted); margin: 0 0 10px; font-weight: 600; }
.examples { display: flex; flex-wrap: wrap; gap: 8px; }
.ex { border: 1px solid var(--line); background: var(--card); color: inherit; border-radius: 999px; padding: 4px 12px; font: inherit; font-size: 14px; cursor: pointer; }
.ex:hover { border-color: var(--accent); }
</style>
</head>
<body>
<main>
<h1>tiny-autocomplete</h1>
<p class="sub" id="load">loading...</p>
<div class="editor">
<pre class="layer" id="ghost" aria-hidden="true"></pre>
<textarea class="layer" id="prompt" rows="1" placeholder="Type a coding prompt, e.g. write a python fun" spellcheck="false" disabled></textarea>
</div>
<div class="row"><span><kbd>Tab</kbd> accept suggestion</span><span id="status"></span></div>
<div class="row"><span id="guess"></span></div>
<label>Confidence threshold <input type="range" id="tau" min="0.3" max="0.99" step="0.01"> <span id="tau-label"></span></label>
<h2>Try</h2>
<div class="examples" id="examples"></div>
</main>
<script type="module" src="demo.js"></script>
</body>
</html>
|