Instructions to use Attoloop/tensor-diet-bf16-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Attoloop/tensor-diet-bf16-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Attoloop/tensor-diet-bf16-gguf:BF16 # Run inference directly in the terminal: llama cli -hf Attoloop/tensor-diet-bf16-gguf:BF16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Attoloop/tensor-diet-bf16-gguf:BF16 # Run inference directly in the terminal: llama cli -hf Attoloop/tensor-diet-bf16-gguf:BF16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Attoloop/tensor-diet-bf16-gguf:BF16 # Run inference directly in the terminal: ./llama-cli -hf Attoloop/tensor-diet-bf16-gguf:BF16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Attoloop/tensor-diet-bf16-gguf:BF16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf Attoloop/tensor-diet-bf16-gguf:BF16
Use Docker
docker model run hf.co/Attoloop/tensor-diet-bf16-gguf:BF16
- LM Studio
- Jan
- Ollama
How to use Attoloop/tensor-diet-bf16-gguf with Ollama:
ollama run hf.co/Attoloop/tensor-diet-bf16-gguf:BF16
- Unsloth Desktop
- Docker Model Runner
How to use Attoloop/tensor-diet-bf16-gguf with Docker Model Runner:
docker model run hf.co/Attoloop/tensor-diet-bf16-gguf:BF16
- Lemonade
How to use Attoloop/tensor-diet-bf16-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Attoloop/tensor-diet-bf16-gguf:BF16
Run and chat with the model
lemonade run user.tensor-diet-bf16-gguf-BF16
List all available models
lemonade list
- Atomic Chat
tensor-diet BF16 GGUF checkpoints
BF16 GGUF conversions of 14 small open-weight checkpoints (0.36B-2.5B parameters), published so that the Attoloop tensor-diet environment can fetch exactly one checkpoint per task at image build, by URL and sha256, instead of copying it into every task.
Conversion
Each file is convert_hf_to_gguf.py MODEL --outtype bf16 --outfile model.bf16.gguf from ggml-org/llama.cpp at commit 5266f24da75dc449bd56cbed7addb9c8e4a6a73e (source tarball sha256 9c2948aa9c79c92dd0e4c98e11ff5cf76dfdcaebdeb18e3e93409e9a98aefdab), run on the upstream repository at the commit listed below.
Local patch: conversion/gpt2.py: discard non-weight attention-mask buffers. The weights are otherwise unmodified; tokenizer and model metadata come from the upstream repository.
Per-file provenance (upstream file sha256s, converter, parameter and tensor counts) is in provenance/<unit-id>.json.
Files
| file | source (repository @ commit) | licence | params | size | sha256 |
|---|---|---|---|---|---|
2dbb8481f1978fab3c9400ca44f08aca7e6e9199092dbc747ed045b0a1b1204c.bf16.gguf |
TinyLlama/TinyLlama-1.1B-Chat-v1.0 @ fe8a4ea1ffed |
Apache-2.0 | 1.10B | 2.05 GiB | b93994130c1b9776e54976a7d22b85c45d1fd88195542563008e765ee37a247e |
38c588c04bab71c21789b16028f30549af65b2f6108000a872e2eb3ebb14d306.bf16.gguf |
openai-community/gpt2-large @ 32b71b12589c |
MIT | 0.81B | 1.45 GiB | 4627b733d5118f725af4f261c6cb4d2b41d872887440a773d30a94418c0e3258 |
6c5d3830e153854ccaacac5e90949c1c7f3f08755c54687e46a215a80577a1bf.bf16.gguf |
HuggingFaceTB/SmolLM2-360M @ f8027fd0eaee |
Apache-2.0 | 0.36B | 0.68 GiB | adc161a2da4352cc8ca70f3827914b6abfbbbe1c13cfbf4eaf4b78a1e6e7e408 |
7d16e57d380c31f1a4eda85dfcfd80834b64f4b31bf65160bfd465b9154e6295.bf16.gguf |
BSC-LT/salamandra-2b @ f3a6092c9394 |
Apache-2.0 | 2.25B | 4.20 GiB | 466f8c1b6d606bf033eb25e4b69693710d537fb0822526e4b69a0af4643fdc57 |
84b565b8cb724dd2f7a87085521ff81701e63c4341883de6dd7d17c1bc6b3e26.bf16.gguf |
Qwen/Qwen3-1.7B @ 70d244cc86cc |
Apache-2.0 | 2.03B | 3.79 GiB | cc611cb04459f91b2887624815371cce9a58559ac1407ebdf6f8094cd9c4e88f |
85a5dcd806d0485d523e97d24f216a9212f5bbacacd0152858c35858ff112433.bf16.gguf |
Qwen/Qwen3-0.6B @ c1899de289a0 |
Apache-2.0 | 0.75B | 1.41 GiB | 42c1d86943a49302083650756249f415b0cd88de8cf08b379b35a4360ec866a4 |
a4c1b3551edc36652d0774e9b74bec9c45bf8531398cb77fe68703b8b95a8dd4.bf16.gguf |
Qwen/Qwen2.5-0.5B @ 060db6499f32 |
Apache-2.0 | 0.49B | 0.93 GiB | 97b0d1723dd1c8532fe8b39468579d56306005c298c35ec3ce18619b809adc45 |
bd8b549664aef799830b4c58ed7dca290f1f0d6372d67dccb9582c39019d0478.bf16.gguf |
EleutherAI/pythia-1b @ f73d7dcc545c |
Apache-2.0 | 1.08B | 1.89 GiB | 485f534f43e64ad8fafddb6287f064232741e1958f346c3078e8f8e3b3103c0d |
bf01e9f09337417adfb1bda016dead50955128c4255eec4019ac8fde024b81df.bf16.gguf |
ibm-granite/granite-3.3-2b-base @ c15dc681f4b2 |
Apache-2.0 | 2.53B | 4.72 GiB | cfde21064c4ab7ff82232e393d75e1dff62e1e6aa2534c3e6102ef0492ae371a |
bf7b1afc780d2a3a2c688b8c0a61039c2dd95287d9d544ff226cfbae41fb38d8.bf16.gguf |
h2oai/h2o-danube3-500m-base @ 0ac6d9d2999a |
Apache-2.0 | 0.51B | 0.96 GiB | 9d67585d2546df5490cb5f1c3059bb5c51e04632802f586668ab0f982461bd8b |
bfc01ca6cd2f12c6c52550ddd057734c43eee2f9b285c75cccdb41f9ac893edf.bf16.gguf |
Qwen/Qwen2.5-1.5B @ 8faed761d45a |
Apache-2.0 | 1.54B | 2.88 GiB | e3735110d9766f1106caf82455c25ba76bee96170318890cf49a4995d82f4f4a |
cdfa10bca2a1f5f032275492f683be47d7c3363da1791eb408148641ea070f4d.bf16.gguf |
HuggingFaceTB/SmolLM2-1.7B @ effd688a1292 |
Apache-2.0 | 1.71B | 3.19 GiB | 9d5a1448c8e3fa9f4e71c75627f791b39f9a2a695266b80b8d77ad06d2268730 |
cf5dbd78a595b52e677fef2fafc94a2431ab63964e7fe01dde5bbbb0efcbd383.bf16.gguf |
allenai/OLMo-2-0425-1B @ a1847dff3500 |
Apache-2.0 | 1.48B | 2.77 GiB | ce26243ac2e1a978485c280aa7aeff7111296e702a71d7fc59e43e578ef58b5d |
eca0b6c29ffc824ae53e327eb557fd1342b4ecdd8157cc45e2fea1389a092a72.bf16.gguf |
EleutherAI/pythia-410m @ 9879c9b5f8be |
Apache-2.0 | 0.51B | 0.76 GiB | e09b74ed6c373bb6dfdd5c8c09fba78465ea384ec0cc57dadd82de6cdc7d8063 |
Licences
Each checkpoint keeps the licence of its upstream repository: Apache-2.0 for 13 of them and MIT for
openai-community/gpt2-large. A copy of each upstream licence text is in licenses/<unit-id>/LICENSE.
These conversions are redistributed under those licences; the original authors hold their rights and have not
reviewed or endorsed this repository. Gated or custom-licensed families were deliberately excluded.
- Downloads last month
- 56
16-bit
Model tree for Attoloop/tensor-diet-bf16-gguf
Base model
BSC-LT/salamandra-2b