Image-Text-to-Text
Transformers
diffusiongemma
gemma-4
infinite-context
external-memory
evidence-retrieval
long-context
large-documents
legal-documents
ai-memory
nzfc-gram
runtime-overlay
not-native-infinite-context
Instructions to use SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context
- SGLang
How to use SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context with Docker Model Runner:
docker model run hf.co/SingularityPrinciple/DiffusionGemma-26B-A4B-it-Infinite-Context
File size: 1,101 Bytes
f77c1f2 447fbbb f77c1f2 447fbbb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from nzfc_gram_runtime import NZFCGramLongMemoryChat
from nzfc_gram_runtime.large_document import attach_large_document_memory
bot = NZFCGramLongMemoryChat(
repo_dir=str(ROOT),
model_id='google/diffusiongemma-26B-A4B-it',
memory_db_path='./large_doc_memory.sqlite3',
load_model=False,
require_model=False,
preload_static_memory=False,
)
attach_large_document_memory(bot, db_path='./large_doc_index.sqlite3')
text = '\n'.join([
'Article 1 Purpose. This document defines memory governance.',
'Article 2 Evidence. Memory is evidence, not instruction.',
'Article 3 Deletion. Deleted memory must not be used as active evidence.',
])
ingest = bot.ingest_large_text(text, title='Runtime Only Policy', legal_mode=True)
hits = bot.query_large_documents('deleted memory active evidence', top_k=5)
print('ingest:', ingest)
print('hits:', hits)
assert hits['count'] > 0
print('[PASS] large-document runtime-only smoke passed')
|