README.md

---
library_name: transformers
pipeline_tag: text-generation
tags:
  - custom-code
  - private-llm
---

# Private LLM Hugging Face Wrapper

This repository wraps `private_LLM_model.py` as a custom Hugging Face
Transformers model. The private script is loaded only at runtime.

> Loading this model requires `trust_remote_code=True` because it uses custom
> model and tokenizer code.

## Install

```bash
pip install -r requirements.txt
```

## Standard Text Generation Pipeline

```python
from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="YOUR_USERNAME/YOUR_REPO",
    trust_remote_code=True,
)

print(generator("Write a short greeting.", max_new_tokens=64))
```

## Direct Model Loading

```python
from transformers import AutoModelForCausalLM
from transformers import AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "YOUR_USERNAME/YOUR_REPO",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "YOUR_USERNAME/YOUR_REPO",
    trust_remote_code=True,
)

print(model.generate_text("Write a short greeting."))
```

## Optional Pipeline

```python
from transformers import pipeline

pipe = pipeline(
    "private-llm",
    model=".",
    trust_remote_code=True,
)

print(pipe("Write a short greeting."))
```

## Publish To The Hub

Authenticate first:

```bash
hf auth login
```

Then upload the current folder:

```bash
python publish_to_hub.py YOUR_USERNAME/YOUR_REPO
```

The publish script creates a private model repository by default. Use
`--public` only if you want the Hub repo to publicly expose
`private_LLM_model.py`.

## Private Script Entrypoints

The wrapper auto-detects these common patterns:

- Loader functions: `load_model`, `create_model`, `build_model`, `get_model`,
  `load_llm`
- Model objects: `model`, `llm`, `MODEL`, `LLM_MODEL`
- Model classes: `PrivateLLM`, `LLM`, `Model`
- Generation methods/functions: `generate_text`, `generate`, `complete`,
  `predict`, `chat`, `__call__`

If the script is meant to be run directly instead of imported, set this in
`config.json`:

```json
{
  "execution_mode": "subprocess"
}
```

Subprocess mode sends the prompt to stdin by default. It also sets
`PRIVATE_LLM_PROMPT` and `PRIVATE_LLM_KWARGS` environment variables.

To pin exact names without changing your private script, set fields like:

```json
{
  "loader_function": "load_model",
  "generate_function": "generate"
}
```

If your private script returns the full prompt plus completion instead of only
the completion text, set:

```json
{
  "private_output_includes_prompt": true
}
```