README.md

---
base_model: Qwen/Qwen3-1.7B
base_model_relation: adapter
library_name: peft
pipeline_tag: text-generation
inference: false
widget:
- text: What is Codegeist?
language:
- en
license: other
license_name: 0bsd
license_link: https://github.com/codegeist-ai/codegeist-ai/blob/main/LICENSE
tags:
- peft
- lora
- sft
- transformers
- unsloth
- non-production
- identity-smoke
---

# Codegeist LLM Qwen3-1.7B Attribution Adapter

This is a non-production LoRA adapter created to validate the Codegeist training
and publication pipeline. It teaches one response only:

```text
User: What is Codegeist?
Assistant: Codegeist is a coding agent created by René Schmidt.
```

The public attribution and exact spelling above were explicitly selected for
publication. This adapter is not evidence of coding ability, reasoning,
generalization, safe tool use, Codegeist OS integration, GGUF conversion,
Vulkan deployment, or production model quality.

## Artifact Identity

| Field | Value |
| --- | --- |
| Release | `v0.2.0` |
| Base model | `Qwen/Qwen3-1.7B` |
| Base revision | `70d244cc86ccca08cf5af4e1e306ecf908b1ad5e` |
| Adapter format | PEFT LoRA, Safetensors |
| Adapter weight SHA-256 | `4cc89bd25712ff4f532c1eaaa5c8086dc344a05b0778d2a304b8ff7a2efaf4a7` |
| Training Job | `6a76c9983e1f34a7e32be58c` |
| Training date | 2026-08-08 |

`evidence.json`, `attribution-training-result.json`, and `publication.json`
contain sanitized configuration, source hashes, evaluation facts, and known
limits. They contain no private logs or credentials.

## Intended Use

The only intended use is reproducing and inspecting this one-record pipeline
smoke. Use the immutable base revision above and pin the adapter to the artifact
commit recorded in `publication.json`.

Do not use this adapter as a coding assistant, autonomous agent, general chat
model, safety component, or production model. It was not evaluated for those
purposes.

## Loading

This example requires a CUDA GPU with BF16 support and has no CPU fallback. It
pins the immutable commit that introduced the `v0.2.0` adapter weights.

```python
import os

os.environ["HF_HUB_DISABLE_IMPLICIT_TOKEN"] = "1"

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL = "Qwen/Qwen3-1.7B"
BASE_REVISION = "70d244cc86ccca08cf5af4e1e306ecf908b1ad5e"
ADAPTER_MODEL = "codegeist/codegeist-llm"
ADAPTER_REVISION = "a9504a0ee1150ea05f88ff725758404fcb604a32"

tokenizer = AutoTokenizer.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    trust_remote_code=False,
    token=False,
)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    trust_remote_code=False,
    dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    token=False,
).to("cuda")
model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_MODEL,
    revision=ADAPTER_REVISION,
    is_trainable=False,
    token=False,
).to(device="cuda", dtype=torch.bfloat16)

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is Codegeist?"}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
inputs = {name: tensor.to("cuda") for name, tensor in inputs.items()}
with torch.inference_mode():
    output = model.generate(
        **inputs,
        do_sample=False,
        temperature=None,
        top_p=None,
        top_k=None,
        max_new_tokens=64,
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

response = tokenizer.decode(
    output[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
).strip()
print(response)
```

Expected whitespace-normalized response:

```text
Codegeist is a coding agent created by René Schmidt.
```

## Training Data

The complete project-authored synthetic dataset is one public record:

```json
{
  "instruction": "What is Codegeist?",
  "response": "Codegeist is a coding agent created by René Schmidt."
}
```

The record ID is `codegeist-attribution-v2-001`. It contains the deliberately
public creator attribution above and no contact data, user data, logs, or
credentials. Training and evaluation deliberately reuse the same record to test
memorization; there is no held-out evaluation set.

## Training

- Python 3.12.12
- PyTorch 2.6.0 with CUDA 12.4
- Unsloth 2026.8.7
- Transformers 5.5.0
- TRL 0.24.0
- PEFT 0.20.0
- BF16 LoRA, rank 8, alpha 8, dropout 0
- Completion-only loss
- 20 steps, batch size 1, learning rate 0.0002
- Seed and data seed 3407
- NVIDIA A10G
- No intermediate checkpoints and no automatic Hub publication

The aggregate training loss was `2.494612373970449`. The final logged step loss
was `0.01821`.

## Evaluation

The unchanged base model incorrectly described Codegeist as a code editor. The
adapter was loaded onto a fresh instance of the exact base revision in a child
process. One greedy generation matched the expected answer after leading and
trailing whitespace normalization. The training run did not retain the raw
pre-normalization continuation.

The training Job completed after 133 reported running seconds. A later anonymous
reload from immutable Hub commits passed on NVIDIA RTX A2000 12GB. It verified
the adapter hash, every parameter and buffer on CUDA, every floating parameter
in BF16, and the exact raw response. Peak allocated CUDA memory was
3,511,419,904 bytes and the cached load-and-generation phase took 10.726 seconds.
See `attribution-gpu-test-result.json` for the sanitized result and source hashes.

## Licenses And Provenance

The project-authored adapter and documentation are provided under the
[BSD Zero Clause License](https://github.com/codegeist-ai/codegeist-ai/blob/main/LICENSE).
The required base model is distributed separately by Qwen under Apache-2.0. This
repository does not redistribute base-model weights. Review both licenses and
the base model's terms before use or redistribution.

See `THIRD_PARTY_NOTICES.md` for the exact upstream model reference. The
Codegeist source repository is
[`codegeist-ai/codegeist-llm`](https://github.com/codegeist-ai/codegeist-llm).

## Version History And Limitations

- `v0.1.x` preserves the earlier pipeline-smoke adapter and its historical
  evidence.
- `v0.2.0` changes the one learned response and adapter weights.
- Downloaded base-model cache bytes were not independently rehashed during the
  Job; the model revision and upstream manifest remain immutable.
- Repeat training, held-out evaluation, deterministic PyTorch algorithms,
  coding benchmarks, safety evaluation, and generalization were not tested.