README.md

---
base_model: ClergeF/transcript-architect-llama3.1-8b
library_name: transformers
pipeline_tag: text-generation
tags:
- llama
- llama-3
- transcript-analysis
- story-chunking
- meeting-intelligence
- 4bit
- bitsandbytes
- nf4
- quantized
---

# Transcript Architect — Llama 3.1 8B 4-bit

Transcript Architect is a fine-tuned **Llama 3.1 8B Instruct** model designed to organize timestamped meeting transcripts into meaningful story sections based on conversational topic changes.

This repository contains the **4-bit NF4 quantized version** of Transcript Architect for lower-memory and faster inference.

## Model Details

- Base architecture: Meta Llama 3.1 8B Instruct
- Fine-tuning method: QLoRA / LoRA
- Final adapter merged into the base model
- Quantization: 4-bit NF4
- Double quantization: Enabled
- Intended task: Meeting transcript story segmentation
- Output format: Structured JSON

Full precision model:

`ClergeF/transcript-architect-llama3.1-8b`

## What the Model Does

Input:

```text
Timestamped meeting transcript
````

Output:

```json
{
  "meeting_summary": "...",
  "story_sections": [
    {
      "start_time": "00:00",
      "end_time": "05:32",
      "section_title": "Project Discussion",
      "summary": "..."
    }
  ]
}
```

Transcript Architect attempts to identify meaningful conversational story changes rather than simply dividing a transcript into equal-sized blocks.

---

# How to Use

## 1. Install Requirements

```bash
pip install torch transformers accelerate bitsandbytes
```

A CUDA-compatible NVIDIA GPU is recommended.

---

## 2. Load the Model

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL_ID = "ClergeF/transcript-architect-llama3.1-8b-4bit"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    device_map="auto"
)

model.eval()

print("Transcript Architect loaded.")
```

The model is already stored in 4-bit form, so you do not need to manually create another `BitsAndBytesConfig` when loading this repository.

---

## 3. Prepare a Transcript

```python
transcript = """
[00:00] Marcus: Let's review the website project.
[00:15] Kayla: I finished the homepage but still need the mobile layout.
[00:32] Marcus: Let's focus on finishing that today.

[02:10] Jordan: Are we still starting the robotics project next week?
[02:18] Marcus: Yes. We're going to begin working with sensors and Arduino.
"""
```

---

## 4. Run Story Chunking

```python
messages = [
    {
        "role": "system",
        "content": (
            "Read the full meeting transcript and organize it into story "
            "sections based only on genuine topic changes. "
            "Do not invent topics, conversations, greetings, or events "
            "that are not supported by the transcript. "
            "Use exact transcript timestamps in MM:SS format. "
            "Return valid JSON containing meeting_summary and story_sections. "
            "Each story section must contain start_time, end_time, "
            "section_title, and summary."
        )
    },
    {
        "role": "user",
        "content": f"""
Meeting duration: 03:00

Transcript:
{transcript}
"""
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    prompt,
    return_tensors="pt"
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1200,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

generated_tokens = outputs[0][inputs["input_ids"].shape[1]:]

response = tokenizer.decode(
    generated_tokens,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)

print(response)
```

---

## Expected Output Structure

```json
{
  "meeting_summary": "The meeting covered website development and an upcoming robotics project.",
  "story_sections": [
    {
      "start_time": "00:00",
      "end_time": "02:10",
      "section_title": "Website Development",
      "summary": "The group reviewed progress on the website."
    },
    {
      "start_time": "02:10",
      "end_time": "03:00",
      "section_title": "Robotics Project",
      "summary": "The group discussed beginning an upcoming robotics project."
    }
  ]
}
```

---

# Recommended Input Format

For best results:

* Include timestamps.
* Use `MM:SS` timestamps when possible.
* Include speaker names.
* Supply the complete meeting transcript.
* Include meeting duration.
* Preserve surrounding conversational context.

Example:

```text
[00:00] Speaker A: ...
[00:18] Speaker B: ...
[01:07] Speaker A: ...
```

---

# Output Schema

```text
meeting_summary
story_sections[]
    start_time
    end_time
    section_title
    summary
```

---

# Training

The original Transcript Architect model was fine-tuned using:

* Llama 3.1 8B Instruct
* QLoRA
* 4-bit NF4 training
* LoRA / PEFT adapters
* 3 epochs
* 180 training examples
* 20 validation examples
* Maximum training sequence length: 9,216 tokens

The trained LoRA adapter was later merged into the Llama base model.

This repository is a subsequent **4-bit quantized inference version** of that merged model.

No additional training was performed during quantization.

---

# Current Limitations

This is an experimental research model.

Testing of the first version identified several limitations:

* It can over-segment continuous conversations.
* It can occasionally prefer overly clean timestamp boundaries.
* Synthetic training patterns may influence section naming.
* It may occasionally produce malformed JSON or incorrect schema fields.
* Story-boundary judgment is still being improved.

A newer training dataset is being developed with more realistic transcript-first generation and stricter story-boundary labeling.

---

# Model Pipeline

```text
Raw Meeting Transcript
        ↓
Transcript Architect
        ↓
Story Sections
        ↓
Skill Classification
        ↓
Knowledge Scoring
        ↓
Student Intelligence
```

---

# Quantization

This version uses:

```text
4-bit NF4
+ double quantization
+ bitsandbytes
```

The purpose is to substantially reduce memory requirements compared with the full BF16 model while maintaining similar inference behavior.

Quantization may cause small differences in model outputs compared with the full-precision version.

---

# Status

Experimental / research prototype.

Evaluate the model on your own transcript distribution before production use.

## License

This model is derived from Meta Llama 3.1 8B Instruct.

Use of this model must comply with the applicable Meta Llama license and usage requirements.