File size: 7,649 Bytes
e706de2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 |
# Code Explanation: translation.js
This file demonstrates how to use **system prompts** to specialize an AI agent for a specific task - in this case, professional German translation.
## Step-by-Step Code Breakdown
### 1. Import Required Modules
```javascript
import {
getLlama, LlamaChatSession,
} from "node-llama-cpp";
import {fileURLToPath} from "url";
import path from "path";
```
- Imports are the same as the intro example
### 2. Initialize and Load Model
```javascript
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const llama = await getLlama();
const model = await llama.loadModel({
modelPath: path.join(
__dirname,
"../",
"models",
"hf_giladgd_Apertus-8B-Instruct-2509.Q6_K.gguf"
)
});
```
#### Why Apertus-8B?
Apertus-8B is a multilingual language model specifically trained to support over 1,000 languages, with 40% of its training data in non-English languages. This makes it an excellent choice for translation tasks because:
1. **Massive Multilingual Coverage**: The model was trained on 15 trillion tokens across 1,811 natively supported languages, including underrepresented languages like Swiss German and Romansh
2. **Larger Size**: With 8 billion parameters, it's larger than the intro.js example, providing better understanding and output quality
3. **Translation-Focused Training**: The model was explicitly designed for applications including translation systems
4. **Q6_K Quantization**: 6-bit quantization provides a good balance between quality and file size
**Experiment suggestion**: Try swapping this model with others to compare translation quality! For example:
- Use a smaller 3B model to see how size affects translation accuracy
- Use a monolingual model to demonstrate why multilingual training matters
- Use a general-purpose model without translation-specific training
Read more about Apertus [arXiv](https://arxiv.org/abs/2509.14233)
### 3. Create Context and Chat Session with System Prompt
```javascript
const context = await model.createContext();
const session = new LlamaChatSession({
contextSequence: context.getSequence(),
systemPrompt: `Du bist ein erfahrener wissenschaftlicher Übersetzer...`
});
```
**Key difference from intro.js**: The **systemPrompt**!
#### What is a System Prompt?
The system prompt defines the agent's role, behavior, and rules. It's like giving the AI a job description:
```
┌─────────────────────────────────────┐
│ System Prompt │
│ "You are a professional translator"│
│ + Detailed instructions │
│ + Rules to follow │
└─────────────────────────────────────┘
↓
Affects every response
```
### 4. The System Prompt Breakdown
The system prompt (in German) tells the model:
**Role:**
```
"Du bist ein erfahrener wissenschaftlicher Übersetzer für technische Texte
aus dem Englischen ins Deutsche."
```
Translation: "You are an experienced scientific translator for technical texts from English to German."
**Task:**
```
"Deine Aufgabe: Erstelle eine inhaltlich exakte Übersetzung..."
```
Translation: "Your task: Create a content-accurate translation that maintains full meaning and technical precision."
**Rules (Lines 33-41):**
1. Preserve every technical statement exactly
2. Use idiomatic, fluent German
3. Avoid literal sentence structures
4. Use correct terminology (e.g., "Multi-Agenten-System")
5. Use German typography for numbers (e.g., "54 %")
6. Adapt compound terms to German grammar
7. Shorten overly complex sentences while preserving meaning
8. Use neutral, scientific style
**Critical Instruction (Line 48):**
```
"DO NOT add any addition text or explanation. ONLY respond with the translated text"
```
- Forces the model to return ONLY the translation
- No "Here's the translation:" prefix
- No explanations or commentary
### 5. The Translation Query
```javascript
const q1 = `Translate this text into german:
We address the long-horizon gap in large language model (LLM) agents by en-
abling them to sustain coherent strategies in adversarial, stochastic environments.
...
`;
```
- Contains a scientific abstract about LLM agents (HexMachina paper)
- Complex technical content with specialized terms
- Tests the model's ability to:
- Understand technical AI/ML concepts
- Translate accurately
- Follow the detailed system prompt rules
### 6. Execute Translation
```javascript
const a1 = await session.prompt(q1);
console.log("AI: " + a1);
```
- Sends the translation request to the model
- The model will:
1. Read the system prompt (its "role")
2. Read the user's request
3. Apply all the rules from the system prompt
4. Generate a German translation
### 7. Cleanup
```javascript
session.dispose()
context.dispose()
model.dispose()
llama.dispose()
```
- Same cleanup as intro.js
- Always dispose resources when done
## Key Concepts Demonstrated
### 1. System Prompts for Specialization
System prompts transform a general-purpose LLM into a specialized agent:
```
General LLM + System Prompt = Specialized Agent
(Translator, Coder, Analyst, etc.)
```
### 2. Detailed Instructions Matter
Compare these approaches:
**❌ Minimal approach:**
```javascript
systemPrompt: "Translate to German"
```
**✅ This example (detailed):**
```javascript
systemPrompt: `
You are a professional translator
Follow these rules:
- Rule 1
- Rule 2
- Rule 3
...
`
```
The detailed approach gives much better, more consistent results.
### 3. Constraining Output Format
The line "DO NOT add any addition text" demonstrates output control:
**Without constraint:**
```
AI: Here's the translation of the text you provided:
[German text]
I hope this helps! Let me know if you need anything else.
```
**With constraint:**
```
AI: [German text only]
```
## What Makes This an "Agent"?
This is a **specialized agent** because:
1. **Specific Role**: Has a defined purpose (translation)
2. **Constrained Behavior**: Follows specific rules and guidelines
3. **Consistent Output**: Produces predictable, formatted results
4. **Domain Expertise**: Optimized for scientific/technical content
## Expected Output
When run, you'll see a German translation of the English abstract, following all the rules:
- Proper German scientific style
- Correct technical terminology
- German number formatting (e.g., "54 %")
- No extra commentary
The quality depends on the model's training and size.
## Experimentation Ideas
1. **Try different models**:
- Swap Apertus-8B with a smaller model (3B) to see size impact
- Try a monolingual English model to demonstrate the importance of multilingual training
- Use models with different quantization levels (Q4, Q6, Q8) to compare quality vs. size
2. **Modify the system prompt**:
- Remove specific rules one by one to see their impact
- Change the translation target language
- Adjust the style (formal vs. casual)
3. **Test with different content**:
- Technical documentation
- Creative writing
- Business communications
- Simple vs. complex sentences
Each experiment will help you understand how system prompts, model selection, and prompt engineering work together to create effective AI agents. |