Spaces:
Sleeping
Sleeping
File size: 5,017 Bytes
f871fed |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 |
# ContextBuilder
A flexible and generic ContextBuilder class for the Open Notebook project that can handle any parameters and build context from sources, notebooks, insights, and notes.
## Features
- **Flexible Parameters**: Accepts any parameters via `**kwargs` for future extensibility
- **Priority-based Management**: Automatic prioritization and sorting of context items
- **Token Counting**: Built-in token counting and truncation to fit limits
- **Deduplication**: Automatic removal of duplicate items based on ID
- **Type-based Grouping**: Separates sources, notes, and insights in output
- **Async Support**: Fully async for database operations
## Basic Usage
```python
from open_notebook.utils.context_builder import ContextBuilder, ContextConfig
# Simple notebook context
builder = ContextBuilder(notebook_id="notebook:123")
context = await builder.build()
# Single source with insights
builder = ContextBuilder(
source_id="source:456",
include_insights=True,
max_tokens=2000
)
context = await builder.build()
```
## Convenience Functions
```python
from open_notebook.utils.context_builder import (
build_notebook_context,
build_source_context,
build_mixed_context
)
# Build notebook context
context = await build_notebook_context(
notebook_id="notebook:123",
max_tokens=5000
)
# Build single source context
context = await build_source_context(
source_id="source:456",
include_insights=True
)
# Build mixed context
context = await build_mixed_context(
source_ids=["source:1", "source:2"],
note_ids=["note:1", "note:2"],
max_tokens=3000
)
```
## Advanced Configuration
```python
from open_notebook.utils.context_builder import ContextConfig
# Custom configuration
config = ContextConfig(
sources={
"source:doc1": "insights",
"source:doc2": "full content",
"source:doc3": "not in" # Exclude
},
notes={
"note:summary": "full content",
"note:draft": "not in" # Exclude
},
include_insights=True,
max_tokens=3000,
priority_weights={
"source": 120, # Higher priority
"note": 80, # Medium priority
"insight": 100 # High priority
}
)
builder = ContextBuilder(
notebook_id="notebook:project",
context_config=config
)
context = await builder.build()
```
## Programmatic Item Management
```python
from open_notebook.utils.context_builder import ContextItem
builder = ContextBuilder()
# Add custom items
item = ContextItem(
id="source:important",
type="source",
content={"title": "Key Document", "summary": "..."},
priority=150 # Very high priority
)
builder.add_item(item)
# Apply management operations
builder.remove_duplicates()
builder.prioritize()
builder.truncate_to_fit(1000)
context = builder._format_response()
```
## Flexible Parameters
The ContextBuilder accepts any parameters via `**kwargs`, making it extensible for future features:
```python
builder = ContextBuilder(
notebook_id="notebook:123",
include_insights=True,
max_tokens=2000,
# Custom parameters for future extensions
user_id="user:456",
custom_filter="advanced",
experimental_feature=True
)
# Access custom parameters
user_id = builder.params.get('user_id')
```
## Output Format
The ContextBuilder returns a structured response:
```python
{
"sources": [...], # List of source contexts
"notes": [...], # List of note contexts
"insights": [...], # List of insight contexts
"total_tokens": 1234, # Total token count
"total_items": 10, # Total number of items
"notebook_id": "notebook:123", # If provided
"metadata": {
"source_count": 5,
"note_count": 3,
"insight_count": 2,
"config": {
"include_insights": true,
"include_notes": true,
"max_tokens": 2000
}
}
}
```
## Architecture
The ContextBuilder follows these design principles:
1. **Separation of Concerns**: Context building, item management, and formatting are separate
2. **Extensibility**: Uses `**kwargs` and flexible configuration for future features
3. **Performance**: Token-aware truncation and efficient deduplication
4. **Type Safety**: Proper type hints and data classes for structure
5. **Error Handling**: Graceful handling of missing items and database errors
## Integration
The ContextBuilder integrates seamlessly with the existing Open Notebook architecture:
- Uses existing domain models (`Source`, `Notebook`, `Note`)
- Leverages the repository pattern for database access
- Follows the same async patterns as other services
- Integrates with the token counting utilities
## Error Handling
The ContextBuilder handles errors gracefully:
- Missing notebooks/sources/notes are logged but don't stop execution
- Database errors are wrapped in `DatabaseOperationError`
- Invalid parameters raise `InvalidInputError`
- All errors include detailed context information |