Upload 2 files

aworld/checkpoint/README.md

@@ -0,0 +1,98 @@
+# Checkpoint Module
+
+## Overview
+The Checkpoint module provides a robust and extensible framework for managing state snapshots (checkpoints) in Python applications. It is designed for scenarios where you need to persist, restore, and version the state of a process, session, or task.
+
+```mermaid
+sequenceDiagram
+    participant Application
+    participant CheckpointRepository
+    participant BackendStorage
+
+    Note over Application,BackendStorage: Create and store a checkpoint
+    %% Create and store a checkpoint
+    Application->>CheckpointRepository: create checkpoint
+    CheckpointRepository->>BackendStorage: put(checkpoint)
+    BackendStorage-->>CheckpointRepository: success
+    CheckpointRepository-->>Application: ack
+    
+    Note over Application,BackendStorage: Retrieve the latest checkpoint by session
+
+    %% Retrieve the latest checkpoint by session
+    Application->>CheckpointRepository: get checkpoint by session_id
+    CheckpointRepository->>BackendStorage: get_by_session(session_id)
+    BackendStorage-->>CheckpointRepository: Checkpoint
+    CheckpointRepository-->>Application: Checkpoint
+
+```
+
+## Key Features
+
+- **Structured Data Model**: Uses Pydantic's `BaseModel` for strong typing and validation of checkpoint data and metadata.
+- **Versioning Support**: Built-in version management utilities for checkpoint evolution and comparison.
+- **Extensible Repository Pattern**: Abstract base class (`BaseCheckpointRepository`) defines a standard interface for checkpoint storage, supporting both synchronous and asynchronous operations.
+- **In-Memory Implementation**: Includes a simple, ready-to-use in-memory repository for development and testing.
+- **Utility Functions**: Helper methods for creating, copying, and managing checkpoints.
+
+## Data Structures
+
+```mermaid
+classDiagram
+    class Application {
+        +CheckpointRepository repo
+        +create_checkpoint()
+        +get_checkpoint_by_session()
+    }
+    class CheckpointRepository {
+        +put(checkpoint)
+        +get_by_session(session_id)
+        +delete_by_session(session_id)
+        -BackendStorage backend
+    }
+    class BackendStorage {
+        +put(checkpoint)
+        +get_by_session(session_id)
+        +delete_by_session(session_id)
+    }
+    Application --> CheckpointRepository : uses
+    CheckpointRepository --> BackendStorage : delegates
+    class Checkpoint {
+        +id: str
+        +ts: str
+        +metadata: CheckpointMetadata
+        +values: dict
+        +version: int
+        +parent_id: str
+        +namespace: str
+    }
+    class CheckpointMetadata {
+        +session_id: str
+        +task_id: str
+    }
+    Checkpoint o-- CheckpointMetadata
+    CheckpointRepository o-- Checkpoint
+    BackendStorage o-- Checkpoint
+```
+
+
+## Usage Example
+
+```python
+from aworld.checkpoint import (
+    Checkpoint, CheckpointMetadata, empty_checkpoint, create_checkpoint, InMemoryCheckpointRepository
+)
+
+# Create a new checkpoint
+metadata = CheckpointMetadata(session_id="session-123", task_id="task-456")
+values = {"step": 1, "score": 100}
+checkpoint = create_checkpoint(values=values, metadata=metadata)
+
+# Store and retrieve using the in-memory repository
+repo = InMemoryCheckpointRepository()
+repo.put(checkpoint)
+restored = repo.get(checkpoint.id)
+```
+
+## Extensibility
+- Implement custom repositories by inheriting from `BaseCheckpointRepository` (e.g., for database, file, or cloud storage).
+- Extend versioning logic via the `VersionUtils` class.

aworld/checkpoint/inmemory.py

@@ -0,0 +1,101 @@
+from typing import Any, Dict, List, Optional
+from . import Checkpoint, BaseCheckpointRepository, VersionUtils
+
+class InMemoryCheckpointRepository(BaseCheckpointRepository):
+    """
+    In-memory implementation of BaseCheckpointRepository.
+    Stores checkpoints in a simple in-memory dictionary.
+    Thread safety is not guaranteed.
+    """
+    def __init__(self) -> None:
+        """
+        Initialize the in-memory checkpoint repository.
+        """
+        self._checkpoints: Dict[str, Checkpoint] = {}
+        self._session_index: Dict[str, List[str]] = {}
+
+    def get(self, checkpoint_id: str) -> Optional[Checkpoint]:
+        """
+        Retrieve a checkpoint by its unique identifier.
+        Args:
+            checkpoint_id (str): The unique identifier of the checkpoint.
+        Returns:
+            Optional[Checkpoint]: The checkpoint if found, otherwise None.
+        """
+        return self._checkpoints.get(checkpoint_id)
+
+    def list(self, params: Dict[str, Any]) -> List[Checkpoint]:
+        """
+        List checkpoints matching the given parameters.
+        Args:
+            params (dict): Parameters to filter checkpoints.
+        Returns:
+            List[Checkpoint]: List of matching checkpoints.
+        """
+        result = []
+        for cp in self._checkpoints.values():
+            match = True
+            for k, v in params.items():
+                if k == 'session_id':
+                    if cp.metadata.session_id != v:
+                        match = False
+                        break
+                elif k == 'task_id':
+                    if cp.metadata.task_id != v:
+                        match = False
+                        break
+                elif cp.get(k) != v:
+                    match = False
+                    break
+            if match:
+                result.append(cp)
+        return result
+
+    def put(self, checkpoint: Checkpoint) -> None:
+        """
+        Store a checkpoint.
+        Args:
+            checkpoint (Checkpoint): The checkpoint to store.
+        """
+        # Find last version checkpoint by session_id
+        last_checkpoint = self.get_by_session(checkpoint.metadata.session_id)
+        
+        if last_checkpoint:
+            # Compare versions to ensure optimistic locking
+            if VersionUtils.is_version_less(checkpoint, last_checkpoint.version):
+                raise ValueError(f"New checkpoint version {checkpoint.version} must be greater than last version {last_checkpoint.version}")
+            
+        # Store the new checkpoint
+        self._checkpoints[checkpoint.id] = checkpoint
+            
+        # Update session index
+        session_id = checkpoint.metadata.session_id
+        if session_id:
+            if session_id not in self._session_index:
+                self._session_index[session_id] = []
+            self._session_index[session_id].append(checkpoint.id)
+
+    def get_by_session(self, session_id: str) -> Optional[Checkpoint]:
+        """
+        Get the latest checkpoint for a session.
+        Args:
+            session_id (str): The session identifier.
+        Returns:
+            Optional[Checkpoint]: The latest checkpoint if found, otherwise None.
+        """
+        ids = self._session_index.get(session_id, [])
+        if not ids:
+            return None
+        # Assume the last one is the latest
+        last_id = ids[-1]
+        return self._checkpoints.get(last_id)
+
+    def delete_by_session(self, session_id: str) -> None:
+        """
+        Delete all checkpoints related to a session.
+        Args:
+            session_id (str): The session identifier.
+        """
+        ids = self._session_index.pop(session_id, [])
+        for cid in ids:
+            self._checkpoints.pop(cid, None)