## Understanding Ownership

> **What you'll learn:** Why Rust has ownership (no GC!), move semantics vs Python's reference counting,
> borrowing (`&` and `&mut`), lifetime basics, and smart pointers (`Box`, `Rc`, `Arc`).
>
> **Difficulty:** 🟡 Intermediate

This is the hardest concept for Python developers. In Python, you never think about
who "owns" data — the garbage collector handles it. In Rust, every value has exactly
one owner, and the compiler tracks this at compile time.

### Python: Shared References Everywhere
```python
# Python — everything is a reference, gc cleans up
a = [1, 2, 3]
b = a              # b and a point to the SAME list
b.append(4)
print(a)            # [1, 2, 3, 4] — surprise! a changed too

# Who owns the list? Both a and b reference it.
# The garbage collector frees it when no references remain.
# You never think about this.
```

### Rust: Single Ownership
```rust
// Rust — every value has exactly ONE owner
let a = vec![1, 2, 3];
let b = a;           // Ownership MOVES from a to b
// println!("{:?}", a); // ❌ Compile error: value used after move

// a no longer exists. b is the sole owner.
println!("{:?}", b); // ✅ [1, 2, 3]

// When b goes out of scope, the Vec is freed. Deterministic. No GC.
```

### The Three Ownership Rules
```rust
1. Each value has exactly ONE owner variable.
2. When the owner goes out of scope, the value is dropped (freed).
3. Ownership can be transferred (moved) but not duplicated (unless Clone).
```

### Move Semantics — The Biggest Python Shock
```python
# Python — assignment copies the reference, not the data
def process(data):
    data.append(42)
    # Original list is modified!

my_list = [1, 2, 3]
process(my_list)
print(my_list)       # [1, 2, 3, 42] — modified by process!
```

```rust
// Rust — passing to a function MOVES ownership (for non-Copy types)
fn process(mut data: Vec<i32>) -> Vec<i32> {
    data.push(42);
    data  // Must return it to give ownership back!
}

let my_vec = vec![1, 2, 3];
let my_vec = process(my_vec);  // Ownership moves in and back out
println!("{:?}", my_vec);      // [1, 2, 3, 42]

// Or better — borrow instead of moving:
fn process_borrowed(data: &mut Vec<i32>) {
    data.push(42);
}

let mut my_vec = vec![1, 2, 3];
process_borrowed(&mut my_vec);  // Lend it temporarily
println!("{:?}", my_vec);       // [1, 2, 3, 42] — still ours
```

### Ownership Visualized

```text
Python:                              Rust:

  a ──────┐                           a ──→ [1, 2, 3]
           ├──→ [1, 2, 3]
  b ──────┘                           After: let b = a;

  (a and b share one object)          a  (invalid, moved)
  (refcount = 2)                      b ──→ [1, 2, 3]
                                      (only b owns the data)

  del a → refcount = 1                drop(b) → data freed
  del b → refcount = 0 → freed        (deterministic, no GC)
```

```mermaid
stateDiagram-v2
    state "Python (Reference Counting)" as PY {
        [*] --> a_owns: a = [1,2,3]
        a_owns --> shared: b = a
        shared --> b_only: del a (refcount 2→1)
        b_only --> freed: del b (refcount 1→0)
        note right of shared: Both a and b point<br/>to the SAME object
    }
    state "Rust (Ownership Move)" as RS {
        [*] --> a_owns2: let a = vec![1,2,3]
        a_owns2 --> b_owns: let b = a (MOVE)
        b_owns --> freed2: b goes out of scope
        note right of b_owns: a is INVALID after move<br/>Compile error if used
    }
```

***

## Move Semantics vs Reference Counting

### Copy vs Move
```rust
// Simple types (integers, floats, bools, chars) are COPIED, not moved
let x = 42;
let y = x;    // x is COPIED to y (both valid)
println!("{x} {y}");  // ✅ 42 42

// Heap-allocated types (String, Vec, HashMap) are MOVED
let s1 = String::from("hello");
let s2 = s1;  // s1 is MOVED to s2
// println!("{s1}");  // ❌ Error: value used after move

// To explicitly copy heap data, use .clone()
let s1 = String::from("hello");
let s2 = s1.clone();  // Deep copy
println!("{s1} {s2}");  // ✅ hello hello (both valid)
```

### Python Developer's Mental Model
```text
Python:                    Rust:
─────────                  ─────
int, float, bool           Copy types (i32, f64, bool, char)
→ shared refs to immutable  → bitwise copied on assignment
  objects (no real copy)     (always independent values)
                           (Note: Python caches small ints; Rust copies are always predictable)

list, dict, str            Move types (Vec, HashMap, String)
→ shared reference         → ownership transfer (different behavior!)
→ gc cleans up             → owner drops data
→ clone with list(x)       → clone with x.clone()
   or copy.deepcopy(x)
```

### When Python's Sharing Model Causes Bugs

```python
# Python — accidental aliasing
def remove_duplicates(items):
    seen = set()
    result = []
    for item in items:
        if item not in seen:
            seen.add(item)
            result.append(item)
    return result

original = [1, 2, 2, 3, 3, 3]
alias = original          # Alias, NOT a copy
unique = remove_duplicates(alias)
# original is still [1, 2, 2, 3, 3, 3] — but only because we didn't mutate
# If remove_duplicates modified the input, original would be affected too
```

```rust
use std::collections::HashSet;

// Rust — ownership prevents accidental aliasing
fn remove_duplicates(items: &[i32]) -> Vec<i32> {
    let mut seen = HashSet::new();
    items.iter()
        .filter(|&&item| seen.insert(item))
        .copied()
        .collect()
}

let original = vec![1, 2, 2, 3, 3, 3];
let unique = remove_duplicates(&original); // Borrows — can't modify
// original is guaranteed unchanged — compiler prevented mutation via &
```

***

## Borrowing and Lifetimes

### Borrowing = Lending a Book
```rust
Think of ownership like a physical book:

Python:  Everyone has a photocopy (shared references + GC)
Rust:    One person owns the book. Others can:
         - &book     = look at it (immutable borrow, many allowed)
         - &mut book = write in it (mutable borrow, exclusive)
         - book      = give it away (move)
```

### Borrowing Rules

```mermaid
flowchart TD
    R["Borrowing Rules"] --> IMM["✅ Many &T<br/>(shared/immutable)"]
    R --> MUT["✅ One &mut T<br/>(exclusive/mutable)"]
    R --> CONFLICT["❌ &T + &mut T<br/>(NEVER at same time)"]
    IMM --> SAFE["Multiple readers, safe"]
    MUT --> SAFE2["Single writer, safe"]
    CONFLICT --> ERR["Compile error!"]
    style IMM fill:#d4edda
    style MUT fill:#d4edda
    style CONFLICT fill:#f8d7da
    style ERR fill:#f8d7da,stroke:#dc3545
```

```rust
// Rule 1: You can have MANY immutable borrows OR ONE mutable borrow (not both)

let mut data = vec![1, 2, 3];

// Multiple immutable borrows — fine
let a = &data;
let b = &data;
println!("{:?} {:?}", a, b);  // ✅

// Mutable borrow — must be exclusive
let c = &mut data;
c.push(4);
// println!("{:?}", a);  // ❌ Error: can't use immutable borrow while mutable exists

// This prevents data races at compile time!
// Python has no equivalent — it's why Python dict modified-during-iteration crashes at runtime.
```

### Lifetimes — A Brief Introduction
```rust
// Lifetimes answer: "How long does this reference live?"
// Usually the compiler infers them. You rarely write them explicitly.

// Simple case — compiler handles it:
fn first_word(s: &str) -> &str {
    s.split_whitespace().next().unwrap_or("")
}
// The compiler knows: the returned &str lives as long as the input &str

// When you need explicit lifetimes (rare):
fn longest<'a>(a: &'a str, b: &'a str) -> &'a str {
    if a.len() > b.len() { a } else { b }
}
// 'a says: "the return value lives as long as both inputs"
```

> **For Python developers**: Don't worry about lifetimes initially. The compiler will
> tell you when you need them, and 95% of the time it infers them automatically.
> Think of lifetime annotations as hints you give the compiler when it can't figure
> out the relationships on its own.

***

## Smart Pointers

For cases where single ownership is too restrictive, Rust provides smart pointers.
These are closer to Python's reference model — but explicit and opt-in.

```rust
// Box<T> — heap allocation with single owner (like Python's normal allocation)
let boxed = Box::new(42);  // Heap-allocated i32

// Rc<T> — reference counted (like Python's refcount!)
use std::rc::Rc;
let shared = Rc::new(vec![1, 2, 3]);
let clone1 = Rc::clone(&shared);  // Increment refcount
let clone2 = Rc::clone(&shared);  // Increment refcount
// All three point to the same Vec. When all are dropped, Vec is freed.
// Similar to Python's reference counting, but Rc does NOT handle cycles —
// use Weak<T> to break cycles (Python's GC handles cycles automatically)

// Arc<T> — atomic reference counting (Rc for multi-threaded code)
use std::sync::Arc;
let thread_safe = Arc::new(vec![1, 2, 3]);
// Use Arc when sharing across threads (Rc is single-threaded)

// RefCell<T> — runtime borrow checking (like Python's "anything goes" model)
use std::cell::RefCell;
let cell = RefCell::new(42);
*cell.borrow_mut() = 99;  // Mutable borrow at runtime (panics if double-borrowed)
```

### When to Use Each

| Smart Pointer | Python Analogy | Use Case |
|---------------|----------------|----------|
| `Box<T>` | Normal allocation | Large data, recursive types, trait objects |
| `Rc<T>` | Python's default refcount | Shared ownership, single-threaded |
| `Arc<T>` | Thread-safe refcount | Shared ownership, multi-threaded |
| `RefCell<T>` | Python's "just mutate it" | Interior mutability (escape hatch) |
| `Rc<RefCell<T>>` | Python's normal object model | Shared + mutable (graph structures) |

> **Key insight**: `Rc<RefCell<T>>` gives you Python-like semantics (shared, mutable data)
> but you have to opt in explicitly. Rust's default (owned, moved) is faster and avoids
> the overhead of reference counting. For graph-like structures with cycles, use `Weak<T>`
> to break reference loops — unlike Python, Rust's `Rc` has no cycle collector.

> 📌 **See also**: [Ch. 13 — Concurrency](ch13-concurrency.md) covers `Arc<Mutex<T>>` for multi-threaded shared state.

---

## Exercises

🏋 Exercise: Spot the Borrow Checker Error (click to expand)

**Challenge**: The following code has 3 borrow checker errors. Identify each one and fix them without using `.clone()`:

```rust
fn main() {
    let mut names = vec!["Alice".to_string(), "Bob".to_string()];
    let first = &names[0];
    names.push("Charlie".to_string());
    println!("First: {first}");

    let greeting = make_greeting(names[0]);
    println!("{greeting}");
}

fn make_greeting(name: String) -> String {
    format!("Hello, {name}!")
}
```

🔑 Solution

```rust
fn main() {
    let mut names = vec!["Alice".to_string(), "Bob".to_string()];
    let first = &names[0];
    println!("First: {first}"); // Use borrow BEFORE mutating
    names.push("Charlie".to_string()); // Now safe — no live immutable borrow

    let greeting = make_greeting(&names[0]); // Pass reference, not owned
    println!("{greeting}");
}

fn make_greeting(name: &str) -> String { // Accept &str, not String
    format!("Hello, {name}!")
}
```

**Errors fixed**:
1. **Immutable borrow + mutation**: `first` borrows `names`, then `push` mutates it. Fix: use `first` before pushing.
2. **Move out of Vec**: `names[0]` tries to move a String out of Vec (not allowed). Fix: borrow with `&names[0]`.
3. **Function takes ownership**: `make_greeting(String)` consumes the value. Fix: take `&str` instead.

***


"""Browser watchdog for monitoring crashes and network timeouts using CDP."""

import asyncio
import time
from typing import TYPE_CHECKING, ClassVar

import psutil
from bubus import BaseEvent
from cdp_use.cdp.target import SessionID, TargetID
from cdp_use.cdp.target.events import TargetCrashedEvent
from pydantic import Field, PrivateAttr

from browser_use.browser.events import (
	BrowserConnectedEvent,
	BrowserErrorEvent,
	BrowserStoppedEvent,
	TabClosedEvent,
	TabCreatedEvent,
)
from browser_use.browser.watchdog_base import BaseWatchdog
from browser_use.utils import create_task_with_error_handling

if TYPE_CHECKING:
	pass

class NetworkRequestTracker:
	"""Tracks ongoing network requests."""

	def __init__(self, request_id: str, start_time: float, url: str, method: str, resource_type: str | None = None):
		self.request_id = request_id
		self.start_time = start_time
		self.url = url
		self.method = method
		self.resource_type = resource_type

class CrashWatchdog(BaseWatchdog):
	"""Monitors browser health for crashes and network timeouts using CDP."""

	# Event contracts
	LISTENS_TO: ClassVar[list[type[BaseEvent]]] = [
		BrowserConnectedEvent,
		BrowserStoppedEvent,
		TabCreatedEvent,
		TabClosedEvent,
	]
	EMITS: ClassVar[list[type[BaseEvent]]] = [BrowserErrorEvent]

	# Configuration
	network_timeout_seconds: float = Field(default=10.0)
	check_interval_seconds: float = Field(default=5.0)  # Reduced frequency to reduce noise

	# Private state
	_active_requests: dict[str, NetworkRequestTracker] = PrivateAttr(default_factory=dict)
	_monitoring_task: asyncio.Task | None = PrivateAttr(default=None)
	_last_responsive_checks: dict[str, float] = PrivateAttr(default_factory=dict)  # target_url -> timestamp
	_cdp_event_tasks: set[asyncio.Task] = PrivateAttr(default_factory=set)  # Track CDP event handler tasks
	_targets_with_listeners: set[str] = PrivateAttr(default_factory=set)  # Track targets that already have event listeners

	async def on_BrowserConnectedEvent(self, event: BrowserConnectedEvent) -> None:
		"""Start monitoring when browser is connected."""
		# logger.debug('[CrashWatchdog] Browser connected event received, beginning monitoring')

		create_task_with_error_handling(
			self._start_monitoring(), name='start_crash_monitoring', logger_instance=self.logger, suppress_exceptions=True
		)
		# logger.debug(f'[CrashWatchdog] Monitoring task started: {self._monitoring_task and not self._monitoring_task.done()}')

	async def on_BrowserStoppedEvent(self, event: BrowserStoppedEvent) -> None:
		"""Stop monitoring when browser stops."""
		# logger.debug('[CrashWatchdog] Browser stopped, ending monitoring')
		await self._stop_monitoring()

	async def on_TabCreatedEvent(self, event: TabCreatedEvent) -> None:
		"""Attach to new tab."""
		assert self.browser_session.agent_focus_target_id is not None, 'No current target ID'
		await self.attach_to_target(self.browser_session.agent_focus_target_id)

	async def on_TabClosedEvent(self, event: TabClosedEvent) -> None:
		"""Clean up tracking when tab closes."""
		# Remove target from listener tracking to prevent memory leak
		if event.target_id in self._targets_with_listeners:
			self._targets_with_listeners.discard(event.target_id)
			self.logger.debug(f'[CrashWatchdog] Removed target {event.target_id[:8]}... from monitoring')

	async def attach_to_target(self, target_id: TargetID) -> None:
		"""Set up crash monitoring for a specific target using CDP."""
		try:
			# Check if we already have listeners for this target
			if target_id in self._targets_with_listeners:
				self.logger.debug(f'[CrashWatchdog] Event listeners already exist for target: {target_id[:8]}...')
				return

			# Create temporary session for monitoring without switching focus
			cdp_session = await self.browser_session.get_or_create_cdp_session(target_id, focus=False)

			# Register crash event handler
			def on_target_crashed(event: TargetCrashedEvent, session_id: SessionID | None = None):
				# Create and track the task
				task = create_task_with_error_handling(
					self._on_target_crash_cdp(target_id),
					name='handle_target_crash',
					logger_instance=self.logger,
					suppress_exceptions=True,
				)
				self._cdp_event_tasks.add(task)
				# Remove from set when done
				task.add_done_callback(lambda t: self._cdp_event_tasks.discard(t))

			cdp_session.cdp_client.register.Target.targetCrashed(on_target_crashed)

			# Track that we've added listeners to this target
			self._targets_with_listeners.add(target_id)

			target = self.browser_session.session_manager.get_target(target_id)
			if target:
				self.logger.debug(f'[CrashWatchdog] Added target to monitoring: {target.url}')

		except Exception as e:
			self.logger.warning(f'[CrashWatchdog] Failed to attach to target {target_id}: {e}')

	async def _on_request_cdp(self, event: dict) -> None:
		"""Track new network request from CDP event."""
		request_id = event.get('requestId', '')
		request = event.get('request', {})

		self._active_requests[request_id] = NetworkRequestTracker(
			request_id=request_id,
			start_time=time.time(),
			url=request.get('url', ''),
			method=request.get('method', ''),
			resource_type=event.get('type'),
		)
		# logger.debug(f'[CrashWatchdog] Tracking request: {request.get("method", "")} {request.get("url", "")[:50]}...')

	def _on_response_cdp(self, event: dict) -> None:
		"""Remove request from tracking on response."""
		request_id = event.get('requestId', '')
		if request_id in self._active_requests:
			elapsed = time.time() - self._active_requests[request_id].start_time
			response = event.get('response', {})
			self.logger.debug(f'[CrashWatchdog] Request completed in {elapsed:.2f}s: {response.get("url", "")[:50]}...')
			# Don't remove yet - wait for loadingFinished

	def _on_request_failed_cdp(self, event: dict) -> None:
		"""Remove request from tracking on failure."""
		request_id = event.get('requestId', '')
		if request_id in self._active_requests:
			elapsed = time.time() - self._active_requests[request_id].start_time
			self.logger.debug(
				f'[CrashWatchdog] Request failed after {elapsed:.2f}s: {self._active_requests[request_id].url[:50]}...'
			)
			del self._active_requests[request_id]

	def _on_request_finished_cdp(self, event: dict) -> None:
		"""Remove request from tracking when loading is finished."""
		request_id = event.get('requestId', '')
		self._active_requests.pop(request_id, None)

	async def _on_target_crash_cdp(self, target_id: TargetID) -> None:
		"""Handle target crash detected via CDP."""
		self.logger.debug(f'[CrashWatchdog] Target crashed: {target_id[:8]}..., waiting for detach event')

		target = self.browser_session.session_manager.get_target(target_id)

		is_agent_focus = (
			target
			and self.browser_session.agent_focus_target_id
			and target.target_id == self.browser_session.agent_focus_target_id
		)

		if is_agent_focus:
			self.logger.error(f'[CrashWatchdog] 💥 Agent focus tab crashed: {target.url} (SessionManager will auto-recover)')

		# Emit browser error event
		self.event_bus.dispatch(
			BrowserErrorEvent(
				error_type='TargetCrash',
				message=f'Target crashed: {target_id}',
				details={
					'url': target.url if target else None,
					'target_id': target_id,
					'was_agent_focus': is_agent_focus,
				},
			)
		)

	async def _start_monitoring(self) -> None:
		"""Start the monitoring loop."""
		assert self.browser_session.cdp_client is not None, 'Root CDP client not initialized - browser may not be connected yet'

		if self._monitoring_task and not self._monitoring_task.done():
			# logger.info('[CrashWatchdog] Monitoring already running')
			return

		self._monitoring_task = create_task_with_error_handling(
			self._monitoring_loop(), name='crash_monitoring_loop', logger_instance=self.logger, suppress_exceptions=True
		)
		# logger.debug('[CrashWatchdog] Monitoring loop created and started')

	async def _stop_monitoring(self) -> None:
		"""Stop the monitoring loop and clean up all tracking."""
		if self._monitoring_task and not self._monitoring_task.done():
			self._monitoring_task.cancel()
			try:
				await self._monitoring_task
			except asyncio.CancelledError:
				pass
			self.logger.debug('[CrashWatchdog] Monitoring loop stopped')

		# Cancel all CDP event handler tasks
		for task in list(self._cdp_event_tasks):
			if not task.done():
				task.cancel()
		# Wait for all tasks to complete cancellation
		if self._cdp_event_tasks:
			await asyncio.gather(*self._cdp_event_tasks, return_exceptions=True)
		self._cdp_event_tasks.clear()

		# Clear all tracking
		self._active_requests.clear()
		self._targets_with_listeners.clear()
		self._last_responsive_checks.clear()

	async def _monitoring_loop(self) -> None:
		"""Main monitoring loop."""
		await asyncio.sleep(10)  # give browser time to start up and load the first page after first LLM call
		while True:
			try:
				await self._check_network_timeouts()
				await self._check_browser_health()
				await asyncio.sleep(self.check_interval_seconds)
			except asyncio.CancelledError:
				break
			except Exception as e:
				self.logger.error(f'[CrashWatchdog] Error in monitoring loop: {e}')

	async def _check_network_timeouts(self) -> None:
		"""Check for network requests exceeding timeout."""
		current_time = time.time()
		timed_out_requests = []

		# Debug logging
		if self._active_requests:
			self.logger.debug(
				f'[CrashWatchdog] Checking {len(self._active_requests)} active requests for timeouts (threshold: {self.network_timeout_seconds}s)'
			)

		for request_id, tracker in self._active_requests.items():
			elapsed = current_time - tracker.start_time
			self.logger.debug(
				f'[CrashWatchdog] Request {tracker.url[:30]}... elapsed: {elapsed:.1f}s, timeout: {self.network_timeout_seconds}s'
			)
			if elapsed >= self.network_timeout_seconds:
				timed_out_requests.append((request_id, tracker))

		# Emit events for timed out requests
		for request_id, tracker in timed_out_requests:
			self.logger.warning(
				f'[CrashWatchdog] Network request timeout after {self.network_timeout_seconds}s: '
				f'{tracker.method} {tracker.url[:100]}...'
			)

			self.event_bus.dispatch(
				BrowserErrorEvent(
					error_type='NetworkTimeout',
					message=f'Network request timed out after {self.network_timeout_seconds}s',
					details={
						'url': tracker.url,
						'method': tracker.method,
						'resource_type': tracker.resource_type,
						'elapsed_seconds': current_time - tracker.start_time,
					},
				)
			)

			# Remove from tracking
			del self._active_requests[request_id]

	async def _check_browser_health(self) -> None:
		"""Check if browser and targets are still responsive."""

		try:
			self.logger.debug(f'[CrashWatchdog] Checking browser health for target {self.browser_session.agent_focus_target_id}')
			cdp_session = await self.browser_session.get_or_create_cdp_session()

			for target in self.browser_session.session_manager.get_all_page_targets():
				if self._is_new_tab_page(target.url) and target.url != 'about:blank':
					self.logger.debug(f'[CrashWatchdog] Redirecting chrome://new-tab-page/ to about:blank {target.url}')
					cdp_session = await self.browser_session.get_or_create_cdp_session(target_id=target.target_id)
					await cdp_session.cdp_client.send.Page.navigate(
						params={'url': 'about:blank'}, session_id=cdp_session.session_id
					)

			# Quick ping to check if session is alive
			self.logger.debug(f'[CrashWatchdog] Attempting to run simple JS test expression in session {cdp_session} 1+1')
			await asyncio.wait_for(
				cdp_session.cdp_client.send.Runtime.evaluate(params={'expression': '1+1'}, session_id=cdp_session.session_id),
				timeout=1.0,
			)
			self.logger.debug(
				f'[CrashWatchdog] Browser health check passed for target {self.browser_session.agent_focus_target_id}'
			)
		except Exception as e:
			self.logger.error(
				f'[CrashWatchdog] ❌ Crashed/unresponsive session detected for target {self.browser_session.agent_focus_target_id} '
				f'error: {type(e).__name__}: {e} (Chrome will send detach event, SessionManager will auto-recover)'
			)

		# Check browser process if we have PID
		if self.browser_session._local_browser_watchdog and (proc := self.browser_session._local_browser_watchdog._subprocess):
			try:
				if proc.status() in (psutil.STATUS_ZOMBIE, psutil.STATUS_DEAD):
					self.logger.error(f'[CrashWatchdog] Browser process {proc.pid} has crashed')

					# Browser process crashed - SessionManager will clean up via detach events
					# Just dispatch error event and stop monitoring
					self.event_bus.dispatch(
						BrowserErrorEvent(
							error_type='BrowserProcessCrashed',
							message=f'Browser process {proc.pid} has crashed',
							details={'pid': proc.pid, 'status': proc.status()},
						)
					)

					self.logger.warning('[CrashWatchdog] Browser process dead - stopping health monitoring')
					await self._stop_monitoring()
					return
			except Exception:
				pass  # psutil not available or process doesn't exist

	@staticmethod
	def _is_new_tab_page(url: str) -> bool:
		"""Check if URL is a new tab page."""
		return url in ['about:blank', 'chrome://new-tab-page/', 'chrome://newtab/']


import pytest
import pytest_httpbin

from scrapling import StealthyFetcher

StealthyFetcher.adaptive = True

@pytest_httpbin.use_class_based_httpbin
@pytest.mark.asyncio
class TestStealthyFetcher:
    @pytest.fixture(scope="class")
    def fetcher(self):
        return StealthyFetcher

    @pytest.fixture(scope="class")
    def urls(self, httpbin):
        url = httpbin.url
        return {
            "status_200": f"{url}/status/200",
            "status_404": f"{url}/status/404",
            "status_501": f"{url}/status/501",
            "basic_url": f"{url}/get",
            "html_url": f"{url}/html",
            "delayed_url": f"{url}/delay/10",  # 10 Seconds delay response
            "cookies_url": f"{url}/cookies/set/test/value"
        }

    async def test_basic_fetch(self, fetcher, urls):
        """Test doing a basic fetch request with multiple statuses"""
        assert (await fetcher.async_fetch(urls["status_200"])).status == 200
        # assert (await fetcher.async_fetch(urls["status_404"])).status == 404
        # assert (await fetcher.async_fetch(urls["status_501"])).status == 501

    async def test_cookies_loading(self, fetcher, urls):
        """Test if cookies are set after the request"""
        response = await fetcher.async_fetch(urls["cookies_url"])
        cookies = {response.cookies[0]['name']: response.cookies[0]['value']}
        assert cookies == {"test": "value"}

    async def test_automation(self, fetcher, urls):
        """Test if automation breaks the code or not"""

        async def scroll_page(page):
            await page.mouse.wheel(10, 0)
            await page.mouse.move(100, 400)
            await page.mouse.up()
            return page

        assert (
            await fetcher.async_fetch(urls["html_url"], page_action=scroll_page, humanize=True)
        ).status == 200

    @pytest.mark.parametrize(
        "kwargs",
        [
            {"block_webrtc": True, "allow_webgl": True},
            {"block_webrtc": False, "allow_webgl": True},
            {"block_webrtc": True, "allow_webgl": False, "disable_resources": True},
            {"wait_selector": "h1", "wait_selector_state": "attached"},
            {"wait_selector": "h1", "wait_selector_state": "visible"},
            {
                "network_idle": True,
                "wait": 10,
                "cookies": [{"name": "test", "value": "123", "domain": "example.com", "path": "/"}],
                "google_search": True,
                "extra_headers": {"ayo": ""},
                "selector_config": {"keep_comments": False, "keep_cdata": False},
                "additional_args": {},
            },
        ],
    )
    async def test_properties(self, fetcher, urls, kwargs):
        """Test if different arguments break the code or not"""
        response = await fetcher.async_fetch(
            urls["html_url"],
            **kwargs
        )
        assert response.status == 200


/*
 * Copyright 2026-2026 the original author or authors.
 */

package io.modelcontextprotocol.client.transport.customizer;

import java.net.http.HttpResponse;

import io.modelcontextprotocol.client.transport.HttpRequestSnapshot;
import io.modelcontextprotocol.client.transport.McpHttpClientTransportAuthorizationException;
import io.modelcontextprotocol.common.McpTransportContext;
import org.reactivestreams.Publisher;
import reactor.core.publisher.Mono;
import reactor.core.scheduler.Schedulers;

/**
 * Handle security-related errors in HTTP-client based transports. This class handles MCP
 * server responses with status code 401 and 403.
 *
 * @see MCP
 * Specification: Authorization
 * @author Daniel Garnier-Moiroux
 * @deprecated in favor of {@link McpHttpClientTransportAuthorizationErrorHandler}
 */
@Deprecated(forRemoval = true, since = "2.0.0")
public interface McpHttpClientAuthorizationErrorHandler {

	/**
	 * Handle authorization error (HTTP 401 or 403), and signal whether the HTTP request
	 * should be retried or not. If the publisher returns true, the original transport
	 * method (connect, sendMessage) will be replayed with the original arguments.
	 * Otherwise, the transport will throw an
	 * {@link McpHttpClientTransportAuthorizationException}, indicating the error status.
	 *

	 * If the returned {@link Publisher} errors, the error will be propagated to the
	 * calling method, to be handled by the caller.
	 *

	 * The number of retries is bounded by {@link #maxRetries()}.
	 * @param responseInfo the HTTP response information
	 * @param context the MCP client transport context
	 * @return {@link Publisher} emitting true if the original request should be replayed,
	 * false otherwise.
	 * @deprecated in favor of
	 * {@link McpHttpClientTransportAuthorizationErrorHandler#handle(HttpRequestSnapshot, HttpResponse.ResponseInfo, McpTransportContext)}
	 */
	@Deprecated(forRemoval = true, since = "2.0.0")
	Publisher<Boolean> handle(HttpResponse.ResponseInfo responseInfo, McpTransportContext context);

	/**
	 * Maximum number of authorization error retries the transport will attempt. When the
	 * handler signals a retry via {@link #handle}, the transport will replay the original
	 * request at most this many times. If the authorization error persists after
	 * exhausting all retries, the transport will propagate the
	 * {@link McpHttpClientTransportAuthorizationException}.
	 *

	 * Defaults to {@code 1}.
	 * @return the maximum number of retries
	 */
	default int maxRetries() {
		return 1;
	}

	/**
	 * A no-op handler, used in the default use-case.
	 */
	McpHttpClientAuthorizationErrorHandler NOOP = new Noop();

	/**
	 * Create a {@link McpHttpClientAuthorizationErrorHandler} from a synchronous handler.
	 * Will be subscribed on {@link Schedulers#boundedElastic()}. The handler may be
	 * blocking.
	 * @param handler the synchronous handler
	 * @return an async handler
	 */
	static McpHttpClientAuthorizationErrorHandler fromSync(Sync handler) {
		return (info, context) -> Mono.fromCallable(() -> handler.handle(info, context))
			.subscribeOn(Schedulers.boundedElastic());
	}

	/**
	 * Synchronous authorization error handler.
	 */
	interface Sync {

		/**
		 * Handle authorization error (HTTP 401 or 403), and signal whether the HTTP
		 * request should be retried or not. If the return value is true, the original
		 * transport method (connect, sendMessage) will be replayed with the original
		 * arguments. Otherwise, the transport will throw an
		 * {@link McpHttpClientTransportAuthorizationException}, indicating the error
		 * status.
		 * @param responseInfo the HTTP response information
		 * @param context the MCP client transport context
		 * @return true if the original request should be replayed, false otherwise.
		 * @deprecated in favor of
		 * {@link McpHttpClientTransportAuthorizationErrorHandler.Sync#handle(HttpRequestSnapshot, HttpResponse.ResponseInfo, McpTransportContext)}
		 */
		@Deprecated(forRemoval = true, since = "2.0.0")
		boolean handle(HttpResponse.ResponseInfo responseInfo, McpTransportContext context);

	}

	class Noop implements McpHttpClientAuthorizationErrorHandler {

		@Override
		public Publisher<Boolean> handle(HttpResponse.ResponseInfo responseInfo, McpTransportContext context) {
			return Mono.just(false);
		}

	}

}


import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import { normalizeReimbursementInput } from './utils.js';
import './check-login.js';
import './reimbursement-draft.js';
import './reimbursement-plan.js';

let tmpDir;
let receiptPath;

beforeEach(() => {
    tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-mercury-test-'));
    receiptPath = path.join(tmpDir, 'receipt.png');
    fs.writeFileSync(receiptPath, 'receipt');
});

afterEach(() => {
    fs.rmSync(tmpDir, { recursive: true, force: true });
});

function validArgs(overrides = {}) {
    return {
        receipt: receiptPath,
        amount: '140.00',
        currency: 'CNY',
        date: '2026-06-26',
        merchant: 'Example Merchant',
        category: 'Marketing & Advertising',
        notes: 'Example business purpose.',
        ...overrides,
    };
}

function createPageMock(evaluateResults, overrides = {}) {
    return {
        goto: vi.fn().mockResolvedValue(undefined),
        wait: vi.fn().mockResolvedValue(undefined),
        evaluate: vi.fn()
            .mockImplementation(() => {
                if (!evaluateResults.length) throw new Error('unexpected evaluate call');
                return evaluateResults.shift();
            }),
        uploadFiles: vi.fn().mockResolvedValue({
            uploaded: true,
            files: 1,
            file_names: ['receipt.png'],
            target: '[data-testid="expense-attachment-upload"]',
            matches_n: 1,
            match_level: 'exact',
        }),
        setFileInput: vi.fn().mockResolvedValue(undefined),
        ...overrides,
    };
}

function loggedInState(overrides = {}) {
    return {
        url: 'https://app.mercury.com/expenses/my-expenses',
        loggedIn: true,
        hasSubmitExpense: true,
        hasReimbursements: true,
        title: 'Mercury',
        ...overrides,
    };
}

function createSurface(overrides = {}) {
    return {
        url: 'https://app.mercury.com/expenses/my-expenses',
        title: 'Mercury',
        hasFinalSubmit: false,
        hasForm: false,
        bodyPreview: 'My Expenses Submit expense',
        ...overrides,
    };
}

function clickResult(clicked = true) {
    return { clicked, blocked: false, text: clicked ? 'clicked' : '' };
}

function fillResult(touched = {}) {
    return {
        touched: {
            amount: true,
            currency: true,
            date: true,
            merchant: true,
            category: true,
            notes: true,
            ...touched,
        },
    };
}

function reviewState(overrides = {}) {
    return {
        url: 'https://app.mercury.com/expenses/review',
        title: 'Mercury',
        hasReview: true,
        hasSubmitExpenseButton: true,
        bodyPreview: 'Review Submit expense',
        ...overrides,
    };
}

describe('mercury reimbursement input validation', () => {
    it('normalizes valid input and keeps receipt absolute internally', () => {
        const input = normalizeReimbursementInput(validArgs());
        expect(input).toMatchObject({
            receipt: receiptPath,
            amount: '140.00',
            currency: 'CNY',
            date: '2026-06-26',
            merchant: 'Example Merchant',
            category: 'Marketing & Advertising',
            notes: 'Example business purpose.',
            ocrWaitSeconds: 8,
            closeAfterReview: false,
        });
    });

    it('rejects malformed local-only arguments before browser work', () => {
        expect(() => normalizeReimbursementInput(validArgs({ amount: '0' }))).toThrow(ArgumentError);
        expect(() => normalizeReimbursementInput(validArgs({ amount: '1.999' }))).toThrow(ArgumentError);
        expect(() => normalizeReimbursementInput(validArgs({ currency: 'US' }))).toThrow(ArgumentError);
        expect(() => normalizeReimbursementInput(validArgs({ date: '2026-02-30' }))).toThrow(ArgumentError);
        expect(() => normalizeReimbursementInput(validArgs({ 'ocr-wait-seconds': 'abc' }))).toThrow(ArgumentError);
        expect(() => normalizeReimbursementInput(validArgs({ receipt: path.join(tmpDir, 'missing.png') }))).toThrow(ArgumentError);
    });
});

describe('mercury reimbursement-plan', () => {
    it('validates locally and returns a deterministic basename-only plan', async () => {
        const command = getRegistry().get('mercury/reimbursement-plan');
        const rows = await command.func(validArgs());
        expect(rows).toEqual([expect.objectContaining({
            status: 'ready',
            receipt: 'receipt.png',
            amount: '140.00',
            currency: 'CNY',
            safety: expect.stringContaining('never clicks final Submit expense'),
        })]);
    });
});

describe('mercury reimbursement-draft', () => {
    const command = getRegistry().get('mercury/reimbursement-draft');

    it('throws AuthRequiredError instead of returning a fake draft when logged out', async () => {
        const page = createPageMock([loggedInState({ loggedIn: false, url: 'https://app.mercury.com/login' })]);
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(AuthRequiredError);
        expect(page.uploadFiles).not.toHaveBeenCalled();
    });

    it('typed-fails malformed login state payloads', async () => {
        const page = createPageMock([{ url: 'https://app.mercury.com/expenses/my-expenses', title: 'Mercury' }]);
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
        expect(page.uploadFiles).not.toHaveBeenCalled();
    });

    it('refuses to click when an existing review/submit surface is already open', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface({ hasFinalSubmit: true, hasForm: true, bodyPreview: 'Review Submit expense amount merchant' }),
        ]);
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
        expect(page.uploadFiles).not.toHaveBeenCalled();
    });

    it('typed-fails if the create click probe sees a submit/review surface', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface(),
            { clicked: false, blocked: true, text: 'Submit expense' },
        ]);
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
        expect(page.uploadFiles).not.toHaveBeenCalled();
    });

    it('typed-fails malformed create expense surface payloads', async () => {
        const page = createPageMock([
            loggedInState(),
            { url: 'https://app.mercury.com/expenses/my-expenses', title: 'Mercury', hasForm: false },
        ]);
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
        expect(page.uploadFiles).not.toHaveBeenCalled();
    });

    it('typed-fails upload confirmation drift', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface(),
            clickResult(true),
        ], {
            uploadFiles: vi.fn().mockResolvedValue({ uploaded: false, files: 0 }),
        });
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
    });

    it('typed-fails upload confirmation for the wrong file name', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface(),
            clickResult(true),
        ], {
            uploadFiles: vi.fn().mockResolvedValue({ uploaded: true, files: 1, file_names: ['other.png'] }),
        });
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
    });

    it('typed-fails upload confirmation for the wrong input target', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface(),
            clickResult(true),
        ], {
            uploadFiles: vi.fn().mockResolvedValue({
                uploaded: true,
                files: 1,
                file_names: ['receipt.png'],
                target: '[data-testid="wrong"]',
                matches_n: 1,
            }),
        });
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
    });

    it('typed-fails missed required field selectors before Review', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface(),
            clickResult(true),
            fillResult({ merchant: false }),
        ]);
        await expect(command.func(page, validArgs())).rejects.toThrow(/merchant/);
    });

    it('typed-fails if Mercury does not reach Review with final submit visible', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface(),
            clickResult(true),
            fillResult(),
            clickResult(true),
            reviewState({ hasReview: false, hasSubmitExpenseButton: false }),
        ]);
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
    });

    it('typed-fails malformed review payloads', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface(),
            clickResult(true),
            fillResult(),
            clickResult(true),
            { url: 'https://app.mercury.com/expenses/review', title: 'Mercury', hasReview: true },
        ]);
        await expect(command.func(page, validArgs())).rejects.toBeInstanceOf(CommandExecutionError);
    });

    it('returns review-ready summary without clicking final Submit expense', async () => {
        const page = createPageMock([
            loggedInState(),
            createSurface(),
            clickResult(true),
            fillResult(),
            clickResult(true),
            reviewState(),
        ]);
        const rows = await command.func(page, validArgs());
        expect(rows).toEqual([expect.objectContaining({
            status: 'review_ready',
            receipt: 'receipt.png',
            uploaded: true,
            reviewReady: true,
            submitBlocked: true,
            warnings: 'final Submit expense was intentionally not clicked',
        })]);
        expect(page.uploadFiles).toHaveBeenCalledWith('[data-testid="expense-attachment-upload"]', [receiptPath]);
        expect(page.evaluate).toHaveBeenCalledTimes(6);
    });
});


"""Test clicking elements inside cross-origin iframes."""

import asyncio

import pytest

from browser_use.browser.profile import BrowserProfile, ViewportSize
from browser_use.browser.session import BrowserSession
from browser_use.tools.service import Tools

@pytest.fixture
async def browser_session():
	"""Create browser session with cross-origin iframe support."""
	session = BrowserSession(
		browser_profile=BrowserProfile(
			headless=True,
			user_data_dir=None,
			keep_alive=True,
			window_size=ViewportSize(width=1920, height=1400),
			cross_origin_iframes=True,  # Enable cross-origin iframe extraction
		)
	)
	await session.start()
	yield session
	await session.kill()

class TestCrossOriginIframeClick:
	"""Test clicking elements inside cross-origin iframes."""

	async def test_click_element_in_cross_origin_iframe(self, httpserver, browser_session: BrowserSession):
		"""Verify that elements inside iframes in different CDP targets can be clicked."""

		# Create iframe content with clickable elements
		iframe_html = """
		<!DOCTYPE html>

		<title>Iframe Page</title>

Iframe Content

			Test Link
			<button id="iframe-button">Iframe Button</button>

		"""

		# Create main page with iframe pointing to our test server
		main_html = """
		<!DOCTYPE html>

		<title>Multi-Target Test</title>

Main Page

			<button id="main-button">Main Button</button>

		"""

		# Serve both pages
		httpserver.expect_request('/multi-target-test').respond_with_data(main_html, content_type='text/html')
		httpserver.expect_request('/iframe-content').respond_with_data(iframe_html, content_type='text/html')
		url = httpserver.url_for('/multi-target-test')

		# Navigate to the page
		await browser_session.navigate_to(url)

		# Wait for iframe to load
		await asyncio.sleep(2)

		# Get DOM state with cross-origin iframe extraction enabled
		# Use browser_session.get_browser_state_summary() instead of directly creating DomService
		# This goes through the proper event bus and watchdog system
		browser_state = await browser_session.get_browser_state_summary(
			include_screenshot=False,
			include_recent_events=False,
		)
		assert browser_state.dom_state is not None
		state = browser_state.dom_state

		print(f'\n📊 Found {len(state.selector_map)} total elements')

		# Find elements from different targets
		targets_found = set()
		main_page_elements = []
		iframe_elements = []

		for idx, element in state.selector_map.items():
			target_id = element.target_id
			targets_found.add(target_id)

			# Check if element is from iframe (identified by id attributes we set)
			# Iframe elements will have a different target_id when cross_origin_iframes=True
			if element.attributes:
				element_id = element.attributes.get('id', '')
				if element_id in ('iframe-link', 'iframe-button'):
					iframe_elements.append((idx, element))
					print(f'   ✅ Found iframe element: [{idx}] {element.tag_name} id={element_id}')
				elif element_id == 'main-button':
					main_page_elements.append((idx, element))

		# Verify we found elements from at least 2 different targets
		print(f'\n🎯 Found elements from {len(targets_found)} different CDP targets')

		# Check if iframe elements were found
		if len(iframe_elements) == 0:
			pytest.fail('Expected to find at least one element from iframe, but found none')

		# Verify we found at least one element from the iframe
		assert len(iframe_elements) > 0, 'Expected to find at least one element from iframe'

		# Try clicking the iframe element
		print('\n🖱  Testing Click on Iframe Element:')
		tools = Tools()

		link_idx, link_element = iframe_elements[0]
		print(f'   Attempting to click element [{link_idx}] from iframe...')

		try:
			result = await tools.click(index=link_idx, browser_session=browser_session)

			# Check for errors
			if result.error:
				pytest.fail(f'Click on iframe element [{link_idx}] failed with error: {result.error}')

			if result.extracted_content and (
				'not available' in result.extracted_content.lower() or 'failed' in result.extracted_content.lower()
			):
				pytest.fail(f'Click on iframe element [{link_idx}] failed: {result.extracted_content}')

			print(f'   ✅ Click succeeded on iframe element [{link_idx}]!')
			print('   🎉 Iframe element clicking works!')

		except Exception as e:
			pytest.fail(f'Exception while clicking iframe element [{link_idx}]: {e}')

		print('\n✅ Test passed: Iframe elements can be clicked')


import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { promises as fs } from 'fs';
import path from 'path';
import os from 'os';
import { CompletionProvider } from '../../../src/core/completions/completion-provider.js';

describe('CompletionProvider', () => {
  let testDir: string;
  let provider: CompletionProvider;

  beforeEach(async () => {
    testDir = await fs.mkdtemp(path.join(os.tmpdir(), 'openspec-test-'));
    provider = new CompletionProvider(2000, testDir);
  });

  afterEach(async () => {
    await fs.rm(testDir, { recursive: true, force: true });
  });

  describe('getChangeIds', () => {
    it('should return empty array when no changes exist', async () => {
      const changeIds = await provider.getChangeIds();
      expect(changeIds).toEqual([]);
    });

    it('should return active change IDs', async () => {
      // Create openspec/changes directory structure
      const changesDir = path.join(testDir, 'openspec', 'changes');
      await fs.mkdir(changesDir, { recursive: true });

      // Create some changes
      await fs.mkdir(path.join(changesDir, 'change-1'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-1', 'proposal.md'), '# Change 1');

      await fs.mkdir(path.join(changesDir, 'change-2'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-2', 'proposal.md'), '# Change 2');

      const changeIds = await provider.getChangeIds();
      expect(changeIds).toEqual(['change-1', 'change-2']);
    });

    it('should exclude archive directory', async () => {
      const changesDir = path.join(testDir, 'openspec', 'changes');
      await fs.mkdir(changesDir, { recursive: true });

      // Create active change
      await fs.mkdir(path.join(changesDir, 'active-change'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'active-change', 'proposal.md'), '# Active');

      // Create archived change
      await fs.mkdir(path.join(changesDir, 'archive', 'old-change'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'archive', 'old-change', 'proposal.md'), '# Old');

      const changeIds = await provider.getChangeIds();
      expect(changeIds).toEqual(['active-change']);
    });

    it('should cache results for the TTL duration', async () => {
      const changesDir = path.join(testDir, 'openspec', 'changes');
      await fs.mkdir(changesDir, { recursive: true });

      await fs.mkdir(path.join(changesDir, 'change-1'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-1', 'proposal.md'), '# Change 1');

      // First call
      const firstResult = await provider.getChangeIds();
      expect(firstResult).toEqual(['change-1']);

      // Add another change
      await fs.mkdir(path.join(changesDir, 'change-2'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-2', 'proposal.md'), '# Change 2');

      // Second call should return cached result (still only change-1)
      const secondResult = await provider.getChangeIds();
      expect(secondResult).toEqual(['change-1']);
    });

    it('should refresh cache after TTL expires', async () => {
      // Use a very short TTL for testing
      const shortTTLProvider = new CompletionProvider(50, testDir);

      const changesDir = path.join(testDir, 'openspec', 'changes');
      await fs.mkdir(changesDir, { recursive: true });

      await fs.mkdir(path.join(changesDir, 'change-1'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-1', 'proposal.md'), '# Change 1');

      // First call
      const firstResult = await shortTTLProvider.getChangeIds();
      expect(firstResult).toEqual(['change-1']);

      // Add another change
      await fs.mkdir(path.join(changesDir, 'change-2'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-2', 'proposal.md'), '# Change 2');

      // Wait for cache to expire
      await new Promise(resolve => setTimeout(resolve, 60));

      // Should now see both changes
      const secondResult = await shortTTLProvider.getChangeIds();
      expect(secondResult).toEqual(['change-1', 'change-2']);
    });
  });

  describe('getSpecIds', () => {
    it('should return empty array when no specs exist', async () => {
      const specIds = await provider.getSpecIds();
      expect(specIds).toEqual([]);
    });

    it('should return spec IDs', async () => {
      const specsDir = path.join(testDir, 'openspec', 'specs');
      await fs.mkdir(specsDir, { recursive: true });

      // Create some specs
      await fs.mkdir(path.join(specsDir, 'spec-1'), { recursive: true });
      await fs.writeFile(path.join(specsDir, 'spec-1', 'spec.md'), '# Spec 1');

      await fs.mkdir(path.join(specsDir, 'spec-2'), { recursive: true });
      await fs.writeFile(path.join(specsDir, 'spec-2', 'spec.md'), '# Spec 2');

      const specIds = await provider.getSpecIds();
      expect(specIds).toEqual(['spec-1', 'spec-2']);
    });

    it('should cache results for the TTL duration', async () => {
      const specsDir = path.join(testDir, 'openspec', 'specs');
      await fs.mkdir(specsDir, { recursive: true });

      await fs.mkdir(path.join(specsDir, 'spec-1'), { recursive: true });
      await fs.writeFile(path.join(specsDir, 'spec-1', 'spec.md'), '# Spec 1');

      // First call
      const firstResult = await provider.getSpecIds();
      expect(firstResult).toEqual(['spec-1']);

      // Add another spec
      await fs.mkdir(path.join(specsDir, 'spec-2'), { recursive: true });
      await fs.writeFile(path.join(specsDir, 'spec-2', 'spec.md'), '# Spec 2');

      // Second call should return cached result
      const secondResult = await provider.getSpecIds();
      expect(secondResult).toEqual(['spec-1']);
    });

    it('should refresh cache after TTL expires', async () => {
      const shortTTLProvider = new CompletionProvider(50, testDir);

      const specsDir = path.join(testDir, 'openspec', 'specs');
      await fs.mkdir(specsDir, { recursive: true });

      await fs.mkdir(path.join(specsDir, 'spec-1'), { recursive: true });
      await fs.writeFile(path.join(specsDir, 'spec-1', 'spec.md'), '# Spec 1');

      const firstResult = await shortTTLProvider.getSpecIds();
      expect(firstResult).toEqual(['spec-1']);

      // Add another spec
      await fs.mkdir(path.join(specsDir, 'spec-2'), { recursive: true });
      await fs.writeFile(path.join(specsDir, 'spec-2', 'spec.md'), '# Spec 2');

      // Wait for cache to expire
      await new Promise(resolve => setTimeout(resolve, 60));

      const secondResult = await shortTTLProvider.getSpecIds();
      expect(secondResult).toEqual(['spec-1', 'spec-2']);
    });
  });

  describe('getAllIds', () => {
    it('should return both change and spec IDs', async () => {
      const changesDir = path.join(testDir, 'openspec', 'changes');
      const specsDir = path.join(testDir, 'openspec', 'specs');
      await fs.mkdir(changesDir, { recursive: true });
      await fs.mkdir(specsDir, { recursive: true });

      // Create a change
      await fs.mkdir(path.join(changesDir, 'my-change'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'my-change', 'proposal.md'), '# Change');

      // Create a spec
      await fs.mkdir(path.join(specsDir, 'my-spec'), { recursive: true });
      await fs.writeFile(path.join(specsDir, 'my-spec', 'spec.md'), '# Spec');

      const result = await provider.getAllIds();
      expect(result).toEqual({
        changeIds: ['my-change'],
        specIds: ['my-spec'],
      });
    });

    it('should return empty arrays when no items exist', async () => {
      const result = await provider.getAllIds();
      expect(result).toEqual({
        changeIds: [],
        specIds: [],
      });
    });
  });

  describe('clearCache', () => {
    it('should clear all cached data', async () => {
      const changesDir = path.join(testDir, 'openspec', 'changes');
      await fs.mkdir(changesDir, { recursive: true });

      await fs.mkdir(path.join(changesDir, 'change-1'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-1', 'proposal.md'), '# Change 1');

      // Populate cache
      await provider.getChangeIds();

      // Clear cache
      provider.clearCache();

      // Add new change
      await fs.mkdir(path.join(changesDir, 'change-2'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-2', 'proposal.md'), '# Change 2');

      // Should see new data immediately
      const result = await provider.getChangeIds();
      expect(result).toEqual(['change-1', 'change-2']);
    });
  });

  describe('getCacheStats', () => {
    it('should report invalid cache when empty', () => {
      const stats = provider.getCacheStats();
      expect(stats.changeCache.valid).toBe(false);
      expect(stats.specCache.valid).toBe(false);
      expect(stats.changeCache.age).toBeUndefined();
      expect(stats.specCache.age).toBeUndefined();
    });

    it('should report valid cache after data is fetched', async () => {
      const changesDir = path.join(testDir, 'openspec', 'changes');
      await fs.mkdir(changesDir, { recursive: true });

      await fs.mkdir(path.join(changesDir, 'change-1'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-1', 'proposal.md'), '# Change 1');

      await provider.getChangeIds();

      const stats = provider.getCacheStats();
      expect(stats.changeCache.valid).toBe(true);
      expect(stats.changeCache.age).toBeDefined();
      expect(stats.changeCache.age).toBeLessThan(100);
    });

    it('should report invalid cache after TTL expires', async () => {
      const shortTTLProvider = new CompletionProvider(50, testDir);

      const changesDir = path.join(testDir, 'openspec', 'changes');
      await fs.mkdir(changesDir, { recursive: true });

      await fs.mkdir(path.join(changesDir, 'change-1'), { recursive: true });
      await fs.writeFile(path.join(changesDir, 'change-1', 'proposal.md'), '# Change 1');

      await shortTTLProvider.getChangeIds();

      // Wait for cache to expire
      await new Promise(resolve => setTimeout(resolve, 60));

      const stats = shortTTLProvider.getCacheStats();
      expect(stats.changeCache.valid).toBe(false);
      expect(stats.changeCache.age).toBeGreaterThan(50);
    });
  });

  describe('constructor', () => {
    it('should use default TTL of 2000ms', async () => {
      const defaultProvider = new CompletionProvider();
      expect(defaultProvider).toBeDefined();
      // We can verify this behavior by checking cache stats after waiting
    });

    it('should accept custom TTL', async () => {
      const customProvider = new CompletionProvider(5000, testDir);
      expect(customProvider).toBeDefined();
    });

    it('should use process.cwd() as default project root', () => {
      const defaultProvider = new CompletionProvider();
      expect(defaultProvider).toBeDefined();
    });
  });
});


use std::io::Write;
use std::path::{Path, PathBuf};

use anyhow::Result;
use clap::Parser;
use tokio::io::AsyncBufReadExt;

use crate::git::get_git_dir;
use crate::hook::Hook;
use crate::hooks::pre_commit_hooks::{hook_filenames, parse_hook_args};
use crate::hooks::run_concurrent_file_checks;
use crate::run::INTERNAL_CONCURRENCY;

const START_PATTERN: &[u8] = b"<<<<<<< ";
const ANCESTOR_PATTERN: &[u8] = b"||||||| ";
const END_PATTERN: &[u8] = b">>>>>>> ";
const SEPARATOR_PATTERNS: &[&[u8]] = &[b"======= ", b"=======\r\n", b"=======\n"];

#[derive(Parser)]
#[command(disable_help_subcommand = true)]
#[command(disable_version_flag = true)]
#[command(disable_help_flag = true)]
struct Args {
    #[arg(long)]
    assume_in_merge: bool,
    #[arg(value_name = "FILENAMES")]
    filenames: Vec<PathBuf>,
}

pub(crate) async fn check_merge_conflict(
    hook: &Hook,
    filenames: &[&Path],
) -> Result<(i32, Vec<u8>)> {
    let args: Args = parse_hook_args(hook)?;

    // Check if we're in a merge state or assuming merge
    if !args.assume_in_merge && !is_in_merge().await? {
        return Ok((0, Vec::new()));
    }

    run_concurrent_file_checks(
        hook_filenames(&args.filenames, filenames),
        *INTERNAL_CONCURRENCY,
        |filename| check_file(hook.project().relative_path(), filename),
    )
    .await
}

async fn is_in_merge() -> Result<bool> {
    // Change directory temporarily or ensure we're in the right directory
    let git_dir = get_git_dir().await?;

    // Check if MERGE_MSG exists
    let merge_msg_exists = git_dir.join("MERGE_MSG").exists();
    if !merge_msg_exists {
        return Ok(false);
    }

    // Check if any of the merge state files exist
    Ok(git_dir.join("MERGE_HEAD").exists()
        || git_dir.join("rebase-apply").exists()
        || git_dir.join("rebase-merge").exists())
}

async fn check_file(file_base: &Path, filename: &Path) -> Result<(i32, Vec<u8>)> {
    let file_path = file_base.join(filename);
    let file = fs_err::tokio::File::open(&file_path).await?;
    let mut reader = tokio::io::BufReader::new(file);

    let mut code = 0;
    let mut output = Vec::new();
    let mut line = Vec::new();
    let mut line_number = 1;
    let mut in_conflict = false;

    let mut report_conflict = |line_number: usize, pattern: &str| -> Result<()> {
        write_conflict_message(&mut output, filename, line_number, pattern)?;
        code = 1;
        Ok(())
    };

    while reader.read_until(b'\n', &mut line).await? != 0 {
        if line.starts_with(START_PATTERN) {
            report_conflict(line_number, "<<<<<<< ")?;
            in_conflict = true;
        } else if in_conflict && line.starts_with(ANCESTOR_PATTERN) {
            report_conflict(line_number, "||||||| ")?;
        } else if in_conflict
            && SEPARATOR_PATTERNS
                .iter()
                .any(|pattern| line.starts_with(pattern))
        {
            report_conflict(line_number, "=======")?;
        } else if line.starts_with(END_PATTERN) {
            report_conflict(line_number, ">>>>>>> ")?;
            in_conflict = false;
        }

        line.clear();
        line_number += 1;
    }

    Ok((code, output))
}

fn write_conflict_message(
    output: &mut Vec<u8>,
    filename: &Path,
    line_number: usize,
    pattern: &str,
) -> std::io::Result<()> {
    writeln!(
        output,
        "{}:{line_number}: Merge conflict string {pattern:?} found",
        filename.display(),
    )
}

#[cfg(test)]
mod tests {
    use super::*;
    use std::path::PathBuf;
    use tempfile::tempdir;

    async fn create_test_file(
        dir: &tempfile::TempDir,
        name: &str,
        content: &[u8],
    ) -> Result<PathBuf> {
        let file_path = dir.path().join(name);
        fs_err::tokio::write(&file_path, content).await?;
        Ok(file_path)
    }

    #[tokio::test]
    async fn test_no_conflict_markers() -> Result<()> {
        let dir = tempdir()?;
        let content = b"This is a normal file\nWith no conflict markers\n";
        let file_path = create_test_file(&dir, "clean.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 0);
        assert!(output.is_empty());
        Ok(())
    }

    #[tokio::test]
    async fn test_conflict_marker_start() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Some content\n<<<<<<< HEAD\nConflicting line\n";
        let file_path = create_test_file(&dir, "conflict.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        assert!(!output.is_empty());
        let output_str = String::from_utf8_lossy(&output);
        assert!(output_str.contains("<<<<<<< "));
        assert!(output_str.contains("conflict.txt:2"));
        Ok(())
    }

    #[tokio::test]
    async fn test_conflict_marker_end() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Some content\n>>>>>>> branch\nMore content\n";
        let file_path = create_test_file(&dir, "conflict.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        assert!(!output.is_empty());
        let output_str = String::from_utf8_lossy(&output);
        assert!(output_str.contains(">>>>>>> "));
        Ok(())
    }

    #[tokio::test]
    async fn test_full_conflict_block() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Before conflict\n<<<<<<< HEAD\nOur changes\n=======\nTheir changes\n>>>>>>> branch\nAfter conflict\n";
        let file_path = create_test_file(&dir, "conflict.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        assert!(!output.is_empty());
        let output_str = String::from_utf8_lossy(&output);
        // Should find all three markers
        assert!(output_str.contains("<<<<<<< "));
        assert!(output_str.contains("======="));
        assert!(output_str.contains(">>>>>>> "));
        Ok(())
    }

    #[tokio::test]
    async fn test_diff3_conflict_block() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Before conflict\n<<<<<<< HEAD\nOur changes\n||||||| base\nCommon ancestor\n=======\nTheir changes\n>>>>>>> branch\nAfter conflict\n";
        let file_path = create_test_file(&dir, "conflict.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        assert!(!output.is_empty());
        let output_str = String::from_utf8_lossy(&output);
        assert!(output_str.contains("<<<<<<< "));
        assert!(output_str.contains("||||||| "));
        assert!(output_str.contains("======="));
        assert!(output_str.contains(">>>>>>> "));
        Ok(())
    }

    #[tokio::test]
    async fn test_conflict_marker_not_at_start() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Some content <<<<<<< HEAD\n";
        let file_path = create_test_file(&dir, "no_conflict.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        // Should not detect conflict since marker is not at line start
        assert_eq!(code, 0);
        assert!(output.is_empty());
        Ok(())
    }

    #[tokio::test]
    async fn test_conflict_marker_crlf() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Some content\r\n<<<<<<< HEAD\r\nConflicting line\r\n=======\r\nOther line\r\n>>>>>>> branch\r\n";
        let file_path = create_test_file(&dir, "conflict_crlf.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        assert!(!output.is_empty());
        Ok(())
    }

    #[tokio::test]
    async fn test_conflict_marker_lf() -> Result<()> {
        let dir = tempdir()?;
        let content =
            b"Some content\n<<<<<<< HEAD\nConflicting line\n=======\nOther line\n>>>>>>> branch\n";
        let file_path = create_test_file(&dir, "conflict_lf.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        assert!(!output.is_empty());
        Ok(())
    }

    #[tokio::test]
    async fn test_separator_reported_without_conflict_end() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Before conflict\n<<<<<<< HEAD\nOur changes\n=======\n";
        let file_path = create_test_file(&dir, "partial_conflict.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        let output_str = String::from_utf8_lossy(&output);
        assert!(output_str.contains("<<<<<<< "));
        assert!(output_str.contains("======="));
        Ok(())
    }

    #[tokio::test]
    async fn test_ancestor_not_reported_without_conflict_start() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Before conflict\n||||||| base\n";
        let file_path = create_test_file(&dir, "partial_conflict.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 0);
        assert!(output.is_empty());
        Ok(())
    }

    #[tokio::test]
    async fn test_rst_heading_is_not_treated_as_conflict() -> Result<()> {
        let dir = tempdir()?;
        let content = b"Depends\n=======\n";
        let file_path = create_test_file(&dir, "doc.rst", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 0);
        assert!(output.is_empty());
        Ok(())
    }

    #[tokio::test]
    async fn test_empty_file() -> Result<()> {
        let dir = tempdir()?;
        let content = b"";
        let file_path = create_test_file(&dir, "empty.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 0);
        assert!(output.is_empty());
        Ok(())
    }

    #[tokio::test]
    async fn test_multiple_conflicts() -> Result<()> {
        let dir = tempdir()?;
        let content = b"<<<<<<< HEAD\nFirst\n=======\nSecond\n>>>>>>> branch\nMiddle\n<<<<<<< HEAD\nThird\n=======\nFourth\n>>>>>>> other\n";
        let file_path = create_test_file(&dir, "multiple.txt", content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        let output_str = String::from_utf8_lossy(&output);
        // Should find all markers from both conflicts (one per line with marker)
        let marker_count = output_str.matches("Merge conflict string").count();
        assert_eq!(marker_count, 6); // 3 markers per conflict * 2 conflicts
        Ok(())
    }

    #[tokio::test]
    async fn test_binary_file_with_conflict() -> Result<()> {
        let dir = tempdir()?;
        let mut content = vec![0xFF, 0xFE, 0xFD];
        content.extend_from_slice(b"\n<<<<<<< HEAD\n");
        let file_path = create_test_file(&dir, "binary.bin", &content).await?;
        let (code, output) = check_file(Path::new(""), &file_path).await?;
        assert_eq!(code, 1);
        assert!(!output.is_empty());
        Ok(())
    }
}


import { Command } from "commander";
import pc from "picocolors";
import ora from "ora";

import { resolveLibrary, getLibraryContext } from "../utils/api.js";
import { recoverLibraryId } from "../utils/library-id.js";
import { log } from "../utils/logger.js";
import { trackEvent } from "../utils/tracking.js";
import { loadTokens, isTokenExpired } from "../utils/auth.js";
import type { LibrarySearchResult, ContextResponse } from "../types.js";

const isTTY = process.stdout.isTTY;

function getReputationLabel(score: number | undefined): "High" | "Medium" | "Low" | "Unknown" {
  if (score === undefined || score < 0) return "Unknown";
  if (score >= 7) return "High";
  if (score >= 4) return "Medium";
  return "Low";
}

function getAccessToken(): string | undefined {
  const tokens = loadTokens();
  if (!tokens || isTokenExpired(tokens)) return undefined;
  return tokens.access_token;
}

function formatLibraryResult(lib: LibrarySearchResult, index: number): string {
  const lines: string[] = [];
  lines.push(`${pc.dim(`${index + 1}.`)} ${pc.bold(`Title: ${lib.title}`)}`);
  lines.push(`   ${pc.cyan(`Context7-compatible library ID: ${lib.id}`)}`);

  if (lib.description) {
    lines.push(`   ${pc.dim(`Description: ${lib.description}`)}`);
  }

  if (lib.totalSnippets) {
    lines.push(`   ${pc.dim(`Code Snippets: ${lib.totalSnippets}`)}`);
  }
  if (lib.trustScore !== undefined) {
    lines.push(`   ${pc.dim(`Source Reputation: ${getReputationLabel(lib.trustScore)}`)}`);
  }
  if (lib.benchmarkScore !== undefined && lib.benchmarkScore > 0) {
    lines.push(`   ${pc.dim(`Benchmark Score: ${lib.benchmarkScore}`)}`);
  }
  if (lib.versions && lib.versions.length > 0) {
    lines.push(`   ${pc.dim(`Versions: ${lib.versions.join(", ")}`)}`);
  }

  return lines.join("\n");
}

async function resolveCommand(
  library: string,
  query: string | undefined,
  options: { json?: boolean }
): Promise<void> {
  trackEvent("command", { name: "library" });

  const spinner = isTTY ? ora(`Searching for "${library}"...`).start() : null;
  const accessToken = getAccessToken();

  let data;
  try {
    data = await resolveLibrary(library, query, accessToken);
  } catch (err) {
    spinner?.fail(`Error: ${err instanceof Error ? err.message : String(err)}`);
    if (!spinner) log.error(err instanceof Error ? err.message : String(err));
    process.exitCode = 1;
    return;
  }

  if (data.error) {
    spinner?.fail(data.message || data.error);
    if (!spinner) log.error(data.message || data.error);
    process.exitCode = 1;
    return;
  }

  if (!data.results || data.results.length === 0) {
    spinner?.warn(`No libraries found matching "${library}"`);
    if (!spinner) log.warn(`No libraries found matching "${library}"`);
    return;
  }

  const results = data.results;

  spinner?.stop();

  if (options.json) {
    console.log(JSON.stringify(results, null, 2));
    return;
  }

  log.blank();

  if (data.searchFilterApplied) {
    log.warn(
      "Your results only include libraries matching your teamspace's library filters. To adjust quality thresholds or blocked libraries, update your filters at https://context7.com/dashboard?tab=policies"
    );
    log.blank();
  }

  for (let i = 0; i < results.length; i++) {
    log.plain(formatLibraryResult(results[i], i));
    log.blank();
  }

  if (isTTY && results.length > 0) {
    const best = results[0];
    log.plain(
      `${pc.bold("Quick command:")}\n` + `  ${pc.cyan(`ctx7 docs "${best.id}" "<your question>"`)}`
    );
    log.blank();
  }
}

async function queryCommand(
  libraryId: string,
  query: string,
  options: { json?: boolean }
): Promise<void> {
  trackEvent("command", { name: "docs" });

  // Git Bash on Windows rewrites "/owner/repo" into a Windows path; recover it.
  libraryId = recoverLibraryId(libraryId);

  if (!libraryId.startsWith("/") || !/^\/[^/]+\/[^/]/.test(libraryId)) {
    log.error(`Invalid library ID: "${libraryId}"`);
    log.info(`Expected format: /owner/repo or /owner/repo/version (e.g., /facebook/react)`);
    log.info(`Run "ctx7 library <name>" to find the correct ID`);
    if (process.platform === "win32") {
      log.info(
        `On Git Bash, prefix the ID with an extra slash to avoid path conversion: ctx7 docs "//facebook/react" "<your question>"`
      );
    }
    process.exitCode = 1;
    return;
  }

  const accessToken = getAccessToken();

  const spinner = isTTY ? ora(`Fetching docs for "${libraryId}"...`).start() : null;
  const outputType = options.json ? "json" : "txt";

  let result;
  try {
    result = await getLibraryContext(libraryId, query, { type: outputType }, accessToken);
  } catch (err) {
    spinner?.fail(`Error: ${err instanceof Error ? err.message : String(err)}`);
    if (!spinner) log.error(err instanceof Error ? err.message : String(err));
    process.exitCode = 1;
    return;
  }

  if (typeof result === "string") {
    spinner?.stop();
    console.log(result);
    return;
  }

  const ctx = result as ContextResponse;

  if (ctx.error) {
    if (ctx.redirectUrl) {
      spinner?.warn("Library has been redirected");
      if (!spinner) log.warn("Library has been redirected");
      log.info(`New ID: ${pc.cyan(ctx.redirectUrl)}`);
      log.info(`Run: ${pc.cyan(`ctx7 docs "${ctx.redirectUrl}" "${query}"`)}`);
      process.exitCode = 1;
      return;
    }

    spinner?.fail(ctx.message || ctx.error);
    if (!spinner) log.error(ctx.message || ctx.error);
    process.exitCode = 1;
    return;
  }

  const total = (ctx.codeSnippets?.length || 0) + (ctx.infoSnippets?.length || 0);
  if (total === 0) {
    spinner?.warn(`No documentation found for: "${query}"`);
    if (!spinner) log.warn(`No documentation found for: "${query}"`);
    return;
  }

  spinner?.stop();

  if (options.json) {
    console.log(JSON.stringify(ctx, null, 2));
    return;
  }

  log.blank();

  if (ctx.codeSnippets) {
    for (const snippet of ctx.codeSnippets) {
      log.plain(pc.bold(snippet.codeTitle));
      if (snippet.codeDescription) log.dim(snippet.codeDescription);
      log.blank();
      for (const code of snippet.codeList) {
        log.plain("```" + code.language);
        log.plain(code.code);
        log.plain("```");
        log.blank();
      }
    }
  }

  if (ctx.infoSnippets) {
    for (const snippet of ctx.infoSnippets) {
      if (snippet.breadcrumb) log.plain(pc.bold(snippet.breadcrumb));
      log.plain(snippet.content);
      log.blank();
    }
  }
}

export function registerDocsCommands(program: Command): void {
  program
    .command("library")
    .argument("<name>", "Library name to search for")
    .argument("[query]", "What to look up in the library's documentation")
    .option("--json", "Output as JSON")
    .description("Resolve a library name to a Context7 library ID")
    .action(async (name: string, query: string | undefined, options: { json?: boolean }) => {
      await resolveCommand(name, query, options);
    });

  program
    .command("docs")
    .argument("<libraryId>", "Context7 library ID (e.g., /facebook/react)")
    .argument(
      "<query>",
      "Single-topic question to get docs for (run a separate query per distinct concept, unless asking how they interact)"
    )
    .option("--json", "Output as JSON")
    .description("Query documentation for a library")
    .action(async (libraryId: string, query: string, options: { json?: boolean }) => {
      await queryCommand(libraryId, query, options);
    });
}


"""
Tests for Selector.iterancestors() and Selector.find_ancestor() methods.
Target file: tests/parser/test_general.py (append to TestElementNavigation class)
"""
import pytest
from scrapling import Selector

@pytest.fixture
def nested_page():
    html = """

deep text

    """
    return Selector(html, adaptive=False)

class TestAncestorNavigation:
    def test_iterancestors_returns_all_ancestors(self, nested_page):
        """iterancestors() should yield every ancestor up to """
        target = nested_page.css("#target")[0]
        ancestor_tags = [a.tag for a in target.iterancestors()]
        # Expected order: p → article → section → div → body → html
        assert ancestor_tags[:4] == ["p", "article", "section", "div"]
        assert "body" in ancestor_tags
        assert "html" in ancestor_tags

    def test_iterancestors_order_is_bottom_up(self, nested_page):
        """iterancestors() should start from the immediate parent, not the root"""
        target = nested_page.css("#target")[0]
        first_ancestor = next(target.iterancestors())
        assert first_ancestor.attrib.get("id") == "level4"

    def test_find_ancestor_returns_first_match(self, nested_page):
        """find_ancestor() should return the closest ancestor matching the predicate"""
        target = nested_page.css("#target")[0]
        # Looking for the nearest ancestor with class "card"
        result = target.find_ancestor(lambda el: el.has_class("card"))
        assert result is not None
        assert result.attrib.get("id") == "level3"

    def test_find_ancestor_returns_none_when_not_found(self, nested_page):
        """find_ancestor() should return None if no ancestor matches"""
        target = nested_page.css("#target")[0]
        result = target.find_ancestor(lambda el: el.has_class("nonexistent-class"))
        assert result is None

    def test_iterancestors_on_text_node_is_empty(self, nested_page):
        """iterancestors() on a text node should yield nothing (not raise)"""
        text_node = nested_page.css("#target::text")[0]
        ancestors = list(text_node.iterancestors())
        assert ancestors == []

    def test_find_ancestor_on_root_element_returns_none(self, nested_page):
        """find_ancestor() on the root  element should return None gracefully"""
        # html element has no ancestors
        html_el = nested_page.css("html")[0]
        result = html_el.find_ancestor(lambda el: True)
        assert result is None


# Agent Configuration & Behavior

## Table of Contents
- [Basic Usage](#basic-usage)
- [All Parameters](#all-parameters)
- [Output Format](#output-format)
- [Structured Output](#structured-output)
- [Prompting Guide](#prompting-guide)
- [Lifecycle Hooks](#lifecycle-hooks)
- [Timeout Environment Variables](#timeout-environment-variables)

---

## Basic Usage

```python
from browser_use import Agent, ChatBrowserUse

agent = Agent(
    task="Search for latest news about AI",
    llm=ChatBrowserUse(),
)

async def main():
    history = await agent.run(max_steps=500)
```

- `task`: The task to automate
- `llm`: LLM instance (see `models.md`)
- `max_steps` (default: `500`): Maximum agent steps

## All Parameters

### Core Settings
- `tools`: Registry of tools the agent can call
- `skills` (or `skill_ids`): List of skill IDs to load (e.g., `['skill-uuid']` or `['*']` for all). Requires `BROWSER_USE_API_KEY`
- `browser`: Browser object for browser settings
- `output_model_schema`: Pydantic model class for structured output validation

### Vision & Processing
- `use_vision` (default: `True`): `True` always includes screenshots, `"auto"` includes screenshot tool but only uses vision when requested, `False` never
- `vision_detail_level` (default: `'auto'`): `'low'`, `'high'`, or `'auto'`
- `page_extraction_llm`: Separate LLM for page content extraction (default: same as `llm`)

### Fallback & Resilience
- `fallback_llm`: Backup LLM when primary fails. Primary exhausts its retry logic (5 attempts with exponential backoff) first. Triggers on: 429 (rate limit), 401 (auth), 402 (payment), 500/502/503/504 (server errors). Once switched, fallback is used for rest of run.

### Actions & Behavior
- `initial_actions`: Actions to run before main task without LLM
- `max_actions_per_step` (default: `5`): Max actions per step (e.g., fill 5 form fields at once)
- `max_failures` (default: `5`): Max retries for steps with errors
- `final_response_after_failure` (default: `True`): Force one final model call after max_failures
- `use_thinking` (default: `True`): Enable explicit reasoning steps
- `flash_mode` (default: `False`): Fast mode — skips evaluation, next goal, thinking; uses memory only. Overrides `use_thinking`

### System Messages
- `override_system_message`: Completely replace default system prompt
- `extend_system_message`: Add instructions to default system prompt

### File & Data Management
- `save_conversation_path`: Path to save conversation history
- `save_conversation_path_encoding` (default: `'utf-8'`)
- `available_file_paths`: File paths the agent can access
- `sensitive_data`: Dict of sensitive data (see `examples.md` for patterns)

### Visual Output
- `generate_gif` (default: `False`): Generate GIF of actions. Set to `True` or string path
- `include_attributes`: HTML attributes to include in page analysis

### Performance & Limits
- `max_history_items`: Max steps to keep in LLM memory (`None` = all)
- `llm_timeout` (default: auto-detected per model — Groq: 30s, Gemini: 75s, Gemini 3 Pro: 90s, o3/Claude/DeepSeek: 90s, others: 75s): Seconds for LLM calls
- `step_timeout` (default: `180`): Seconds for each step
- `directly_open_url` (default: `True`): Auto-open URLs detected in task

### Advanced
- `calculate_cost` (default: `False`): Track API costs (access via `history.usage`)
- `display_files_in_done_text` (default: `True`)

### Backwards Compatibility
- `controller` → alias for `tools`
- `browser_session` → alias for `browser`

---

## Output Format

`run()` returns an `AgentHistoryList`:

```python
history = await agent.run()

# Basic access
history.urls()                    # Visited URLs
history.screenshot_paths()        # Screenshot file paths
history.screenshots()             # Screenshots as base64
history.action_names()            # Executed action names
history.extracted_content()       # Extracted content from all actions
history.errors()                  # Errors (None for clean steps)
history.model_actions()           # All actions with parameters
history.model_outputs()           # All model outputs
history.last_action()             # Last action

# Analysis
history.final_result()            # Final extracted content (last step)
history.is_done()                 # Agent completed?
history.is_successful()           # Completed successfully? (None if not done)
history.has_errors()              # Any errors?
history.model_thoughts()          # Reasoning (AgentBrain objects)
history.action_results()          # All ActionResult objects
history.action_history()          # Truncated action history
history.number_of_steps()         # Step count
history.total_duration_seconds()  # Total duration

# Structured output
history.structured_output         # Parsed structured output (if output_model_schema set)
```

## Structured Output

Use `output_model_schema` with a Pydantic model:

```python
from pydantic import BaseModel

class SearchResult(BaseModel):
    title: str
    url: str

agent = Agent(task="...", llm=llm, output_model_schema=SearchResult)
history = await agent.run()
result = history.structured_output  # SearchResult instance
```

---

## Prompting Guide

### Be Specific

```python
# Good
task = """
1. Go to https://quotes.toscrape.com/
2. Use extract action with the query "first 3 quotes with their authors"
3. Save results to quotes.csv using write_file action
"""

# Bad
task = "Go to web and make money"
```

### Name Actions Directly

```python
task = """
1. Use search action to find "Python tutorials"
2. Use click to open first result in a new tab
3. Use scroll action to scroll down 2 pages
4. Use extract to extract the names of the first 5 items
"""
```

### Handle Interaction Problems via Keyboard

```python
task = """
If the submit button cannot be clicked:
1. Use send_keys action with "Tab Tab Enter"
2. Or use send_keys with "ArrowDown ArrowDown Enter"
"""
```

### Custom Actions Integration

```python
@tools.action("Get 2FA code from authenticator app")
async def get_2fa_code():
    pass

task = """
Login with 2FA:
1. Enter username/password
2. When prompted for 2FA, use get_2fa_code action
3. NEVER try to extract 2FA codes from the page manually
"""
```

### Error Recovery

```python
task = """
1. Go to openai.com to find their CEO
2. If navigation fails due to anti-bot protection:
   - Use google search to find the CEO
3. If page times out, use go_back and try alternative approach
"""
```

---

## Lifecycle Hooks

Two hooks available via `agent.run()`:

| Hook | When Called |
|------|------------|
| `on_step_start` | Before agent processes current state |
| `on_step_end` | After agent executes all actions for step |

```python
async def my_hook(agent: Agent):
    state = await agent.browser_session.get_browser_state_summary()
    print(f'Current URL: {state.url}')

await agent.run(on_step_start=my_hook, on_step_end=my_hook)
```

### Data Available in Hooks

Full access to Agent instance:

- `agent.task` — current task; `agent.add_new_task(...)` — queue new task
- `agent.tools` — Tools() object and Registry
  - `agent.tools.registry.execute_action('click', {'index': 123}, browser_session=agent.browser_session)`
- `agent.sensitive_data` — sensitive data dict (mutable)
- `agent.settings` — all config options
- `agent.llm` — direct LLM access
- `agent.state` — internal state (thoughts, outputs, actions)
- `agent.history` — execution history:
  - `.model_thoughts()`, `.model_outputs()`, `.model_actions()`
  - `.extracted_content()`, `.urls()`
- `agent.browser_session` — BrowserSession + CDP:
  - `.agent_focus_target_id` — current target ID
  - `.get_or_create_cdp_session()` — CDP session
  - `.get_tabs()`, `.get_current_page_url()`, `.get_current_page_title()`
- `agent.pause()` / `agent.resume()` — control execution

### Hook Example: CDP Access

```python
async def my_hook(agent: Agent):
    cdp_session = await agent.browser_session.get_or_create_cdp_session()
    doc = await cdp_session.cdp_client.send.DOM.getDocument(session_id=cdp_session.session_id)
    html = await cdp_session.cdp_client.send.DOM.getOuterHTML(
        params={'nodeId': doc['root']['nodeId']}, session_id=cdp_session.session_id
    )
```

**Tips:**
- Keep hooks efficient (same execution thread)
- Most use cases are better served by custom tools
- Increase `step_timeout` if hooks take long

---

## Timeout Environment Variables

Fine-tune timeouts via environment variables (values in seconds):

### Browser Actions
| Variable | Default |
|----------|---------|
| `TIMEOUT_NavigateToUrlEvent` | 30.0 |
| `TIMEOUT_ClickElementEvent` | 15.0 |
| `TIMEOUT_ClickCoordinateEvent` | 15.0 |
| `TIMEOUT_TypeTextEvent` | 60.0 |
| `TIMEOUT_ScrollEvent` | 8.0 |
| `TIMEOUT_ScrollToTextEvent` | 15.0 |
| `TIMEOUT_SendKeysEvent` | 60.0 |
| `TIMEOUT_UploadFileEvent` | 30.0 |
| `TIMEOUT_GetDropdownOptionsEvent` | 15.0 |
| `TIMEOUT_SelectDropdownOptionEvent` | 8.0 |
| `TIMEOUT_GoBackEvent` | 15.0 |
| `TIMEOUT_GoForwardEvent` | 15.0 |
| `TIMEOUT_RefreshEvent` | 15.0 |
| `TIMEOUT_WaitEvent` | 60.0 |
| `TIMEOUT_ScreenshotEvent` | 15.0 |
| `TIMEOUT_BrowserStateRequestEvent` | 30.0 |

### Browser Lifecycle
| Variable | Default |
|----------|---------|
| `TIMEOUT_BrowserStartEvent` | 30.0 |
| `TIMEOUT_BrowserStopEvent` | 45.0 |
| `TIMEOUT_BrowserLaunchEvent` | 30.0 |
| `TIMEOUT_BrowserKillEvent` | 30.0 |
| `TIMEOUT_BrowserConnectedEvent` | 30.0 |

### Tab Management
| Variable | Default |
|----------|---------|
| `TIMEOUT_SwitchTabEvent` | 10.0 |
| `TIMEOUT_CloseTabEvent` | 10.0 |
| `TIMEOUT_TabCreatedEvent` | 30.0 |
| `TIMEOUT_TabClosedEvent` | 10.0 |

### Storage & Downloads
| Variable | Default |
|----------|---------|
| `TIMEOUT_SaveStorageStateEvent` | 45.0 |
| `TIMEOUT_LoadStorageStateEvent` | 45.0 |
| `TIMEOUT_FileDownloadedEvent` | 30.0 |