opticparse-js/README.md

Opticparse SDK & CLI Client

The official, strongly-typed TypeScript/JavaScript SDK and CLI tool for the Opticparse AI Vision Web Scraper.

By capturing visual snapshots (screenshots) of target web pages and parsing them with Gemini's vision-language models, Opticparse allows you to scrape dynamic JS-heavy websites without maintaining CSS or XPath selectors. If the website redesigns its code, your scraper continues to work flawlessly.

---

Features

• 🛡️ Fully typed interfaces with TypeScript autocomplete.
• ⚡ Selector-free scraping via AI multimodal visual extraction.
• 🌐 Works seamlessly with both direct Render deployments and the opticparse.com Gateway.
• 🐚 Terminal CLI companion tool to inspect and test scraping results locally.

---

Installation

Install the package via npm:

npm install opticparse-js

---

Authentication Configuration

The client requires an API Key. You can pass it explicitly in the code constructor or set it as environment variables:

Direct Server Configuration

OPTICPARSE_API_KEY=your_secure_opticparse_api_key_here
OPTICPARSE_API_URL=https://opticparse.onrender.com # defaults to http://localhost:8000

opticparse.com Proxy Configuration

opticparse.com_KEY=your_opticparse.com_developer_key_here
OPTICPARSE_USE_opticparse.com=true

---

Library Usage (JS/TS)

Direct Mode Example

Querying your direct/private Render or local backend server:

import { OpticparseClient } from 'opticparse-js';

const client = new OpticparseClient({
  apiKey: 'your_secure_opticparse_api_key_here',
  apiUrl: 'https://opticparse.onrender.com'
});

async function run() {
  try {
    const result = await client.scrape({
      targetUrl: 'https://news.ycombinator.com',
      extractionQuery: 'Extract the top 5 article titles, their link URLs, and points as a JSON list.',
    });
    
    console.log(result);
  } catch (error) {
    console.error('Scrape error:', error);
  }
}

run();

opticparse.com Mode Example

Querying the API hosted on the opticparse.com Hub:

import { OpticparseClient } from 'opticparse-js';

const client = new OpticparseClient({
  apiKey: 'your_opticparse.com_developer_key_here',
  useopticparse.com: true
});

async function run() {
  const result = await client.scrape({
    targetUrl: 'https://example.com',
    extractionQuery: 'Extract the main heading text.'
  });
  console.log(result);
}

run();

---

Command Line Interface (CLI)

You can run scrapers directly from your shell.

Global Installation (Optional)

npm install -g opticparse-js

Direct Server Query

# Set environment key
export OPTICPARSE_API_KEY="your_key"

# Scrape using CLI
npx opticparse scrape \
  --url "https://example.com" \
  --query "Extract the page header text" \
  --api-url "https://opticparse.onrender.com"

opticparse.com Query

npx opticparse scrape \
  --url "https://example.com" \
  --query "Extract the page header text" \
  --key "your_opticparse.com_developer_key_here" \
  --opticparse.com

---

API Reference Options

The scrape() method supports the following options:

Option	Type	Default	Description
targetUrl	string	Required	The target website URL to scrape.
extractionQuery	string	Required	Instructions on what data structure and values to extract.
viewportWidth	number	1280	Browser viewport width in pixels.
viewportHeight	number	800	Browser viewport height in pixels.
waitUntil	'networkidle' | 'load' | 'domcontentloaded'	'networkidle'	Playwright page load synchronization state.
timeout	number	30000	Request and navigation timeout threshold in milliseconds.

---

License

MIT