Hugging Face's logo

Spaces:
paras9909
/
opticparse-python
Paused

App Files Community
opticparse-python
File size: 3,864 Bytes
bcf46c3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
# Opticparse SDK & CLI Client

The official, strongly-typed TypeScript/JavaScript SDK and CLI tool for the **Opticparse AI Vision Web Scraper**. 

By capturing visual snapshots (screenshots) of target web pages and parsing them with Gemini's vision-language models, Opticparse allows you to scrape dynamic JS-heavy websites **without maintaining CSS or XPath selectors**. If the website redesigns its code, your scraper continues to work flawlessly.

---

## Features
* 🛡️ **Fully typed** interfaces with TypeScript autocomplete.
* ⚡ **Selector-free scraping** via AI multimodal visual extraction.
* 🌐 Works seamlessly with both **direct Render deployments** and the **opticparse.com Gateway**.
* 🐚 **Terminal CLI companion tool** to inspect and test scraping results locally.

---

## Installation

Install the package via npm:

```bash
npm install opticparse-js
```

---

## Authentication Configuration

The client requires an API Key. You can pass it explicitly in the code constructor or set it as environment variables:

### Direct Server Configuration
```env
OPTICPARSE_API_KEY=your_secure_opticparse_api_key_here
OPTICPARSE_API_URL=https://opticparse.onrender.com # defaults to http://localhost:8000
```

### opticparse.com Proxy Configuration
```env
opticparse.com_KEY=your_opticparse.com_developer_key_here
OPTICPARSE_USE_opticparse.com=true
```

---

## Library Usage (JS/TS)

### Direct Mode Example
Querying your direct/private Render or local backend server:

```typescript
import { OpticparseClient } from 'opticparse-js';

const client = new OpticparseClient({
  apiKey: 'your_secure_opticparse_api_key_here',
  apiUrl: 'https://opticparse.onrender.com'
});

async function run() {
  try {
    const result = await client.scrape({
      targetUrl: 'https://news.ycombinator.com',
      extractionQuery: 'Extract the top 5 article titles, their link URLs, and points as a JSON list.',
    });
    
    console.log(result);
  } catch (error) {
    console.error('Scrape error:', error);
  }
}

run();
```

### opticparse.com Mode Example
Querying the API hosted on the opticparse.com Hub:

```typescript
import { OpticparseClient } from 'opticparse-js';

const client = new OpticparseClient({
  apiKey: 'your_opticparse.com_developer_key_here',
  useopticparse.com: true
});

async function run() {
  const result = await client.scrape({
    targetUrl: 'https://example.com',
    extractionQuery: 'Extract the main heading text.'
  });
  console.log(result);
}

run();
```

---

## Command Line Interface (CLI)

You can run scrapers directly from your shell.

### Global Installation (Optional)
```bash
npm install -g opticparse-js
```

### Direct Server Query
```bash
# Set environment key
export OPTICPARSE_API_KEY="your_key"

# Scrape using CLI
npx opticparse scrape \
  --url "https://example.com" \
  --query "Extract the page header text" \
  --api-url "https://opticparse.onrender.com"
```

### opticparse.com Query
```bash
npx opticparse scrape \
  --url "https://example.com" \
  --query "Extract the page header text" \
  --key "your_opticparse.com_developer_key_here" \
  --opticparse.com
```

---

## API Reference Options

The `scrape()` method supports the following options:

| Option | Type | Default | Description |
|---|---|---|---|
| `targetUrl` | `string` | **Required** | The target website URL to scrape. |
| `extractionQuery` | `string` | **Required** | Instructions on what data structure and values to extract. |
| `viewportWidth` | `number` | `1280` | Browser viewport width in pixels. |
| `viewportHeight` | `number` | `800` | Browser viewport height in pixels. |
| `waitUntil` | `'networkidle' \| 'load' \| 'domcontentloaded'` | `'networkidle'` | Playwright page load synchronization state. |
| `timeout` | `number` | `30000` | Request and navigation timeout threshold in milliseconds. |

---

## License
MIT