Reorganize: agent/ (vidai package) + web/ (extract_product + results); mirror local layout
Browse files- .gitattributes +7 -0
- README.md +7 -130
- .env.example → agent/.env.example +0 -0
- .gitignore → agent/.gitignore +0 -0
- agent/README.md +132 -0
- agent/out/test_seevio.mp4 +3 -0
- pyproject.toml → agent/pyproject.toml +0 -0
- {scripts → agent/scripts}/test_seevio.py +0 -0
- {tests → agent/tests}/test_e2e_mock.py +0 -0
- {vidai → agent/vidai}/__init__.py +0 -0
- {vidai → agent/vidai}/agent/__init__.py +0 -0
- {vidai → agent/vidai}/agent/harness.py +0 -0
- {vidai → agent/vidai}/agent/models.py +0 -0
- {vidai → agent/vidai}/agent/prompts/__init__.py +0 -0
- {vidai → agent/vidai}/agent/prompts/loader.py +0 -0
- {vidai → agent/vidai}/agent/prompts/system.md +0 -0
- {vidai → agent/vidai}/agent/providers/__init__.py +0 -0
- {vidai → agent/vidai}/agent/providers/anthropic_native.py +0 -0
- {vidai → agent/vidai}/agent/providers/base.py +0 -0
- {vidai → agent/vidai}/agent/providers/chat_completions.py +0 -0
- {vidai → agent/vidai}/agent/providers/mock.py +0 -0
- {vidai → agent/vidai}/agent/run_context.py +0 -0
- {vidai → agent/vidai}/agent/seedance.py +0 -0
- {vidai → agent/vidai}/agent/single_run.py +0 -0
- {vidai → agent/vidai}/agent/tools/__init__.py +0 -0
- {vidai → agent/vidai}/agent/tools/analyze_style.py +0 -0
- {vidai → agent/vidai}/agent/tools/base.py +0 -0
- {vidai → agent/vidai}/agent/tools/critique_video.py +0 -0
- {vidai → agent/vidai}/agent/tools/finalize.py +0 -0
- {vidai → agent/vidai}/agent/tools/generate_video.py +0 -0
- {vidai → agent/vidai}/agent/tools/probe_video.py +0 -0
- {vidai → agent/vidai}/agent/tools/registry.py +0 -0
- {vidai → agent/vidai}/agent/tools/scrape_reference.py +0 -0
- {vidai → agent/vidai}/cli.py +0 -0
- {vidai → agent/vidai}/config.py +0 -0
- {vidai → agent/vidai}/storage/__init__.py +0 -0
- {vidai → agent/vidai}/storage/layout.py +0 -0
- {vidai → agent/vidai}/storage/records.py +0 -0
- web/extract_product.py +462 -0
- web/results/tt_bissell/images/description_01.jpg +3 -0
- web/results/tt_bissell/images/gallery_01.webp +0 -0
- web/results/tt_bissell/images/gallery_02.webp +3 -0
- web/results/tt_bissell/images/gallery_03.webp +3 -0
- web/results/tt_bissell/images/gallery_04.webp +3 -0
- web/results/tt_bissell/images/gallery_05.webp +0 -0
- web/results/tt_bissell/images/gallery_06.webp +0 -0
- web/results/tt_bissell/images/gallery_07.webp +3 -0
- web/results/tt_bissell/images/gallery_08.webp +3 -0
- web/results/tt_bissell/product.json +222 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
agent/out/test_seevio.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
web/results/tt_bissell/images/description_01.jpg filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
web/results/tt_bissell/images/gallery_02.webp filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
web/results/tt_bissell/images/gallery_03.webp filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
web/results/tt_bissell/images/gallery_04.webp filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
web/results/tt_bissell/images/gallery_07.webp filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
web/results/tt_bissell/images/gallery_08.webp filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,132 +1,9 @@
|
|
| 1 |
-
---
|
| 2 |
-
tags:
|
| 3 |
-
- agent
|
| 4 |
-
- video-generation
|
| 5 |
-
- seedance
|
| 6 |
-
- product-video
|
| 7 |
-
---
|
| 8 |
-
|
| 9 |
# vidai
|
| 10 |
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
product
|
| 17 |
-
|
| 18 |
-
first frame), then a vision judge compares candidate keyframes against reference keyframes and
|
| 19 |
-
sends the agent back to regenerate with concrete revisions until the result is indistinguishable
|
| 20 |
-
in style — photoreal people and scenes included.
|
| 21 |
-
|
| 22 |
-
## Pipeline
|
| 23 |
-
|
| 24 |
-
```
|
| 25 |
-
product photo + reference URL + requirements
|
| 26 |
-
│
|
| 27 |
-
▼
|
| 28 |
-
┌─ VidaiAgent (multi-turn tool loop, trace.jsonl) ─────────────────────┐
|
| 29 |
-
│ scrape_reference → download reference videos + page context │
|
| 30 |
-
│ probe_video → ffprobe metadata + keyframe JPEGs │
|
| 31 |
-
│ analyze_style → VLM style profile from reference keyframes │
|
| 32 |
-
│ generate_video → Seedance task API (photo = first frame) │
|
| 33 |
-
│ critique_video → VLM judge: realism / style-match / suggestions │
|
| 34 |
-
│ └── verdict "revise" → new prompt → generate_video again │
|
| 35 |
-
│ finalize → pick passing candidate, end run │
|
| 36 |
-
└──────────────────────────────────────────────────────────────────────┘
|
| 37 |
-
│
|
| 38 |
-
▼
|
| 39 |
-
data/records/rec_*/ (final.mp4, record.json, style_profile.json,
|
| 40 |
-
critiques.json, trace.jsonl, input photo)
|
| 41 |
-
```
|
| 42 |
-
|
| 43 |
-
## Setup
|
| 44 |
-
|
| 45 |
-
```bash
|
| 46 |
-
cd vidai
|
| 47 |
-
cp .env.example .env # fill in ARK_API_KEY and VIDAI_SEEDANCE_MODEL
|
| 48 |
-
pip install -e ".[dev]" # or just ensure httpx/requests/bs4/pydantic present
|
| 49 |
-
```
|
| 50 |
-
|
| 51 |
-
Two video backends are supported, selected by `VIDAI_SEEDANCE_BASE_URL`:
|
| 52 |
-
|
| 53 |
-
- **seevio.ai** (aggregator): base URL `https://api.seevio.ai`, model `seedance-2-5`,
|
| 54 |
-
key from the seevio dashboard. Image-to-video takes public URLs only, so local first
|
| 55 |
-
frames are staged on a temporary file host; i2v forces `aspect_ratio=adaptive`.
|
| 56 |
-
- **Volcano Engine Ark / BytePlus ModelArk** (first-party): `ARK_API_KEY` from
|
| 57 |
-
console.volcengine.com/ark and the exact versioned model id from your console
|
| 58 |
-
("开通模型" page), e.g. `doubao-seedance-2-5-pro-<date>`.
|
| 59 |
-
|
| 60 |
-
Two planner/vision providers, selected by `VIDAI_LLM_MODEL`:
|
| 61 |
-
|
| 62 |
-
- **Claude** (native Anthropic Messages API): any model id starting with `claude`
|
| 63 |
-
(e.g. `claude-opus-5`) with an Anthropic key in `VIDAI_LLM_API_KEY`. Adaptive
|
| 64 |
-
thinking on; server-side refusal fallbacks enabled by default.
|
| 65 |
-
- **OpenAI-compatible**: any `/chat/completions` endpoint with vision + function
|
| 66 |
-
calling via `VIDAI_LLM_BASE_URL/MODEL/API_KEY` (gpt-4o, doubao vision, Qwen-VL...).
|
| 67 |
-
|
| 68 |
-
## Usage
|
| 69 |
-
|
| 70 |
-
```bash
|
| 71 |
-
# Real run
|
| 72 |
-
python -m vidai.cli generate \
|
| 73 |
-
--image product.jpg \
|
| 74 |
-
--url "https://example.com/product-page" \
|
| 75 |
-
--duration 5 --ratio 16:9 --resolution 1080p \
|
| 76 |
-
"真人手部交互,温馨厨房场景"
|
| 77 |
-
|
| 78 |
-
# Offline dry run — full agent loop with mocked LLM + mocked Seedance (ffmpeg synth)
|
| 79 |
-
python -m vidai.cli generate --mock --image product.jpg --url page.html "..."
|
| 80 |
-
|
| 81 |
-
python -m vidai.cli list # browse finished records
|
| 82 |
-
```
|
| 83 |
-
|
| 84 |
-
Exit code 0 = success. Every run prints per-candidate critique scores; the full decision trace
|
| 85 |
-
is in the record's `trace.jsonl`.
|
| 86 |
-
|
| 87 |
-
## Debugging the workflow
|
| 88 |
-
|
| 89 |
-
- `--mock` runs the entire loop offline: scripted planner policy
|
| 90 |
-
(`vidai/agent/providers/mock.py`), canned VLM replies (first critique = revise, second = pass,
|
| 91 |
-
so the revision loop is always exercised), and ffmpeg-synthesized "Seedance" output.
|
| 92 |
-
- `tests/test_e2e_mock.py` builds a local fixture product page + video and asserts the whole
|
| 93 |
-
pipeline: 2 generation attempts, revise→pass critiques, record/manifest/trace on disk.
|
| 94 |
-
Run with `python -m pytest tests/ -q`.
|
| 95 |
-
- `scrape_reference` also accepts local HTML paths / `file://` URLs — handy for reproducing a
|
| 96 |
-
problematic page offline.
|
| 97 |
-
- Intermediate artifacts live under `data/cache/runs/<run_id>/` (references, extracted frames,
|
| 98 |
-
every candidate video); records only keep the final picks.
|
| 99 |
-
|
| 100 |
-
## Notes & limits
|
| 101 |
-
|
| 102 |
-
- Scraper covers `<video>/<source>`, `og:video`, JSON-LD `VideoObject`, and raw mp4/webm/mov
|
| 103 |
-
links. Pages that stream via JS blobs (many SPAs) yield nothing — pass a direct video URL
|
| 104 |
-
instead; the tool's error message says so.
|
| 105 |
-
- Generation attempts are budgeted (`VIDAI_MAX_ATTEMPTS`, default 3); when exhausted the agent
|
| 106 |
-
must finalize the best candidate instead of burning more Seedance credits.
|
| 107 |
-
- The critique judges sampled keyframes, not motion; temporal artifacts (flicker, morphing
|
| 108 |
-
between frames) can slip through. Raise `num_frames` or review candidates by eye for
|
| 109 |
-
high-stakes outputs.
|
| 110 |
-
- Only scrape pages you have the right to use as style references, and follow the target site's
|
| 111 |
-
terms of service.
|
| 112 |
-
|
| 113 |
-
## Layout (articraft-style)
|
| 114 |
-
|
| 115 |
-
```
|
| 116 |
-
vidai/
|
| 117 |
-
├── vidai/
|
| 118 |
-
│ ├── config.py # .env + settings
|
| 119 |
-
│ ├── cli.py # generate / list
|
| 120 |
-
│ ├── agent/
|
| 121 |
-
│ │ ├── harness.py # VidaiAgent tool loop + trace (≈ articraft harness.py)
|
| 122 |
-
│ │ ├── single_run.py # one-job orchestration (≈ single_run.py)
|
| 123 |
-
│ │ ├── run_context.py # per-run workspace + state (≈ run_context.py)
|
| 124 |
-
│ │ ├── seedance.py # Seedance clients: Ark + seevio task APIs + mock
|
| 125 |
-
│ │ ├── models.py # RunInput / AgentOutcome
|
| 126 |
-
│ │ ├── prompts/ # system.md + loader
|
| 127 |
-
│ │ ├── providers/ # base protocol, chat-completions, anthropic (Claude), mock
|
| 128 |
-
│ │ └── tools/ # base, registry + the 6 tools
|
| 129 |
-
│ └── storage/ # DataLayout, records, manifest.jsonl
|
| 130 |
-
├── tests/ # offline e2e (mock everything)
|
| 131 |
-
└── data/ # gitignored run artifacts + records
|
| 132 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# vidai
|
| 2 |
|
| 3 |
+
- `agent/` — the agentic product-video generator (Python package `vidai`, tests, config).
|
| 4 |
+
Run from inside it: `cd agent && python -m vidai.cli generate --image ... --url ... "..."`.
|
| 5 |
+
Secrets live in `agent/.env` (never commit / upload).
|
| 6 |
+
- `web/` — product-page extraction: `web/extract_product.py <url>` pulls title, price, SKUs,
|
| 7 |
+
seller, rating, description and images from a TikTok Shop PDP or a brand site into
|
| 8 |
+
`web/results/<slug>/product.json` + `images/`. `web/results/tt_bissell/` is the BISSELL
|
| 9 |
+
Little Green example.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.env.example → agent/.env.example
RENAMED
|
File without changes
|
.gitignore → agent/.gitignore
RENAMED
|
File without changes
|
agent/README.md
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- agent
|
| 4 |
+
- video-generation
|
| 5 |
+
- seedance
|
| 6 |
+
- product-video
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# vidai
|
| 10 |
+
|
| 11 |
+
**Agentic product-video generation**, modeled on [articraft](../articraft)'s agentic base
|
| 12 |
+
architecture (declarative tools + registry, provider abstraction, multi-turn harness,
|
| 13 |
+
records/manifest storage, per-run trace).
|
| 14 |
+
|
| 15 |
+
Give it a **product photo** and a **reference product-page URL**. The agent scrapes the page's
|
| 16 |
+
product videos, distills their style (scene, people, camera, lighting, pacing, grade, duration,
|
| 17 |
+
aspect ratio), generates a candidate with **Seedance 2.5** (image-to-video, your photo as the
|
| 18 |
+
first frame), then a vision judge compares candidate keyframes against reference keyframes and
|
| 19 |
+
sends the agent back to regenerate with concrete revisions until the result is indistinguishable
|
| 20 |
+
in style — photoreal people and scenes included.
|
| 21 |
+
|
| 22 |
+
## Pipeline
|
| 23 |
+
|
| 24 |
+
```
|
| 25 |
+
product photo + reference URL + requirements
|
| 26 |
+
│
|
| 27 |
+
▼
|
| 28 |
+
┌─ VidaiAgent (multi-turn tool loop, trace.jsonl) ─────────────────────┐
|
| 29 |
+
│ scrape_reference → download reference videos + page context │
|
| 30 |
+
│ probe_video → ffprobe metadata + keyframe JPEGs │
|
| 31 |
+
│ analyze_style → VLM style profile from reference keyframes │
|
| 32 |
+
│ generate_video → Seedance task API (photo = first frame) │
|
| 33 |
+
│ critique_video → VLM judge: realism / style-match / suggestions │
|
| 34 |
+
│ └── verdict "revise" → new prompt → generate_video again │
|
| 35 |
+
│ finalize → pick passing candidate, end run │
|
| 36 |
+
└──────────────────────────────────────────────────────────────────────┘
|
| 37 |
+
│
|
| 38 |
+
▼
|
| 39 |
+
data/records/rec_*/ (final.mp4, record.json, style_profile.json,
|
| 40 |
+
critiques.json, trace.jsonl, input photo)
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
## Setup
|
| 44 |
+
|
| 45 |
+
```bash
|
| 46 |
+
cd vidai/agent
|
| 47 |
+
cp .env.example .env # fill in ARK_API_KEY and VIDAI_SEEDANCE_MODEL
|
| 48 |
+
pip install -e ".[dev]" # or just ensure httpx/requests/bs4/pydantic present
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
Two video backends are supported, selected by `VIDAI_SEEDANCE_BASE_URL`:
|
| 52 |
+
|
| 53 |
+
- **seevio.ai** (aggregator): base URL `https://api.seevio.ai`, model `seedance-2-5`,
|
| 54 |
+
key from the seevio dashboard. Image-to-video takes public URLs only, so local first
|
| 55 |
+
frames are staged on a temporary file host; i2v forces `aspect_ratio=adaptive`.
|
| 56 |
+
- **Volcano Engine Ark / BytePlus ModelArk** (first-party): `ARK_API_KEY` from
|
| 57 |
+
console.volcengine.com/ark and the exact versioned model id from your console
|
| 58 |
+
("开通模型" page), e.g. `doubao-seedance-2-5-pro-<date>`.
|
| 59 |
+
|
| 60 |
+
Two planner/vision providers, selected by `VIDAI_LLM_MODEL`:
|
| 61 |
+
|
| 62 |
+
- **Claude** (native Anthropic Messages API): any model id starting with `claude`
|
| 63 |
+
(e.g. `claude-opus-5`) with an Anthropic key in `VIDAI_LLM_API_KEY`. Adaptive
|
| 64 |
+
thinking on; server-side refusal fallbacks enabled by default.
|
| 65 |
+
- **OpenAI-compatible**: any `/chat/completions` endpoint with vision + function
|
| 66 |
+
calling via `VIDAI_LLM_BASE_URL/MODEL/API_KEY` (gpt-4o, doubao vision, Qwen-VL...).
|
| 67 |
+
|
| 68 |
+
## Usage
|
| 69 |
+
|
| 70 |
+
```bash
|
| 71 |
+
# Real run
|
| 72 |
+
python -m vidai.cli generate \
|
| 73 |
+
--image product.jpg \
|
| 74 |
+
--url "https://example.com/product-page" \
|
| 75 |
+
--duration 5 --ratio 16:9 --resolution 1080p \
|
| 76 |
+
"真人手部交互,温馨厨房场景"
|
| 77 |
+
|
| 78 |
+
# Offline dry run — full agent loop with mocked LLM + mocked Seedance (ffmpeg synth)
|
| 79 |
+
python -m vidai.cli generate --mock --image product.jpg --url page.html "..."
|
| 80 |
+
|
| 81 |
+
python -m vidai.cli list # browse finished records
|
| 82 |
+
```
|
| 83 |
+
|
| 84 |
+
Exit code 0 = success. Every run prints per-candidate critique scores; the full decision trace
|
| 85 |
+
is in the record's `trace.jsonl`.
|
| 86 |
+
|
| 87 |
+
## Debugging the workflow
|
| 88 |
+
|
| 89 |
+
- `--mock` runs the entire loop offline: scripted planner policy
|
| 90 |
+
(`vidai/agent/providers/mock.py`), canned VLM replies (first critique = revise, second = pass,
|
| 91 |
+
so the revision loop is always exercised), and ffmpeg-synthesized "Seedance" output.
|
| 92 |
+
- `tests/test_e2e_mock.py` builds a local fixture product page + video and asserts the whole
|
| 93 |
+
pipeline: 2 generation attempts, revise→pass critiques, record/manifest/trace on disk.
|
| 94 |
+
Run with `python -m pytest tests/ -q`.
|
| 95 |
+
- `scrape_reference` also accepts local HTML paths / `file://` URLs — handy for reproducing a
|
| 96 |
+
problematic page offline.
|
| 97 |
+
- Intermediate artifacts live under `data/cache/runs/<run_id>/` (references, extracted frames,
|
| 98 |
+
every candidate video); records only keep the final picks.
|
| 99 |
+
|
| 100 |
+
## Notes & limits
|
| 101 |
+
|
| 102 |
+
- Scraper covers `<video>/<source>`, `og:video`, JSON-LD `VideoObject`, and raw mp4/webm/mov
|
| 103 |
+
links. Pages that stream via JS blobs (many SPAs) yield nothing — pass a direct video URL
|
| 104 |
+
instead; the tool's error message says so.
|
| 105 |
+
- Generation attempts are budgeted (`VIDAI_MAX_ATTEMPTS`, default 3); when exhausted the agent
|
| 106 |
+
must finalize the best candidate instead of burning more Seedance credits.
|
| 107 |
+
- The critique judges sampled keyframes, not motion; temporal artifacts (flicker, morphing
|
| 108 |
+
between frames) can slip through. Raise `num_frames` or review candidates by eye for
|
| 109 |
+
high-stakes outputs.
|
| 110 |
+
- Only scrape pages you have the right to use as style references, and follow the target site's
|
| 111 |
+
terms of service.
|
| 112 |
+
|
| 113 |
+
## Layout (articraft-style)
|
| 114 |
+
|
| 115 |
+
```
|
| 116 |
+
vidai/
|
| 117 |
+
├── vidai/
|
| 118 |
+
│ ├── config.py # .env + settings
|
| 119 |
+
│ ├── cli.py # generate / list
|
| 120 |
+
│ ├── agent/
|
| 121 |
+
│ │ ├── harness.py # VidaiAgent tool loop + trace (≈ articraft harness.py)
|
| 122 |
+
│ │ ├── single_run.py # one-job orchestration (≈ single_run.py)
|
| 123 |
+
│ │ ├── run_context.py # per-run workspace + state (≈ run_context.py)
|
| 124 |
+
│ │ ├── seedance.py # Seedance clients: Ark + seevio task APIs + mock
|
| 125 |
+
│ │ ├── models.py # RunInput / AgentOutcome
|
| 126 |
+
│ │ ├── prompts/ # system.md + loader
|
| 127 |
+
│ │ ├── providers/ # base protocol, chat-completions, anthropic (Claude), mock
|
| 128 |
+
│ │ └── tools/ # base, registry + the 6 tools
|
| 129 |
+
│ └── storage/ # DataLayout, records, manifest.jsonl
|
| 130 |
+
├── tests/ # offline e2e (mock everything)
|
| 131 |
+
└── data/ # gitignored run artifacts + records
|
| 132 |
+
```
|
agent/out/test_seevio.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cd3a7758dac4c597ead1559df631aaad5c1d8b25164eb53f011e48b88adf9e53
|
| 3 |
+
size 1333388
|
pyproject.toml → agent/pyproject.toml
RENAMED
|
File without changes
|
{scripts → agent/scripts}/test_seevio.py
RENAMED
|
File without changes
|
{tests → agent/tests}/test_e2e_mock.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/__init__.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/__init__.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/harness.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/models.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/prompts/__init__.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/prompts/loader.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/prompts/system.md
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/providers/__init__.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/providers/anthropic_native.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/providers/base.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/providers/chat_completions.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/providers/mock.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/run_context.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/seedance.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/single_run.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/__init__.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/analyze_style.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/base.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/critique_video.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/finalize.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/generate_video.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/probe_video.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/registry.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/agent/tools/scrape_reference.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/cli.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/config.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/storage/__init__.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/storage/layout.py
RENAMED
|
File without changes
|
{vidai → agent/vidai}/storage/records.py
RENAMED
|
File without changes
|
web/extract_product.py
ADDED
|
@@ -0,0 +1,462 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Extract product info + images from a TikTok Shop PDP or a brand/official product page.
|
| 3 |
+
|
| 4 |
+
Usage:
|
| 5 |
+
python web/extract_product.py <url> [--out DIR] [--max-images N] [--no-images]
|
| 6 |
+
|
| 7 |
+
Writes <out>/product.json (unified schema) and downloads images to <out>/images/ (default out: web/results/<slug>).
|
| 8 |
+
|
| 9 |
+
TikTok Shop: parses the server-rendered `__MODERN_ROUTER_DATA__` JSON (components_map ->
|
| 10 |
+
product_info): title, price, gallery, description blocks (+images), SKUs, attributes,
|
| 11 |
+
seller/shop, rating + review sample, shipping, categories. Falls back to og:* meta.
|
| 12 |
+
Official sites: Shopify `/products/<handle>.json` -> JSON-LD Product -> og:* / itemprop
|
| 13 |
+
-> <img> gallery heuristics. Bot-protected pages (403 / empty shells) are reported, not faked.
|
| 14 |
+
"""
|
| 15 |
+
|
| 16 |
+
from __future__ import annotations
|
| 17 |
+
|
| 18 |
+
import argparse
|
| 19 |
+
import html as html_mod
|
| 20 |
+
import json
|
| 21 |
+
import mimetypes
|
| 22 |
+
import re
|
| 23 |
+
import sys
|
| 24 |
+
from pathlib import Path
|
| 25 |
+
from typing import Any
|
| 26 |
+
from urllib.parse import urljoin, urlparse
|
| 27 |
+
|
| 28 |
+
import httpx
|
| 29 |
+
from bs4 import BeautifulSoup
|
| 30 |
+
|
| 31 |
+
UA = (
|
| 32 |
+
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
| 33 |
+
"(KHTML, like Gecko) Chrome/126.0 Safari/537.36"
|
| 34 |
+
)
|
| 35 |
+
HEADERS = {
|
| 36 |
+
"User-Agent": UA,
|
| 37 |
+
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
|
| 38 |
+
"Accept-Language": "en-US,en;q=0.9",
|
| 39 |
+
}
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
# ----------------------------------------------------------------------------- helpers
|
| 43 |
+
def fetch(client: httpx.Client, url: str) -> httpx.Response:
|
| 44 |
+
return client.get(url, headers=HEADERS, follow_redirects=True, timeout=40.0)
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def clean_text(s: str | None) -> str:
|
| 48 |
+
return re.sub(r"\s+", " ", html_mod.unescape(s or "")).strip()
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def html_to_text(fragment: str | None) -> str:
|
| 52 |
+
if not fragment:
|
| 53 |
+
return ""
|
| 54 |
+
return clean_text(BeautifulSoup(fragment, "html.parser").get_text(" "))
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def first_url(obj: Any) -> str | None:
|
| 58 |
+
"""TikTok image objects carry url_list; take the first reachable CDN URL."""
|
| 59 |
+
if isinstance(obj, dict):
|
| 60 |
+
lst = obj.get("url_list") or []
|
| 61 |
+
return lst[0] if lst else obj.get("url")
|
| 62 |
+
if isinstance(obj, str):
|
| 63 |
+
return obj
|
| 64 |
+
return None
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
def blank_product(url: str, platform: str) -> dict[str, Any]:
|
| 68 |
+
return {
|
| 69 |
+
"platform": platform,
|
| 70 |
+
"url": url,
|
| 71 |
+
"title": None,
|
| 72 |
+
"brand": None,
|
| 73 |
+
"price": None,
|
| 74 |
+
"description": "",
|
| 75 |
+
"bullet_points": [],
|
| 76 |
+
"images": [],
|
| 77 |
+
"videos": [],
|
| 78 |
+
"seller": {},
|
| 79 |
+
"rating": {},
|
| 80 |
+
"sold_count": None,
|
| 81 |
+
"skus": [],
|
| 82 |
+
"attributes": {},
|
| 83 |
+
"shipping": {},
|
| 84 |
+
"categories": [],
|
| 85 |
+
"blocked": False,
|
| 86 |
+
"notes": [],
|
| 87 |
+
}
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
# ------------------------------------------------------------------------- TikTok Shop
|
| 91 |
+
def is_tiktok_shop(url: str) -> bool:
|
| 92 |
+
host = urlparse(url).netloc.lower()
|
| 93 |
+
return "tiktok.com" in host and ("/pdp/" in url or "/product/" in url or host.startswith("vt."))
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
def extract_router_data(raw: str) -> dict[str, Any] | None:
|
| 97 |
+
m = re.search(
|
| 98 |
+
r'<script type="application/json" id="__MODERN_ROUTER_DATA__">(.*?)</script>', raw, re.S
|
| 99 |
+
)
|
| 100 |
+
if not m:
|
| 101 |
+
return None
|
| 102 |
+
try:
|
| 103 |
+
return json.loads(html_mod.unescape(m.group(1)))
|
| 104 |
+
except json.JSONDecodeError:
|
| 105 |
+
return None
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
def extract_tiktok(raw: str, url: str) -> dict[str, Any]:
|
| 109 |
+
out = blank_product(url, "tiktok_shop")
|
| 110 |
+
soup = BeautifulSoup(raw, "html.parser")
|
| 111 |
+
og = {m.get("property"): m.get("content") for m in soup.find_all("meta", property=True)}
|
| 112 |
+
out["title"] = clean_text(og.get("og:title"))
|
| 113 |
+
out["description"] = clean_text(og.get("og:description"))
|
| 114 |
+
if og.get("og:image"):
|
| 115 |
+
out["images"].append({"url": og["og:image"], "kind": "og", "width": None, "height": None})
|
| 116 |
+
|
| 117 |
+
data = extract_router_data(raw)
|
| 118 |
+
page = None
|
| 119 |
+
if data:
|
| 120 |
+
page = next(
|
| 121 |
+
(v for v in (data.get("loaderData") or {}).values() if isinstance(v, dict) and "page_config" in v),
|
| 122 |
+
None,
|
| 123 |
+
)
|
| 124 |
+
if not page:
|
| 125 |
+
out["notes"].append("no __MODERN_ROUTER_DATA__ product payload; og:* fallback only")
|
| 126 |
+
out["blocked"] = not out["title"]
|
| 127 |
+
return out
|
| 128 |
+
|
| 129 |
+
comps = {c.get("component_name"): c.get("component_data") for c in page["page_config"].get("components_map", [])}
|
| 130 |
+
pinfo = comps.get("product_info") or {}
|
| 131 |
+
core = (pinfo.get("product_info") or {})
|
| 132 |
+
pm = core.get("product_model") or {}
|
| 133 |
+
if not pm.get("name"):
|
| 134 |
+
out["notes"].append("product_model empty (bot-gated response); og:* fallback only")
|
| 135 |
+
return out
|
| 136 |
+
|
| 137 |
+
# --- identity / basics
|
| 138 |
+
out["product_id"] = pm.get("product_id")
|
| 139 |
+
out["title"] = clean_text(pm.get("name")) or out["title"]
|
| 140 |
+
out["sold_count"] = pm.get("sold_count")
|
| 141 |
+
out["categories"] = [c.get("category_name") for c in pinfo.get("categories") or [] if c.get("category_name")]
|
| 142 |
+
|
| 143 |
+
# --- price
|
| 144 |
+
promo = (core.get("promotion_model") or {}).get("promotion_product_price") or {}
|
| 145 |
+
mp = promo.get("min_price") or {}
|
| 146 |
+
if mp:
|
| 147 |
+
out["price"] = {
|
| 148 |
+
"amount": mp.get("sale_price_decimal"),
|
| 149 |
+
"currency": mp.get("currency_name"),
|
| 150 |
+
"symbol": mp.get("currency_symbol"),
|
| 151 |
+
"display": f"{mp.get('currency_symbol', '')}{mp.get('sale_price_format', '')}",
|
| 152 |
+
}
|
| 153 |
+
sku_prices = promo.get("skus_price") or {}
|
| 154 |
+
|
| 155 |
+
# --- gallery images (url_list[0] is the CDN variant the page itself uses)
|
| 156 |
+
out["images"] = []
|
| 157 |
+
for i, img in enumerate(pm.get("images") or []):
|
| 158 |
+
u = first_url(img)
|
| 159 |
+
if u:
|
| 160 |
+
out["images"].append({"url": u, "kind": "gallery", "index": i, "width": img.get("width"), "height": img.get("height"), "uri": img.get("uri")})
|
| 161 |
+
|
| 162 |
+
# --- description blocks (JSON string: text + image blocks)
|
| 163 |
+
desc = pm.get("description")
|
| 164 |
+
if isinstance(desc, str):
|
| 165 |
+
try:
|
| 166 |
+
desc = json.loads(desc)
|
| 167 |
+
except json.JSONDecodeError:
|
| 168 |
+
desc = [{"type": "text", "text": desc}]
|
| 169 |
+
paragraphs: list[str] = []
|
| 170 |
+
for j, block in enumerate(desc or []):
|
| 171 |
+
if block.get("type") == "text" and block.get("text"):
|
| 172 |
+
paragraphs.append(clean_text(block["text"]))
|
| 173 |
+
elif block.get("type") == "image":
|
| 174 |
+
u = first_url(block.get("image"))
|
| 175 |
+
if u:
|
| 176 |
+
img = block["image"]
|
| 177 |
+
out["images"].append({"url": u, "kind": "description", "index": j, "width": img.get("width"), "height": img.get("height"), "uri": img.get("uri")})
|
| 178 |
+
if paragraphs:
|
| 179 |
+
out["description"] = "\n".join(paragraphs)
|
| 180 |
+
|
| 181 |
+
# --- videos (product videos, when present)
|
| 182 |
+
vids = pm.get("videos") or {}
|
| 183 |
+
for v in (vids.values() if isinstance(vids, dict) else vids):
|
| 184 |
+
u = first_url(v if isinstance(v, dict) else {}) or (v.get("play_addr", {}).get("url_list") or [None])[0] if isinstance(v, dict) else None
|
| 185 |
+
if u:
|
| 186 |
+
out["videos"].append({"url": u})
|
| 187 |
+
|
| 188 |
+
# --- SKUs
|
| 189 |
+
for sku in pm.get("skus") or []:
|
| 190 |
+
sp = sku_prices.get(str(sku.get("sku_id"))) or {}
|
| 191 |
+
out["skus"].append({
|
| 192 |
+
"sku_id": sku.get("sku_id"),
|
| 193 |
+
"name": sku.get("sku_name"),
|
| 194 |
+
"options": {p.get("sku_property_name"): p.get("sku_property_value_name") for p in sku.get("property_pairs") or []},
|
| 195 |
+
"price": sp.get("sale_price_decimal"),
|
| 196 |
+
"currency": sp.get("currency_name"),
|
| 197 |
+
"available_quantity": (sku.get("sku_quantity") or {}).get("available_quantity"),
|
| 198 |
+
"in_stock": sku.get("sku_stock_status") == 1,
|
| 199 |
+
"gtin": (sku.get("gtin") or {}).get("gtin_code"),
|
| 200 |
+
"weight": sku.get("weight"),
|
| 201 |
+
"dimension": sku.get("dimension"),
|
| 202 |
+
})
|
| 203 |
+
|
| 204 |
+
# --- attributes (product + sale properties)
|
| 205 |
+
attrs: dict[str, Any] = {}
|
| 206 |
+
for prop in (pm.get("product_properties") or []) + (pm.get("sale_properties") or []):
|
| 207 |
+
name = prop.get("property_name")
|
| 208 |
+
vals = [v.get("property_value_name") for v in prop.get("property_values") or [] if v.get("property_value_name")]
|
| 209 |
+
if name and vals:
|
| 210 |
+
attrs[name] = vals if len(vals) > 1 else vals[0]
|
| 211 |
+
out["attributes"] = attrs
|
| 212 |
+
brand_key = next((k for k in attrs if k.lower() == "brand"), None)
|
| 213 |
+
out["brand"] = attrs.get(brand_key) if brand_key else None
|
| 214 |
+
|
| 215 |
+
# --- rating & reviews
|
| 216 |
+
rm = core.get("review_model") or {}
|
| 217 |
+
rinfo = pinfo.get("review_info") or {}
|
| 218 |
+
out["rating"] = {
|
| 219 |
+
"score": rm.get("product_overall_score") or (rinfo.get("review_ratings") or {}).get("overall_score"),
|
| 220 |
+
"count": rm.get("product_review_count") or rinfo.get("total_reviews"),
|
| 221 |
+
"distribution": (rinfo.get("review_ratings") or {}).get("rating_result"),
|
| 222 |
+
"sample_reviews": [
|
| 223 |
+
{
|
| 224 |
+
"rating": r.get("review_rating"),
|
| 225 |
+
"text": clean_text(r.get("review_text")),
|
| 226 |
+
"reviewer": r.get("reviewer_name"),
|
| 227 |
+
"verified": r.get("is_verified_purchase"),
|
| 228 |
+
"images": [first_url(x) for x in r.get("review_images") or [] if first_url(x)],
|
| 229 |
+
}
|
| 230 |
+
for r in rinfo.get("product_reviews") or []
|
| 231 |
+
],
|
| 232 |
+
}
|
| 233 |
+
|
| 234 |
+
# --- seller / shop
|
| 235 |
+
sm = core.get("seller_model") or {}
|
| 236 |
+
shop = comps.get("shop_info") or pinfo.get("shop_info") or {}
|
| 237 |
+
comp = (core.get("safety_model") or {}).get("business_compliance_info") or {}
|
| 238 |
+
out["seller"] = {
|
| 239 |
+
"seller_id": pm.get("seller_id"),
|
| 240 |
+
"shop_name": sm.get("shop_name") or shop.get("shop_name"),
|
| 241 |
+
"shop_link": shop.get("shop_link"),
|
| 242 |
+
"shop_rating": shop.get("shop_rating"),
|
| 243 |
+
"followers": shop.get("followers_count"),
|
| 244 |
+
"shop_sold_count": shop.get("sold_count"),
|
| 245 |
+
"official": (shop.get("shop_identity_label") or {}).get("identity_label_text"),
|
| 246 |
+
"logo": first_url(sm.get("shop_logo") or shop.get("shop_logo")),
|
| 247 |
+
"business_name": comp.get("business_name"),
|
| 248 |
+
"business_address": comp.get("business_address"),
|
| 249 |
+
}
|
| 250 |
+
if not out["brand"] and out["seller"].get("business_name"):
|
| 251 |
+
out["brand"] = out["seller"]["shop_name"]
|
| 252 |
+
|
| 253 |
+
# --- shipping
|
| 254 |
+
pkgs = (core.get("logistic_model") or {}).get("pkg_of_service") or {}
|
| 255 |
+
if pkgs:
|
| 256 |
+
p0 = next(iter(pkgs.values()))
|
| 257 |
+
out["shipping"] = {
|
| 258 |
+
"delivery_min_days": p0.get("delivery_min_days"),
|
| 259 |
+
"delivery_max_days": p0.get("delivery_max_days"),
|
| 260 |
+
"shipping_fee": p0.get("shipping_fee"),
|
| 261 |
+
"currency": p0.get("currency"),
|
| 262 |
+
}
|
| 263 |
+
|
| 264 |
+
# --- AI marketing copy the page ships (bullets / FAQ)
|
| 265 |
+
aigc = pinfo.get("aigc_info") or {}
|
| 266 |
+
if aigc.get("bullet_points"):
|
| 267 |
+
out["bullet_points"] = [clean_text(b) for b in re.split(r"\n|•", aigc["bullet_points"]) if clean_text(b)]
|
| 268 |
+
if aigc.get("faq"):
|
| 269 |
+
out["faq"] = clean_text(aigc["faq"])
|
| 270 |
+
return out
|
| 271 |
+
|
| 272 |
+
|
| 273 |
+
# ----------------------------------------------------------------------- official sites
|
| 274 |
+
def extract_shopify_json(client: httpx.Client, url: str, out: dict[str, Any]) -> bool:
|
| 275 |
+
m = re.search(r"(https?://[^/]+)(?:/[a-z]{2}(?:-[a-z]{2})?)?/products/([A-Za-z0-9\-_.%]+)", url)
|
| 276 |
+
if not m:
|
| 277 |
+
return False
|
| 278 |
+
try:
|
| 279 |
+
r = client.get(f"{m.group(1)}/products/{m.group(2)}.json", headers=HEADERS, follow_redirects=True, timeout=30.0)
|
| 280 |
+
if r.status_code != 200 or "application/json" not in r.headers.get("content-type", ""):
|
| 281 |
+
return False
|
| 282 |
+
p = r.json()["product"]
|
| 283 |
+
except Exception:
|
| 284 |
+
return False
|
| 285 |
+
out["title"] = out["title"] or p.get("title")
|
| 286 |
+
out["brand"] = out["brand"] or p.get("vendor")
|
| 287 |
+
out["description"] = out["description"] or html_to_text(p.get("body_html"))
|
| 288 |
+
variants = p.get("variants") or []
|
| 289 |
+
if variants and not out["price"]:
|
| 290 |
+
out["price"] = {"amount": variants[0].get("price"), "currency": None, "display": variants[0].get("price")}
|
| 291 |
+
out["skus"] = [
|
| 292 |
+
{"sku_id": v.get("id"), "name": v.get("title"), "price": v.get("price"), "sku": v.get("sku"), "in_stock": v.get("available")}
|
| 293 |
+
for v in variants
|
| 294 |
+
]
|
| 295 |
+
for i, img in enumerate(p.get("images") or []):
|
| 296 |
+
out["images"].append({"url": img.get("src"), "kind": "gallery", "index": i, "width": img.get("width"), "height": img.get("height")})
|
| 297 |
+
out["notes"].append("shopify products.json")
|
| 298 |
+
return True
|
| 299 |
+
|
| 300 |
+
|
| 301 |
+
def extract_jsonld(soup: BeautifulSoup, out: dict[str, Any]) -> None:
|
| 302 |
+
for s in soup.find_all("script", type="application/ld+json"):
|
| 303 |
+
try:
|
| 304 |
+
d = json.loads(s.string or "")
|
| 305 |
+
except Exception:
|
| 306 |
+
continue
|
| 307 |
+
items = d if isinstance(d, list) else [d]
|
| 308 |
+
items += [x for it in items if isinstance(it, dict) for x in it.get("@graph", [])]
|
| 309 |
+
for it in items:
|
| 310 |
+
if not isinstance(it, dict) or "Product" not in str(it.get("@type")):
|
| 311 |
+
continue
|
| 312 |
+
out["title"] = out["title"] or clean_text(it.get("name"))
|
| 313 |
+
out["description"] = out["description"] or html_to_text(it.get("description"))
|
| 314 |
+
brand = it.get("brand")
|
| 315 |
+
out["brand"] = out["brand"] or (brand.get("name") if isinstance(brand, dict) else brand)
|
| 316 |
+
out["sku"] = out.get("sku") or it.get("sku")
|
| 317 |
+
imgs = it.get("image") or []
|
| 318 |
+
for u in ([imgs] if isinstance(imgs, str) else imgs):
|
| 319 |
+
u = u.get("url") if isinstance(u, dict) else u
|
| 320 |
+
if u and all(u != x["url"] for x in out["images"]):
|
| 321 |
+
out["images"].append({"url": u, "kind": "gallery", "width": None, "height": None})
|
| 322 |
+
offers = it.get("offers")
|
| 323 |
+
offers = offers[0] if isinstance(offers, list) and offers else offers
|
| 324 |
+
if isinstance(offers, dict) and not out["price"]:
|
| 325 |
+
price = offers.get("price") or offers.get("lowPrice")
|
| 326 |
+
if price:
|
| 327 |
+
out["price"] = {"amount": str(price), "currency": offers.get("priceCurrency"), "display": f"{price} {offers.get('priceCurrency') or ''}".strip(), "availability": offers.get("availability")}
|
| 328 |
+
rating = it.get("aggregateRating")
|
| 329 |
+
if isinstance(rating, dict):
|
| 330 |
+
out["rating"] = {"score": rating.get("ratingValue"), "count": rating.get("reviewCount") or rating.get("ratingCount")}
|
| 331 |
+
out["notes"].append("json-ld Product")
|
| 332 |
+
|
| 333 |
+
|
| 334 |
+
def extract_meta(soup: BeautifulSoup, out: dict[str, Any]) -> None:
|
| 335 |
+
og = {m.get("property"): m.get("content") for m in soup.find_all("meta", property=True)}
|
| 336 |
+
out["title"] = out["title"] or clean_text(og.get("og:title")) or (clean_text(soup.title.get_text()) if soup.title else None)
|
| 337 |
+
out["description"] = out["description"] or clean_text(og.get("og:description")) or clean_text((soup.find("meta", attrs={"name": "description"}) or {}).get("content"))
|
| 338 |
+
if og.get("og:image") and all(og["og:image"] != x["url"] for x in out["images"]):
|
| 339 |
+
out["images"].append({"url": og["og:image"], "kind": "og", "width": og.get("og:image:width"), "height": og.get("og:image:height")})
|
| 340 |
+
if not out["price"]:
|
| 341 |
+
amount = og.get("og:price:amount") or og.get("product:price:amount")
|
| 342 |
+
if not amount:
|
| 343 |
+
ip = soup.find(attrs={"itemprop": "price"})
|
| 344 |
+
amount = ip.get("content") if ip else None
|
| 345 |
+
if amount:
|
| 346 |
+
out["price"] = {"amount": amount, "currency": og.get("og:price:currency") or og.get("product:price:currency"), "display": amount}
|
| 347 |
+
if not out["price"]:
|
| 348 |
+
m = re.search(r"(?:\$|€|£|¥)\s?\d[\d,]*(?:\.\d{2})?", soup.get_text(" "))
|
| 349 |
+
if m:
|
| 350 |
+
out["price"] = {"amount": m.group(0), "currency": None, "display": m.group(0), "note": "regex from page text"}
|
| 351 |
+
|
| 352 |
+
|
| 353 |
+
def extract_img_tags(soup: BeautifulSoup, base: str, out: dict[str, Any], limit: int) -> None:
|
| 354 |
+
seen = {x["url"] for x in out["images"]}
|
| 355 |
+
skip = re.compile(r"(logo|icon|sprite|badge|flag|payment|pixel|tracking|\.svg|\.gif|1x1|avatar)", re.I)
|
| 356 |
+
for img in soup.find_all("img"):
|
| 357 |
+
src = img.get("src") or img.get("data-src") or img.get("data-original")
|
| 358 |
+
if not src and img.get("srcset"):
|
| 359 |
+
src = img["srcset"].split(",")[-1].strip().split(" ")[0]
|
| 360 |
+
if not src:
|
| 361 |
+
continue
|
| 362 |
+
u = urljoin(base, src.strip())
|
| 363 |
+
if u in seen or skip.search(u) or u.startswith("data:"):
|
| 364 |
+
continue
|
| 365 |
+
w = img.get("width")
|
| 366 |
+
if w and str(w).isdigit() and int(w) < 200:
|
| 367 |
+
continue
|
| 368 |
+
seen.add(u)
|
| 369 |
+
out["images"].append({"url": u, "kind": "img-tag", "alt": clean_text(img.get("alt")), "width": w, "height": img.get("height")})
|
| 370 |
+
if len(out["images"]) >= limit:
|
| 371 |
+
break
|
| 372 |
+
|
| 373 |
+
|
| 374 |
+
def extract_generic(client: httpx.Client, url: str, max_images: int) -> dict[str, Any]:
|
| 375 |
+
out = blank_product(url, "web")
|
| 376 |
+
r = fetch(client, url)
|
| 377 |
+
out["final_url"] = str(r.url)
|
| 378 |
+
if r.status_code in (401, 403, 429, 503) or len(r.text) < 1500:
|
| 379 |
+
out["blocked"] = True
|
| 380 |
+
out["notes"].append(f"HTTP {r.status_code}, {len(r.text)} bytes — bot protection / JS-only page; needs a real browser (Playwright)")
|
| 381 |
+
soup = BeautifulSoup(r.text, "html.parser")
|
| 382 |
+
extract_shopify_json(client, str(r.url), out)
|
| 383 |
+
extract_jsonld(soup, out)
|
| 384 |
+
extract_meta(soup, out)
|
| 385 |
+
if len(out["images"]) < 3:
|
| 386 |
+
extract_img_tags(soup, str(r.url), out, max_images)
|
| 387 |
+
if out["title"] and out["blocked"] and len(r.text) >= 1500:
|
| 388 |
+
out["blocked"] = False
|
| 389 |
+
return out
|
| 390 |
+
|
| 391 |
+
|
| 392 |
+
# --------------------------------------------------------------------------- downloads
|
| 393 |
+
def download_images(client: httpx.Client, out: dict[str, Any], out_dir: Path, max_images: int) -> None:
|
| 394 |
+
img_dir = out_dir / "images"
|
| 395 |
+
img_dir.mkdir(parents=True, exist_ok=True)
|
| 396 |
+
counters: dict[str, int] = {}
|
| 397 |
+
done = 0
|
| 398 |
+
for img in out["images"]:
|
| 399 |
+
if done >= max_images:
|
| 400 |
+
break
|
| 401 |
+
kind = img.get("kind", "img")
|
| 402 |
+
counters[kind] = counters.get(kind, 0) + 1
|
| 403 |
+
try:
|
| 404 |
+
r = client.get(img["url"], headers={"User-Agent": UA, "Referer": out["url"]}, follow_redirects=True, timeout=40.0)
|
| 405 |
+
if r.status_code != 200 or not r.headers.get("content-type", "").startswith("image/"):
|
| 406 |
+
img["local"] = None
|
| 407 |
+
img["error"] = f"HTTP {r.status_code} {r.headers.get('content-type', '')}"
|
| 408 |
+
continue
|
| 409 |
+
ext = mimetypes.guess_extension(r.headers["content-type"].split(";")[0]) or ".img"
|
| 410 |
+
path = img_dir / f"{kind}_{counters[kind]:02d}{ext}"
|
| 411 |
+
path.write_bytes(r.content)
|
| 412 |
+
img["local"] = str(path)
|
| 413 |
+
img["bytes"] = len(r.content)
|
| 414 |
+
done += 1
|
| 415 |
+
except Exception as exc: # keep going; one dead CDN link must not kill the run
|
| 416 |
+
img["local"] = None
|
| 417 |
+
img["error"] = str(exc)[:120]
|
| 418 |
+
|
| 419 |
+
|
| 420 |
+
# -------------------------------------------------------------------------------- main
|
| 421 |
+
def main() -> int:
|
| 422 |
+
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
| 423 |
+
ap.add_argument("url")
|
| 424 |
+
ap.add_argument("--out", default=None, help="output dir (default: data/products/<slug>)")
|
| 425 |
+
ap.add_argument("--max-images", type=int, default=30)
|
| 426 |
+
ap.add_argument("--no-images", action="store_true")
|
| 427 |
+
args = ap.parse_args()
|
| 428 |
+
|
| 429 |
+
with httpx.Client() as client:
|
| 430 |
+
if is_tiktok_shop(args.url):
|
| 431 |
+
r = fetch(client, args.url)
|
| 432 |
+
product = extract_tiktok(r.text, args.url)
|
| 433 |
+
product["final_url"] = str(r.url)
|
| 434 |
+
else:
|
| 435 |
+
product = extract_generic(client, args.url, args.max_images)
|
| 436 |
+
|
| 437 |
+
slug = re.sub(r"[^a-z0-9]+", "-", (product.get("title") or "product").lower())[:60].strip("-")
|
| 438 |
+
out_dir = Path(args.out) if args.out else Path(__file__).resolve().parent / "results" / slug
|
| 439 |
+
out_dir.mkdir(parents=True, exist_ok=True)
|
| 440 |
+
if not args.no_images:
|
| 441 |
+
download_images(client, product, out_dir, args.max_images)
|
| 442 |
+
|
| 443 |
+
(out_dir / "product.json").write_text(json.dumps(product, ensure_ascii=False, indent=2), encoding="utf-8")
|
| 444 |
+
|
| 445 |
+
print(f"platform : {product['platform']}{' [BLOCKED]' if product.get('blocked') else ''}")
|
| 446 |
+
print(f"title : {product.get('title')}")
|
| 447 |
+
print(f"brand : {product.get('brand')} price: {(product.get('price') or {}).get('display')}")
|
| 448 |
+
if product.get("rating"):
|
| 449 |
+
print(f"rating : {product['rating'].get('score')} ({product['rating'].get('count')} reviews) sold: {product.get('sold_count')}")
|
| 450 |
+
if product.get("seller"):
|
| 451 |
+
print(f"seller : {product['seller'].get('shop_name')} {product['seller'].get('official') or ''}")
|
| 452 |
+
saved = sum(1 for i in product["images"] if i.get("local"))
|
| 453 |
+
print(f"images : {len(product['images'])} found, {saved} downloaded -> {out_dir / 'images'}")
|
| 454 |
+
print(f"desc : {(product.get('description') or '')[:160]}")
|
| 455 |
+
for n in product.get("notes", []):
|
| 456 |
+
print(f"note : {n}")
|
| 457 |
+
print(f"json : {out_dir / 'product.json'}")
|
| 458 |
+
return 2 if product.get("blocked") else 0
|
| 459 |
+
|
| 460 |
+
|
| 461 |
+
if __name__ == "__main__":
|
| 462 |
+
sys.exit(main())
|
web/results/tt_bissell/images/description_01.jpg
ADDED
|
Git LFS Details
|
web/results/tt_bissell/images/gallery_01.webp
ADDED
|
web/results/tt_bissell/images/gallery_02.webp
ADDED
|
Git LFS Details
|
web/results/tt_bissell/images/gallery_03.webp
ADDED
|
Git LFS Details
|
web/results/tt_bissell/images/gallery_04.webp
ADDED
|
Git LFS Details
|
web/results/tt_bissell/images/gallery_05.webp
ADDED
|
web/results/tt_bissell/images/gallery_06.webp
ADDED
|
web/results/tt_bissell/images/gallery_07.webp
ADDED
|
Git LFS Details
|
web/results/tt_bissell/images/gallery_08.webp
ADDED
|
Git LFS Details
|
web/results/tt_bissell/product.json
ADDED
|
@@ -0,0 +1,222 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"platform": "tiktok_shop",
|
| 3 |
+
"url": "https://shop.tiktok.com/us/pdp/1729467215382352745?_t=ZT-98aNj03V4X7",
|
| 4 |
+
"title": "BISSELL® Little Green® Portable Carpet & Upholstery Cleaner for Pet Stains, Car Interiors, Rugs, Sofas & Deep Spot Cleaning at Home",
|
| 5 |
+
"brand": "BISSELL Clean",
|
| 6 |
+
"price": {
|
| 7 |
+
"amount": "129.99",
|
| 8 |
+
"currency": "USD",
|
| 9 |
+
"symbol": "$",
|
| 10 |
+
"display": "$129.99"
|
| 11 |
+
},
|
| 12 |
+
"description": "Effortlessly erase spots, stains, and spills from upholstery, area rugs, carpet, and auto interiors with the Little Green Portable Carpet & Upholstery Cleaner, our social media sensation with over one million sold.\nRigorously tested for real pet messes\nLittle Green portable deep cleaner is a viral best-seller that carries the Pet Proven certification for its stand-out cleaning performance on yuck messes and tough stains. Its powerful suction and strong spray make it the right tool for cleaning up after pets. Pet Proven products are rigorously tested to meet the unique cleaning needs of pet parents.\nTackle various surfaces\nWith its compact design, this portable upholstery and carpet cleaner delivers quick and thorough pet mess and stain cleanup on various surfaces found in your home or car, including pet beds, stairs, auto interiors, and carpet.\nExtended reach for easy cleaning\nWith a 4.5-foot hose and a 15-foot cord, this machine can easily clean hard-to-reach areas. Spray, scrub, and suction away tough spots, stains, and spills throughout your car and home.\nLarge capacity for efficient cleaning\nSpend more time cleaning and less refilling the clean tank and emptying the dirty tank with the machine's large dual tanks, streamlining your cleaning routine with no interruptions.\nTake on stains\nEach purchase comes with a trial size 8-ounce bottle of Little Green Formula with OXY along with a Spraying Crevice Tool, a 3\"\" Tough Stain Tool, and a HydroRinse Self-Cleaning Hose Tool to combat various spills and pet messes.",
|
| 13 |
+
"bullet_points": [
|
| 14 |
+
"Portable carpet and upholstery cleaner for spots, stains, and spills",
|
| 15 |
+
"Pet Proven certification for tested cleaning performance on pet messes",
|
| 16 |
+
"Compact design for use on rugs, sofas, stairs, pet beds, and car interiors",
|
| 17 |
+
"4.5-foot hose and 15-foot cord for extended reach",
|
| 18 |
+
"Includes 8-ounce trial size Little Green Formula with OXY plus cleaning tools"
|
| 19 |
+
],
|
| 20 |
+
"images": [
|
| 21 |
+
{
|
| 22 |
+
"url": "https://p19-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/f723b50fb012422686e9541f6f640f38~tplv-fhlh96nyum-crop-webp:1300:1300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 23 |
+
"kind": "gallery",
|
| 24 |
+
"index": 0,
|
| 25 |
+
"width": 1300,
|
| 26 |
+
"height": 1300,
|
| 27 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/f723b50fb012422686e9541f6f640f38",
|
| 28 |
+
"local": "data/products/tt_bissell/images/gallery_01.webp",
|
| 29 |
+
"bytes": 56880
|
| 30 |
+
},
|
| 31 |
+
{
|
| 32 |
+
"url": "https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/33b04b065ed1463a923934d3dfbaad26~tplv-fhlh96nyum-crop-webp:1500:1500.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 33 |
+
"kind": "gallery",
|
| 34 |
+
"index": 1,
|
| 35 |
+
"width": 1500,
|
| 36 |
+
"height": 1500,
|
| 37 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/33b04b065ed1463a923934d3dfbaad26",
|
| 38 |
+
"local": "data/products/tt_bissell/images/gallery_02.webp",
|
| 39 |
+
"bytes": 133054
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"url": "https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/732833fb1a9943adb30d15254a69b327~tplv-fhlh96nyum-crop-webp:1500:1500.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 43 |
+
"kind": "gallery",
|
| 44 |
+
"index": 2,
|
| 45 |
+
"width": 1500,
|
| 46 |
+
"height": 1500,
|
| 47 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/732833fb1a9943adb30d15254a69b327",
|
| 48 |
+
"local": "data/products/tt_bissell/images/gallery_03.webp",
|
| 49 |
+
"bytes": 347006
|
| 50 |
+
},
|
| 51 |
+
{
|
| 52 |
+
"url": "https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/8452888ff005464da23372106fe1106e~tplv-fhlh96nyum-crop-webp:1500:1500.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 53 |
+
"kind": "gallery",
|
| 54 |
+
"index": 3,
|
| 55 |
+
"width": 1500,
|
| 56 |
+
"height": 1500,
|
| 57 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/8452888ff005464da23372106fe1106e",
|
| 58 |
+
"local": "data/products/tt_bissell/images/gallery_04.webp",
|
| 59 |
+
"bytes": 119686
|
| 60 |
+
},
|
| 61 |
+
{
|
| 62 |
+
"url": "https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/2c7a0022def44ffdad4017672637c63c~tplv-fhlh96nyum-crop-webp:1500:1500.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 63 |
+
"kind": "gallery",
|
| 64 |
+
"index": 4,
|
| 65 |
+
"width": 1500,
|
| 66 |
+
"height": 1500,
|
| 67 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/2c7a0022def44ffdad4017672637c63c",
|
| 68 |
+
"local": "data/products/tt_bissell/images/gallery_05.webp",
|
| 69 |
+
"bytes": 86628
|
| 70 |
+
},
|
| 71 |
+
{
|
| 72 |
+
"url": "https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/a38e6cfabb5c47c1871185b167344776~tplv-fhlh96nyum-crop-webp:1500:1500.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 73 |
+
"kind": "gallery",
|
| 74 |
+
"index": 5,
|
| 75 |
+
"width": 1500,
|
| 76 |
+
"height": 1500,
|
| 77 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/a38e6cfabb5c47c1871185b167344776",
|
| 78 |
+
"local": "data/products/tt_bissell/images/gallery_06.webp",
|
| 79 |
+
"bytes": 41364
|
| 80 |
+
},
|
| 81 |
+
{
|
| 82 |
+
"url": "https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/10d1abb2cd9a4bb0a4c08cf02601e024~tplv-fhlh96nyum-crop-webp:1500:1500.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 83 |
+
"kind": "gallery",
|
| 84 |
+
"index": 6,
|
| 85 |
+
"width": 1500,
|
| 86 |
+
"height": 1500,
|
| 87 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/10d1abb2cd9a4bb0a4c08cf02601e024",
|
| 88 |
+
"local": "data/products/tt_bissell/images/gallery_07.webp",
|
| 89 |
+
"bytes": 113126
|
| 90 |
+
},
|
| 91 |
+
{
|
| 92 |
+
"url": "https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/d2f36f67a0ac417e907c7c4f4a7b73cb~tplv-fhlh96nyum-crop-webp:1500:1500.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 93 |
+
"kind": "gallery",
|
| 94 |
+
"index": 7,
|
| 95 |
+
"width": 1500,
|
| 96 |
+
"height": 1500,
|
| 97 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/d2f36f67a0ac417e907c7c4f4a7b73cb",
|
| 98 |
+
"local": "data/products/tt_bissell/images/gallery_08.webp",
|
| 99 |
+
"bytes": 201334
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"url": "https://p19-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/71aca41f3fd647f299502ffe58cd8bb6~tplv-fhlh96nyum-origin-jpeg.jpeg?dr=12178&t=555f072d&ps=933b5bde&shp=a3510d86&shcp=6ce186a1&idc=useast5&from=2739998086",
|
| 103 |
+
"kind": "description",
|
| 104 |
+
"index": 0,
|
| 105 |
+
"width": 2449,
|
| 106 |
+
"height": 2449,
|
| 107 |
+
"uri": "tos-useast5-i-omjb5zjo8w-tx/71aca41f3fd647f299502ffe58cd8bb6",
|
| 108 |
+
"local": "data/products/tt_bissell/images/description_01.jpg",
|
| 109 |
+
"bytes": 187380
|
| 110 |
+
}
|
| 111 |
+
],
|
| 112 |
+
"videos": [],
|
| 113 |
+
"seller": {
|
| 114 |
+
"seller_id": "7495710630069177193",
|
| 115 |
+
"shop_name": "BISSELL Clean",
|
| 116 |
+
"shop_link": "https://shop.tiktok.com/us/store/bissell-clean/7495710630069177193",
|
| 117 |
+
"shop_rating": "4.6",
|
| 118 |
+
"followers": "138534",
|
| 119 |
+
"shop_sold_count": 54556,
|
| 120 |
+
"official": "OFFICIAL SHOP",
|
| 121 |
+
"logo": "https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/3caf9ac3931249dd9cdb75b9f2e06286~tplv-fhlh96nyum-resize-png:300:300.png?dr=12184&t=555f072d&ps=933b5bde&shp=905da467&shcp=6ce186a1&idc=useast5&from=2422056039",
|
| 122 |
+
"business_name": "BISSELL HOMECARE, INC.",
|
| 123 |
+
"business_address": "2345 Walker Ave NW, Grand Rapids, Michigan, 49544, The United States of America"
|
| 124 |
+
},
|
| 125 |
+
"rating": {
|
| 126 |
+
"score": 4.5,
|
| 127 |
+
"count": "370",
|
| 128 |
+
"distribution": {
|
| 129 |
+
"1": "24",
|
| 130 |
+
"2": "6",
|
| 131 |
+
"3": "12",
|
| 132 |
+
"4": "33",
|
| 133 |
+
"5": "295"
|
| 134 |
+
},
|
| 135 |
+
"sample_reviews": [
|
| 136 |
+
{
|
| 137 |
+
"rating": 5,
|
| 138 |
+
"text": "I really love this machine. It really works great",
|
| 139 |
+
"reviewer": "c**r",
|
| 140 |
+
"verified": true,
|
| 141 |
+
"images": [
|
| 142 |
+
"https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/e96fbca726df42cb925ecc25689d7f78~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 143 |
+
"https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/e0eef89acabe48dc8c8fe30009f80085~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 144 |
+
"https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/0d69813bec044de0bd0fbbfd52acd018~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839"
|
| 145 |
+
]
|
| 146 |
+
},
|
| 147 |
+
{
|
| 148 |
+
"rating": 1,
|
| 149 |
+
"text": "Would not such up any of the water the whole machine is leaking and has my steps super wet",
|
| 150 |
+
"reviewer": "2**Y",
|
| 151 |
+
"verified": true,
|
| 152 |
+
"images": [
|
| 153 |
+
"https://p19-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/1e22688b24fc482c9fab184e7185afcd~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 154 |
+
"https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/24f78731fa074482a4c4f68b36c04dc2~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 155 |
+
"https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/e6cd349f73c44886a698f4de17bc2c93~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 156 |
+
"https://p19-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/d7ae8447f6724a5cba78cb723a2c0bcf~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839",
|
| 157 |
+
"https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/1bd7213747dc403c90928513ae7cb81e~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839"
|
| 158 |
+
]
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"rating": 5,
|
| 162 |
+
"text": "These couch have been in the family for a very long time. Always had a cover on them but they have been without covers for a year. I decided to purchase this BISSELL and it works so well. Does use very much water but the clean is great. I was shocked with how much dirt came out of this fabric. It does make noise but I don’t care. It cleans well, the sound is the least of my worries. Easy to use, easy to dispose of dirty water and add new solution and water. It’s good quality. I was able to finish one couch and still had water and solution left over. It’s durable.",
|
| 163 |
+
"reviewer": "C**l **",
|
| 164 |
+
"verified": true,
|
| 165 |
+
"images": [
|
| 166 |
+
"https://p16-oec-general-useast5.ttcdn-us.com/tos-useast5-i-omjb5zjo8w-tx/2179c06105a940169523698691290f98~tplv-fhlh96nyum-crop-webp:300:300.webp?dr=12190&t=555f072d&ps=933b5bde&shp=8dbd94bf&shcp=607f11de&idc=useast5&from=2378011839"
|
| 167 |
+
]
|
| 168 |
+
}
|
| 169 |
+
]
|
| 170 |
+
},
|
| 171 |
+
"sold_count": "5123",
|
| 172 |
+
"skus": [
|
| 173 |
+
{
|
| 174 |
+
"sku_id": "1729467215382418281",
|
| 175 |
+
"name": "default",
|
| 176 |
+
"options": {
|
| 177 |
+
"Specifications": "Default"
|
| 178 |
+
},
|
| 179 |
+
"price": "129.99",
|
| 180 |
+
"currency": "USD",
|
| 181 |
+
"available_quantity": 155544,
|
| 182 |
+
"in_stock": true,
|
| 183 |
+
"gtin": "011120007503",
|
| 184 |
+
"weight": {
|
| 185 |
+
"weight": "13.39",
|
| 186 |
+
"unit": 2
|
| 187 |
+
},
|
| 188 |
+
"dimension": {
|
| 189 |
+
"length": "13",
|
| 190 |
+
"width": "9",
|
| 191 |
+
"height": "19",
|
| 192 |
+
"unit": 2
|
| 193 |
+
}
|
| 194 |
+
}
|
| 195 |
+
],
|
| 196 |
+
"attributes": {
|
| 197 |
+
"CA prop 65: repro. chems": "No",
|
| 198 |
+
"CA prop 65: carcinogens": "No",
|
| 199 |
+
"Dangerous goods or hazardous materials": "No",
|
| 200 |
+
"Volume capacity (l)": "1.42L",
|
| 201 |
+
"Cord length": "15 ft",
|
| 202 |
+
"Corded or cordless": "Corded",
|
| 203 |
+
"Specifications": "Default"
|
| 204 |
+
},
|
| 205 |
+
"shipping": {
|
| 206 |
+
"delivery_min_days": 4,
|
| 207 |
+
"delivery_max_days": 7,
|
| 208 |
+
"shipping_fee": "9.99",
|
| 209 |
+
"currency": "USD"
|
| 210 |
+
},
|
| 211 |
+
"categories": [
|
| 212 |
+
"Household Appliances",
|
| 213 |
+
"Home Appliances",
|
| 214 |
+
"Vacuums & Floor Care",
|
| 215 |
+
"Carpet Washers"
|
| 216 |
+
],
|
| 217 |
+
"blocked": false,
|
| 218 |
+
"notes": [],
|
| 219 |
+
"product_id": "1729467215382352745",
|
| 220 |
+
"faq": "Q:What surfaces can the BISSELL Little Green cleaner be used on? A:It is designed for upholstery, area rugs, carpet, pet beds, stairs, and auto interiors. Q:Is the BISSELL Little Green cleaner suitable for pet messes? A:Yes, it carries Pet Proven certification and is tested for real pet messes. Q:Is this cleaner corded or cordless? A:It is corded. Q:What is included with the cleaner? A:It includes a trial size 8-ounce bottle of Little Green Formula with OXY, a Spraying Crevice Tool, a 3\" Tough Stain Tool, and a HydroRinse Self-Cleaning Hose Tool. Q:What is the cord length? A:The cord length is 15 feet.",
|
| 221 |
+
"final_url": "https://shop.tiktok.com/us/pdp/1729467215382352745?_d=ed5m2he17k03h2&_svg=3&checksum=77158044e3da654de6949bc0d718e390185ef8cfee0a98544268b5ed917f67c4&encode_params=MIIBUwQMNHPNYH9OaklGA1LRBIIBLwjBbXyCa3BYHx6uN8PPx_EbKcaRXZJxDPtkKS2os20M6UCHwaZEhDkSHr1BKko1dbvVGpdTtQlIIZgwTZBU_keTgYcpytL6B6a7pxNKVnAtgv6gydx9ZxrrxIHUctyd0YRXkDTFN1CNrvSoyOpj0-n_XrbjDUKnjyB6n0yOfn7k_jW-g-n3cWhMo3IH4koppR1jDy_FR_ax_fHc26JH0TA68qPr7TQSoS9COxujXAQx4DHoXSMqvfdv3bMatArT9lgVLpenhxy0kY76kBxjbgVUlaGhbwNO6CgIm9jg-oxBCr42v4Tx6f35TU85aLhnunvtZ1WVnwqi-4a-nqLpNFrHXy6Cilcxv0bJ0EJYkQxMtW5XfnM3iJirlEZ-30AallArZGrwcci5vF1JSkDlyQQQ4LyRvO2I7WUPxPhwGCWepg%3D%3D&og_info=%7B%22title%22%3A%22BISSELL%C2%AE+Little+Green%C2%AE+Portable+Carpet+%26+Upholstery+Cleaner+for+Pet+Stains%2C+Car+Interiors%2C+Rugs%2C+Sofas+%26+Deep+Spot+Cleaning+at+Home%22%2C%22image%22%3A%22https%3A%5C%2F%5C%2Fp16-oec-general-useast5.ttcdn-us.com%5C%2Ftos-useast5-i-omjb5zjo8w-tx%5C%2Ff723b50fb012422686e9541f6f640f38~tplv-fhlh96nyum-resize-webp%3A260%3A260.webp%3Fdr%3D12185%26t%3D555f072d%26ps%3D933b5bde%26shp%3D7745054a%26shcp%3D9b759fb9%26idc%3Duseast5%26from%3D2001012042%22%7D&panel_source_v2=share_panel&sec_user_id=MS4wLjABAAAAsc7ZFlveT8XTgmahd56mKXMjcMEH-Dpb42RZAYOs-41SBimWfu02Mc1xhbrGtsSc&share_app_id=1233&share_enter_from=&share_link_id=EDFDC162-F3BA-43D2-B98E-6586D1EF40EA&share_region=US&social_share_type=15×tamp=1785795900&trackParams=%7B%22device_id%22%3A%227347079598221395499%22%2C%22enable_shop_tab_popup%22%3A1%2C%22traffic_source_list%22%3A%5B7%5D%2C%22traffic_source%22%3A7%2C%22enter_from_info%22%3A%22product_share_outside%22%2C%22source_page_type%22%3A%22product_share%22%7D&tt_from=copy&u_code=ED5M313%3AEDKBH1&ug_btm=b0%2Cb6661&unique_id=goat4show&user_id=7347080110320878635&utm_campaign=client_share&utm_medium=ios&utm_source=copy"
|
| 222 |
+
}
|