Websites & SEO
Website to RAG Chunks Crawler
Tested and ready
Crawl a whole website (or docs/help center) and get ready-to-embed Markdown chunks: heading path, token count, chunk ID, source URL. Incremental mode returns only new or changed pages since the last run, so you re-embed only what changed. No browser, cheap, robots.txt-aware.
What data it extracts
- url
- Chunk Index
- Heading Path
- text
- tokens
- Chunk Id
- Page Title
- Page Hash
- Change Status
- depth
- markdown
- lang
Output format
Each run writes an Apify dataset you can download as JSON, CSV or Excel, or read through the Apify API.
Schedule
Run it on demand in the Apify Console, or put it on an hourly, daily or weekly schedule.
Price
$0.90 per 1,000 results
Pay per result.
Get access
This scraper is available. Request access and we’ll set you up today.
Use the request form on this site, or email [email protected].