Web Scraping API: Any URL to Markdown or JSON

What the scrape endpoint does

POST /api/jobs/scrape takes one URL and returns a job. The engine fetches the page, renders JavaScript where the page needs it, strips navigation and boilerplate, and stores the result as markdown beside the formats you asked for.

Formats — markdown, html, links, json and images, in any combination.
Documents too — with include_docs, linked PDFs and Office files are parsed to markdown in the same job. Scanned PDFs go through OCR.
Asynchronous by design — every call is a job you poll, so a slow page never holds a connection open, and a crawl of 500 pages uses the same pattern as one.
Stored, not thrown away — results live in a knowledge base you can read back, export to JSONL or CSV, or search.

A scrape in three calls

# 1. Queue a scrape — the page lands in a knowledge base
curl -s -X POST https://crawl.sarv.com/api/jobs/scrape \
  -H "x-api-key: $KB_API_KEY" -H "Content-Type: application/json" \
  -d '{"kb_id":"kb_abc123","url":"https://example.com","formats":["markdown"]}'
# → {"job_id":"dacf0540-…","status":"queued"}

# 2. Poll the job until it completes
curl -s -H "x-api-key: $KB_API_KEY" https://crawl.sarv.com/api/jobs/dacf0540-…

# 3. Read the markdown back
curl -s -H "x-api-key: $KB_API_KEY" https://crawl.sarv.com/api/kb/kb_abc123/pages

The same client covers crawl, map, web search, upload and monitor. If you want a whole site rather than a page, see the web crawler API.

Why scrape into a knowledge base

Most scraping APIs hand you a page and stop. If what you are building answers questions from that content, the page is step one of five: after it come chunking, embedding, an index and a retrieval policy.

SarvCrawl keeps going. Each page is chunked on its own headings and embedded, and on paid plans two search endpoints sit on top: BM25 keyword search and hybrid search that fuses dense vectors with BM25. You can use the scraper alone — but you do not have to build the rest.

The cost of switching scraping APIs is a day. The cost of switching what owns your chunks and embeddings is a re-ingest of everything.

How it compares

	SarvCrawl	Typical scraping API
Clean markdown output	Yes	Usually
JavaScript rendering	Yes	Usually, often at a higher credit cost
PDF and Office parsing	Yes, with OCR for scans	Varies
Chunking and embeddings	Included	You build it
Hosted search over results	Paid plans	You build it
Heavy anti-bot / proxy rotation	Not the focus	Specialists do this best

For a vendor-by-vendor breakdown with prices, see the 8 best web scraping APIs in 2026.

Questions

Is there a free web scraping API tier?

Yes. Every account starts with 1 billion tokens and no card. Scrape, crawl, map, web search, upload and monitor work on every plan, including Free.

Does it render JavaScript?

Yes. JavaScript-heavy pages are rendered in a headless browser before extraction, so single-page apps return their real content.

Can I get structured JSON instead of markdown?

Yes. Ask for the json format alongside or instead of markdown. Links and image lists are available as their own formats.

Is the API synchronous?

No. Every scrape is a job: you get a job id back immediately, poll it, then read the page. The MCP tools tell an agent to do exactly that, so no glue code is needed.

Which languages are supported?

Any language that can make an HTTP request. There are official Node and Python SDKs, a CLI called sarv-kb, an n8n node and an MCP server.

A web scraping API that returns clean markdown