SarvCrawl vs Firecrawl: same engine, different finish line

The fetching is close to identical, because it is the same lineage. The difference is what happens after the page is fetched.

With Firecrawl, you get the content back and the pipeline is yours: split it into chunks, pick an embedding model, stand up a vector store, write the retrieval and keep it fresh. That is the right trade if you want to choose every one of those pieces.

With SarvCrawl, every page and document lands in a knowledge base that is chunked on its own headings and embedded automatically, with BM25 and hybrid search on top. The trade is that those choices are ours. If you wanted to make them yourself, Firecrawl is the better fit.

# 1. Queue a scrape — the page lands in a knowledge base
curl -s -X POST https://crawl.sarv.com/api/jobs/scrape \
  -H "x-api-key: $KB_API_KEY" -H "Content-Type: application/json" \
  -d '{"kb_id":"kb_abc123","url":"https://example.com","formats":["markdown"]}'
# → {"job_id":"dacf0540-…","status":"queued"}

# 2. Poll the job until it completes
curl -s -H "x-api-key: $KB_API_KEY" https://crawl.sarv.com/api/jobs/dacf0540-…

# 3. Read the markdown back
curl -s -H "x-api-key: $KB_API_KEY" https://crawl.sarv.com/api/kb/kb_abc123/pages

The request shapes are familiar — a URL, a list of formats, depth and limit for crawls — with one addition: every job names the kb_id it writes into. Create a knowledge base, swap the endpoint, and poll jobs instead of waiting on a synchronous scrape.

If you already run a vector database fed by Firecrawl, you can keep it: read pages back over GET /api/kb/:id/pages or export chunks to JSONL. You do not have to adopt our search to use our crawler.