What Is a Web Search API? Results, Content and Uses

What a web search API is

A search engine page is built for a person: ten links, ads, and a box of snippets. A web search API returns the same kind of results as JSON, so a program can use them. The program might be a research script, a lead list builder, or an AI agent that needs information newer than its training data.

The usual inputs are the query, a result count, and a language or region. The usual output is a ranked list with a title, URL and snippet for each result.

Snippets versus full content

Search APIs split into two kinds by what they return, and the difference decides what you can build.

	Snippet results	Results with content
Returns	Title, URL, short snippet	Title, URL and the page as text or markdown
Next step	You fetch and clean each page	Ready to read, chunk or pass to a model
Good for	Link lists, rank tracking	Research, RAG, agents
Speed	Fast	Slower, since each page is fetched

A snippet is a sentence or two the engine picked to match the query. It is rarely enough to answer a question and never enough to quote reliably, so an AI application built on snippet results ends up writing its own scraper for the second step.

How it is used

Grounding AI answers in current sources. The agent searches, reads the results, and answers with citations. See LLM grounding.
Deep research. Many searches feed one report, with each source kept for checking.
Lead and company enrichment. Search for a company name, then read its pages for facts.
Monitoring a topic. Repeat the same query on a schedule and look at what is new.

Search is also a discovery step for crawling: the results show which sites are worth a full crawl.

What to check

What comes back. Snippets only, or the page content, and in what format.
JavaScript pages. If the service fetches results, does it render them in a browser?
Result limits and language. How many results per query, and whether language can be set.
Where results go. A response you must store yourself, or content kept and searchable later.
Query handling. Some services rewrite or expand your query; others search it exactly as sent. Know which you have.

A search API that returns links has done half the job. The other half is reading the pages.

How SarvCrawl handles it

SarvCrawl's web search is an asynchronous job: POST /api/jobs/search with query, limit (default 10, maximum 100), lang (default en) and output formats. Each result is scraped and stored as a page in your knowledge base, so you get the content of the pages, not only snippets. The query is searched as sent; web search does not expand or rewrite it.

curl -s -X POST https://crawl.sarv.com/api/jobs/search \
  -H "x-api-key: $KB_API_KEY" -H "Content-Type: application/json" \
  -d '{"kb_id":"kb_abc123","query":"postgres logical replication lag",
       "limit":20,"lang":"en","formats":["markdown"]}'
# → 202 {"job_id":"…"}  then poll GET /api/jobs/:id

Because the results land in a knowledge base, they can be searched again later with keyword, hybrid or smart search on paid plans. Agents can call it through the kb_web_search tool on the MCP server.

Frequently asked questions

What is the difference between a web search API and a scraping API?

A search API starts from a query and finds pages; a scraping API starts from a URL you already have and fetches its content. Some services combine the two by scraping each search result.

Why do AI agents need a web search API?

A model's knowledge stops at its training cutoff. Searching the web lets an agent find current pages and answer from them, with sources it can cite.

Are search snippets enough for RAG?

Rarely. Snippets are short and chosen to match the query, not to answer it. Retrieval works better on the full page content, cleaned and chunked.

Does a web search API expand my query?

It depends on the service. Some rewrite queries before searching; others send them exactly as given. Check, because it changes which results you get.

What is a web search API?