CSS Selectors vs XPath for Web Scraping, with Examples

Selectors in more depth

Once a page is parsed into a DOM, a scraper needs a way to say “the price”, “every product title” or “the second table”. Selectors are that address. The same selector run against the same page always returns the same elements, which is what makes rule-based extraction repeatable.

CSS selectors come from stylesheets and are what document.querySelectorAll and libraries such as BeautifulSoup and cheerio accept. XPath comes from the XML world; browsers support it through document.evaluate, and lxml and Scrapy support both.

The same queries in both

Goal	CSS selector	XPath
Every link	`a`	`//a`
Element by id	`#price`	`//*[@id="price"]`
By class	`.product .title`	`//[contains(@class,"product")]//[contains(@class,"title")]`
Attribute value	`a[href$=".pdf"]`	`//a[ends-with(@href,".pdf")]` (XPath 2.0; not in browsers)
Direct child	`ul > li`	`//ul/li`
By text	not possible	`//button[contains(text(),"Next")]`
Parent of a match	`div:has(> img)` (newer engines only)	`//img/..`

from lxml import html

tree = html.fromstring(page_html)
titles = tree.cssselect(".product .title")          # needs the cssselect package
prices = tree.xpath('//span[@class="price"]/text()')

Which to use

CSS selectors for most work: shorter, easier to read, and the same syntax front-end developers already use.
XPath when you need to match on text, climb to a parent or ancestor, or select by position relative to another element.
Neither when you want the whole readable body of many different sites. Writing selectors per site does not scale; main-content extraction finds the article without per-site rules.

Why selectors break

Generated class names. Build tools emit classes like css-1x2y3z that change on every deploy. Prefer ids, semantic tags, data-* attributes or text.
Positional paths. /html/body/div[3]/div[2]/span fails as soon as a banner is added. Copy-as-XPath from devtools produces exactly these.
Selecting before rendering. If the element is inserted by JavaScript, it is not in the raw HTML to select.
Silent failure. A selector that stops matching returns nothing rather than an error. Check for empty results.

How SarvCrawl handles it

SarvCrawl does not take CSS selectors or XPath. Extraction is automatic: navigation, footers, asides, scripts and styles are removed, and the main content comes back as markdown, so you do not write or maintain per-site rules. If you need specific fields, ask the scrape endpoint for html, which is the main content after JavaScript has run with navigation and boilerplate already removed, and apply your own selectors to it. Elements stripped as boilerplate will not be there to select.

curl -s -X POST https://crawl.sarv.com/api/jobs/scrape \
  -H "x-api-key: $KB_API_KEY" -H "Content-Type: application/json" \
  -d '{"kb_id":"kb_abc123","url":"https://example.com/product/42","formats":["html"]}'
# → 202 {"job_id":"…"}; run your selectors on the returned HTML

Frequently asked questions

Is XPath faster than CSS selectors?

Rarely enough to matter. Many libraries translate CSS selectors to XPath internally. Choose for readability and what you need to express.

Can CSS selectors match text?

No. Standard CSS selectors match elements by tag, id, class, attributes and position, not by their text. Use XPath, or select broadly and filter text in code.

How do I find a selector for an element?

Inspect it in browser devtools and read its tag, id, classes and attributes. Write the shortest selector that is unlikely to change, rather than copying the full path devtools generates.

Do I need selectors to scrape a site?

Only for specific fields on known layouts. For readable text from many sites, automatic main-content extraction avoids writing selectors at all.

What are CSS selectors and XPath?