Free tool
Find every URL on any website
Enter a domain and get the list of its pages, gathered from its sitemaps and links in seconds. Use it to plan a crawl, audit a site or build a scrape list.
What the URL extractor finds
Sitemaps and links
Reads every sitemap the site publishes and adds the URLs its pages link to.
De-duplicated
Each page appears once, whatever order or format it was found in.
CSV export
Download URL and title as CSV with a free account.
No rendering
Pages are not downloaded, so even large sites come back in seconds.
Ready to crawl
Send the list to a crawl and turn every page into searchable markdown.
See the structure
Want the link hierarchy instead? Try the website tree generator.
How to extract all URLs from a website
- 01 / 03
Enter the site
Type a domain or a section, like docs.example.com.
- 02 / 03
Extract URLs
Sitemaps and links are read; the first 25 URLs show instantly.
- 03 / 03
Export
Sign in free to see every URL and download them as CSV.
Questions
How do I find all the pages on a website?
Read its sitemaps and follow its links. This tool does both and removes duplicates, so you get one clean list.
How many URLs can it find?
Up to 5,000 per site. The free preview shows the first 25; a free account shows them all.
Does it work without a sitemap?
Yes, links found on the site are used too, though a site with neither a sitemap nor crawlable links will return few URLs.
Is there an API?
Yes, the map endpoint returns the same list for any site.
Related
Turn the list into a knowledge base
Free account, 1 billion tokens, no card. Full results, downloads and the API.