Chrome extension
Sitemap Explorer
See the whole site as a tree, then send a branch to the scraper.
All nine toolsThe explorer reads the sitemap, follows the sitemap index files and builds a tree of the site by address structure, with a count on every branch. That is how you learn what is there before you extract any of it.
When a site publishes no sitemap, it crawls politely from the home page to a depth you set. Either way you end up with a branch you can hand to the page extractor without pasting a single address.
Screenshot coming with the tool. Nothing here is a drawing, so this space stays empty until the real capture exists.
How it works
- Step 1
Enter a domain
Tabitha looks for the sitemap in robots.txt and at the usual addresses, then reads the index files it finds.
- Step 2
Read the tree
Every segment of the address becomes a node with a count, so you can see at a glance that a shop has 4 000 product pages and 12 category pages.
- Step 3
Pick a branch and filter
Select a branch, add a pattern or exclude one, and check the last modified date where the sitemap provides it.
- Step 4
Send it onward
Push the selection into the page extractor as a URL list, or export it as CSV.
When to reach for it
- Planning a job: how many pages are there and how are they arranged.
- Content and SEO audits where the address structure is the question.
- Building a URL list without clicking through pagination.
What it will not do
- A sitemap says what the site chose to publish. Pages left out of it will not appear.
- Crawling a site without a sitemap is slower on purpose, because it is more requests.
- Very large sitemaps are read in parts to keep the browser responsive.