Crawling seo tools

Compose robots.txt, a URL list sitemap, or a rel=canonical tag. The HTTP status tool here is a code lookup, not a live crawl.

robots.txt, sitemaps, canonicals, slugs, UTM builders, and URL parsing.

Create a simple robots.txt that tells crawlers which paths to skip and where your sitemap is.

Paste a list of page URLs and get a valid sitemap.xml that you can upload to your site and submit in search console tools.

Create a rel=canonical link tag that tells search engines which URL is the main version of a page.

Look up status codes. This tool does not fetch remote URLs.