
Scan sites on the Internet and collect data on website links, images, CSS, scripts and apps to generate XML sitemaps in the program featuring a search and extraction engine with multiple filters by content type. Identification of redirected elements is possible.
Screaming Frog SEO Spider can help you crawl a website. The program can quickly analyze the structure of sites to provide various types of reports, which can be used to favor SEO positioning. Such elements as URL, page title, metadata, and headings can be exported to an Excel worksheet. Logically, the product is not intended for the general user; it is a great tool in the hands of web designers, though.
The program has a nice interface. Most of the results of the crawling are shown in tabular form in the main region of its window. There are two tabs between which you can switch to access page titles and meta description. Moreover, this information can be easily filtered to facilitate the analysis. A panel on the right shows summarized information about the site.
Screaming Frog SEO Spider lets you identify problems in the selected site. In this respect, it can efficiently find broken links as well as redirect links that lead to chains or loops. Similarly, you can use this program to discover excessively long page titles and meta descriptions, as well as omissions and duplicates. It is great the program can also create sitemaps, of which you can control the types of elements to include. Finally, the tool would not be complete if it could not connect to Google Analytics to download user data, which is essential to have a real grasp of the site’s usage.
All in all, Screaming Frog SEO Spider can definitely increase your productivity, mostly when you need to analyze the structure of large websites. The product has a free version, which is, regrettably, limited to 500 URLs only. It is pity that to avoid this limitation you need to purchase the full version, which can be unaffordable for some.
v8.2 [Oct 30, 2017]
- Allow restoring of Majestic, AHREFs and Moz API data.
- Sped up remove/re-spider URLs, which were much slower in the update.
- Restored the ‘copy pretty/ugly url options’ for AJAX URLs.
- Add ‘Open URL in Browser’, to multi select options in tree view.
- Ensure file extension is always present when exporting/saving.
- Show file type drop down on Mac when saving/exporting.
- Redirect chains report now shows JavaScript redirects.
- Windows installer now supports hi resolution displays.
- Insecure content report now includes HTTP-> HTTPS redirects.
- List mode is now limited to 500 URLs without a licence, like regular Spider mode.
- Fix a JavaScript crawling crash.
- Fix bug with duplicate custom extraction names causing incorrect exporting.
- Fix various bugs in forms based authentication.
- Fix issue with Windows XP start-up.
- Fix Ubuntu package warnings.
- Fix saving/restoring of screenshots.
- Fix a Majestic API crash.
- Fix a bug where double clicking an .seospider file on Mac didn’t load the file.
- Fix an issue with lower window loosing sort, when clicking URLs in the master view.