Screaming Frog tracks the crawl level, internal links, outbound links, title tags, and meta descriptions of the pages it crawls. It also adds HTTP status codes, title tag length, meta description length, H1, H2 tags and lengths, and canonical tags. Many of these elements are also broken into their own tabs at the top so that you can easily focus in on problem areas. You can right-click on any URL for many more options. The program is definitely valuable to anyone who is looking to gauge factors regarding SEO. However, if you are not a web designer/admin, you're going to lack an understanding of the information it provides.
Collects all indexed HTML pages of the site. This helps for when we are redirecting old URLs to the URLs of a clients new site.
Finds all page titles that are too long or too short and allows us to address them more quickly than waiting for a program such as Moz. It's more immediate.
If we have duplicate meta data on pages, this program allows us to find them, sort them, export them and take care of them in a timely manner. In our business (as for many) time is money.
Customizing the crawler - It takes a little bit of playing around with the settings to understand what each of them controls and what options need to be checked or unchecked to get the right amount of information.
Cannot Crawl a large site - Screaming Frog runs out of memory when asked to crawl a large site, even after playing around with memory settings and allocating additional memory.
The tool has become integrated into our teams daily workings and I have yet to find a tool other than scremaing frog to replicate all of our use cases for it. It's a great tool and we're sticking with it
The main problem of Screaming Frog SEO Spider is it being an offline software which limits itself to a single pc and that creates problem when multi user agents want to get involved or even work on it, it’s a difficult task to collaborate on it and do research for keywords at one place
Screaming Frog is a relatively primitive system, and doesn't need to be supported by devs or other software. Screaming Frog does interface directly with some programs that are most needed (Google Analytics, Search Console, Page Speed Insights), so that's convenient. It isn't widely supported by other programs, but it also doesn't need to be.
The biggest difference between DeepCrawl and Screaming Frog is cost. Screaming Frog is significantly cheaper, and fits within a budget much easier. For most websites that aren't huge it crawls quicker to. DeepCrawl is the better tool for analysing larger websites and is easier to use in some ways, with a very friendly interface and some extras features that are useful. Ultimately price was the deciding factor though.
It has allowed me to make impactful and intelligent recommendations to clients. This has in turned deepened the client relationships and led to steady ongoing ROI.
It has also allowed me to diagnose interesting technical issues for non-client sites (like Forbes) which formed the basis for very cool company blog posts.
The investment is very low - this tool does 99% of what I need for a fraction of what other industry crawlers costs. Kudos to the Screaming Frog team!
