If you just want to understand how much traffic you're getting from Google organic search results, and study the queries and landing pages that make up this traffic, this is a great tool. If you also want to figure out where you may be missing opportunities or making it hard for people to find your site, this can help with that. However, GSC is not going to give you step by step instructions for how to fix SEO issues, and won't help prioritize anything either. It's essentially like a car dashboard, but doesn't do any driving or tell you how to drive.
Screaming Frog tracks the crawl level, internal links, outbound links, title tags, and meta descriptions of the pages it crawls. It also adds HTTP status codes, title tag length, meta description length, H1, H2 tags and lengths, and canonical tags. Many of these elements are also broken into their own tabs at the top so that you can easily focus in on problem areas. You can right-click on any URL for many more options. The program is definitely valuable to anyone who is looking to gauge factors regarding SEO. However, if you are not a web designer/admin, you're going to lack an understanding of the information it provides.
Collects all indexed HTML pages of the site. This helps for when we are redirecting old URLs to the URLs of a clients new site.
Finds all page titles that are too long or too short and allows us to address them more quickly than waiting for a program such as Moz. It's more immediate.
If we have duplicate meta data on pages, this program allows us to find them, sort them, export them and take care of them in a timely manner. In our business (as for many) time is money.
I'd like more instructions about how to respond to certain errors I see - it can be difficult to understand when you see a red error message what it really means and how to fix it.
I'd appreciate warnings being 'ranked' somehow to understand what's really a big deal, and what might be a minor issue.
Search analytics is helpful but not comprehensive - you can only go back so far in the data, and can't see all the search terms that are sending traffic to your site.
Customizing the crawler - It takes a little bit of playing around with the settings to understand what each of them controls and what options need to be checked or unchecked to get the right amount of information.
Cannot Crawl a large site - Screaming Frog runs out of memory when asked to crawl a large site, even after playing around with memory settings and allocating additional memory.
The tool has become integrated into our teams daily workings and I have yet to find a tool other than scremaing frog to replicate all of our use cases for it. It's a great tool and we're sticking with it
It's easy to use, but some features are lacking a clear explanation. Somethings don't always match up. For example, the Core Web Vitals often doesn't match what you would see in the Chrome Lighthouse report or the Google Page Speed Insights tool. The tool itself is a little too basic and has to be used alongside other SEO tools and other Google properties such as Analytics
The main problem of Screaming Frog SEO Spider is it being an offline software which limits itself to a single pc and that creates problem when multi user agents want to get involved or even work on it, it’s a difficult task to collaborate on it and do research for keywords at one place
As with all Google software, your primary source of help is their forums, their knowledge base articles, or whatever tutorials you can find on the web. Often answers on their forums are not straightforward and may not address the actual issue you're experiencing. The KB articles are typically written like instruction manuals - for better or for worse. Tutorials on the web may vary, but the odds are good someone out there had the same questions as you and was kind enough to document their experience.
Screaming Frog is a relatively primitive system, and doesn't need to be supported by devs or other software. Screaming Frog does interface directly with some programs that are most needed (Google Analytics, Search Console, Page Speed Insights), so that's convenient. It isn't widely supported by other programs, but it also doesn't need to be.
We are using Ahrefs and SEMRush along with the search console. Ahrefs helps in creating our backlink strategy while SEMRush for content gaps and opportunities. Search console is more preferred as it is completely free and data comes straight from Google. But you cannot see backlinks and ads in search console.
The biggest difference between DeepCrawl and Screaming Frog is cost. Screaming Frog is significantly cheaper, and fits within a budget much easier. For most websites that aren't huge it crawls quicker to. DeepCrawl is the better tool for analysing larger websites and is easier to use in some ways, with a very friendly interface and some extras features that are useful. Ultimately price was the deciding factor though.
It has allowed me to make impactful and intelligent recommendations to clients. This has in turned deepened the client relationships and led to steady ongoing ROI.
It has also allowed me to diagnose interesting technical issues for non-client sites (like Forbes) which formed the basis for very cool company blog posts.
The investment is very low - this tool does 99% of what I need for a fraction of what other industry crawlers costs. Kudos to the Screaming Frog team!
