In this video interview, Adam Lasnik, Google Search Evangelist, explains how their Analyze Robots.txt tool helps webmasters spot problems that could block the GoogleBot from crawling files. The Web Crawl Analyzer tools can spot other types of crawling mistakes. These are found under Google Webmaster Tools, designed to help site owners diagnose and correct website problems.
