Live briefing
Everything about Googlebot
Web crawler used by Google
Googlebot is the web crawler software used by Google that collects documents from the web to build a searchable index for the Google Search engine. This name is actually used to refer to two different types of web crawlers: a desktop crawler and a mobile crawler. Wikipedia →
🧠Start here: a 6-step reading path
- Read the overview: GooglebotA 5-minute orientation
- Pick up a classic book: Googling securityThe most-reprinted book on the subject
- Read the landmark paper: Web Data Mining and Crawling: A Detailed Overview in the Aspect of Googlebot, Apache Nutch and BingbotCited 0 times
- Go sideways: Robots.txtA closely related topic
- Go sideways: Search engine optimizationA closely related topic
đź“– Definitions
No dictionary entry for this exact phrase.
🗂️ The subject at a glance
🏆 Landmark papers (most cited)
đź“š Books worth reading
Googling securityGreg Conti · 2008- 📕Googling securityGreg Conti · 2008
- 📕SEO from a GoogleBots PerspectiveStirling Halifax · 2017
Ranked by number of editions on Open Library, a rough measure of how lasting a book is.
đź”— Related topics
- Robots.txt The Robots Exclusion Protocol (often referred to by the filename used to implement it, robots.txt) is a standard used by websites to indicate to visiting…
- Search engine optimization Search engine optimization (SEO) is the practice of improving the visibility and overall performance of websites and web pages in search engine results…
- Sitemaps Sitemaps is a protocol in XML format meant for a webmaster to inform search engines about URLs on a website that are available for web crawling. It allows…
- Web crawler A web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web…
- Site map A site map or sitemap is a list of pages of a web site within a domain. There are three primary kinds of sitemap: Sitemaps used during the planning of…
- Google Search Console "Google Search Console" (formerly Google Webmaster Tools) is a web service by Google which allows webmasters to check indexing status, search queries,…
- Bingbot Bingbot is a web-crawling robot (type of internet bot), deployed by Microsoft October 2010 to supply Bing. It collects documents from the web to build…
- Nofollow Nofollow is a setting on a web page hyperlink that directs search engines not to use the link for page ranking calculations. It is specified in the page…