{"id":1282,"date":"2012-12-04T14:42:59","date_gmt":"2012-12-04T14:42:59","guid":{"rendered":"https:\/\/www.zuantechnologies.com\/blog\/?p=1282"},"modified":"2020-12-15T08:27:47","modified_gmt":"2020-12-15T08:27:47","slug":"google-crawls-techniques","status":"publish","type":"post","link":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/","title":{"rendered":"Google Crawls Techniques"},"content":{"rendered":"<p><!--<a href=\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignleft wp-image-1283\" title=\"Glossy Robot with Magnifying Glass\" src=\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg\" alt=\"\" width=\"268\" height=\"273\" \/><\/a>-->To understand why this works you need a brief understanding of how Google currently crawls the web.<\/p>\n<p>Googlebot, as it is widely known, has always been a text-based web crawler, capturing the web by recording and organizing sites and pages looking at the code that makes up a site.<\/p>\n<p>In recent years the appearance of visual snapshots and an understanding of headless browsers and the theory that Google uses its Chrome browser as part of that crawl\u00a0have pushed us toward the belief that Google actually \u201csees\u201d the web page too.<br \/>\n<!--more--><br \/>\nThe problem is that the trick being used here suggests that those two crawls aren\u2019t in parallel, or don\u2019t talk to each other at least, to match what the text crawler is seeing to that of the visual crawler.<\/p>\n<h3>The Trick<\/h3>\n<p>I say this because in the case of several successful black hat sites they appear to be using a clever CSS trick, hiding links in powerful places that pass huge chunks of link equity, while part fooling Googlebot, buying them precious time at the top.<\/p>\n<p>A lot of key links are \u201cplaced\u201d in a position so high up on the page that they are \u201cinvisible\u201d to the normal user, often sat in the header in pixel position -9999px or similar. That way the user, and visual crawler, doesn\u2019t see the link and so it takes Google much longer to find out how that site is actually ranking.<\/p>\n<p>Here\u2019s what the offending script usually looks like:<\/p>\n<p>As an added bonus, as well as buying time for the site, Google may also be seeing this link as a header link, passing even more link juice across because of it. A 2004 patent application by Google suggested they planned on assigning greater relevance to links in such positions and I wrote a little more about personalized Page Rank in \u201cIs Google Afraid of the Big Bad Wolfram?\u201d<\/p>\n<p>Those making money out of the sites know this, and they also know that by the time Google\u2019s crawlers piece together the picture from their main \u201cbase\u201d crawl, not just their regular visual and \u201cfresh\u201d crawls, that they have already made a chunk of money.<\/p>\n<p>The time comes, of course, when the site will be taken out, either by sandbox or by a Panda or Penguin crawl, but by that time the money is made and time bought to simply line up another site. And the process is then repeated.<\/p>\n<p>Thanks to All<\/p>\n<p>by<\/p>\n<p><a href=\"http:\/\/www.zuantechnology.com\/\">Zuan Technologies<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>To understand why this works you need a brief understanding of how Google currently crawls the web. Googlebot, as it is widely known, has always been a text-based web crawler, capturing the web by recording and organizing sites and pages looking at the code that makes up a site. In recent years the appearance of [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1283,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[32],"tags":[91,92],"class_list":["post-1282","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-seo","tag-google-crawls-techniques","tag-how-google-crawls-the-web"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Google Crawls Techniques - Zuan Technologies<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Google Crawls Techniques - Zuan Technologies\" \/>\n<meta property=\"og:description\" content=\"To understand why this works you need a brief understanding of how Google currently crawls the web. Googlebot, as it is widely known, has always been a text-based web crawler, capturing the web by recording and organizing sites and pages looking at the code that makes up a site. In recent years the appearance of [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/\" \/>\n<meta property=\"og:site_name\" content=\"Zuan Technologies\" \/>\n<meta property=\"article:published_time\" content=\"2012-12-04T14:42:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-12-15T08:27:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"320\" \/>\n\t<meta property=\"og:image:height\" content=\"365\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/person\/fc1e4fab91dc1c141c2dc159b5da4bee\"},\"headline\":\"Google Crawls Techniques\",\"datePublished\":\"2012-12-04T14:42:59+00:00\",\"dateModified\":\"2020-12-15T08:27:47+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/\"},\"wordCount\":420,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg\",\"keywords\":[\"Google Crawls Techniques\",\"How Google Crawls the Web\"],\"articleSection\":[\"SEO\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/\",\"url\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/\",\"name\":\"Google Crawls Techniques - Zuan Technologies\",\"isPartOf\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg\",\"datePublished\":\"2012-12-04T14:42:59+00:00\",\"dateModified\":\"2020-12-15T08:27:47+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#primaryimage\",\"url\":\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg\",\"contentUrl\":\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg\",\"width\":320,\"height\":365},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.zuantechnologies.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Google Crawls Techniques\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#website\",\"url\":\"https:\/\/www.zuantechnologies.com\/blog\/\",\"name\":\"Zuan Technologies\",\"description\":\"Blog\",\"publisher\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.zuantechnologies.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#organization\",\"name\":\"Zuan Technologies\",\"url\":\"https:\/\/www.zuantechnologies.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2018\/01\/Zuan-Logo-FB.png\",\"contentUrl\":\"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2018\/01\/Zuan-Logo-FB.png\",\"width\":180,\"height\":180,\"caption\":\"Zuan Technologies\"},\"image\":{\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/person\/fc1e4fab91dc1c141c2dc159b5da4bee\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/5bfa6b3669e29838e04d72ce0451ea0b873f08264c6a99602161af43703aa8af?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/5bfa6b3669e29838e04d72ce0451ea0b873f08264c6a99602161af43703aa8af?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"url\":\"https:\/\/www.zuantechnologies.com\/blog\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Google Crawls Techniques - Zuan Technologies","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/","og_locale":"en_US","og_type":"article","og_title":"Google Crawls Techniques - Zuan Technologies","og_description":"To understand why this works you need a brief understanding of how Google currently crawls the web. Googlebot, as it is widely known, has always been a text-based web crawler, capturing the web by recording and organizing sites and pages looking at the code that makes up a site. In recent years the appearance of [&hellip;]","og_url":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/","og_site_name":"Zuan Technologies","article_published_time":"2012-12-04T14:42:59+00:00","article_modified_time":"2020-12-15T08:27:47+00:00","og_image":[{"width":320,"height":365,"url":"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg","type":"image\/jpeg"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#article","isPartOf":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/"},"author":{"name":"admin","@id":"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/person\/fc1e4fab91dc1c141c2dc159b5da4bee"},"headline":"Google Crawls Techniques","datePublished":"2012-12-04T14:42:59+00:00","dateModified":"2020-12-15T08:27:47+00:00","mainEntityOfPage":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/"},"wordCount":420,"commentCount":0,"publisher":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#primaryimage"},"thumbnailUrl":"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg","keywords":["Google Crawls Techniques","How Google Crawls the Web"],"articleSection":["SEO"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/","url":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/","name":"Google Crawls Techniques - Zuan Technologies","isPartOf":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#primaryimage"},"image":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#primaryimage"},"thumbnailUrl":"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg","datePublished":"2012-12-04T14:42:59+00:00","dateModified":"2020-12-15T08:27:47+00:00","breadcrumb":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#primaryimage","url":"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg","contentUrl":"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2012\/12\/crawling-bot.jpg","width":320,"height":365},{"@type":"BreadcrumbList","@id":"https:\/\/www.zuantechnologies.com\/blog\/google-crawls-techniques\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.zuantechnologies.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Google Crawls Techniques"}]},{"@type":"WebSite","@id":"https:\/\/www.zuantechnologies.com\/blog\/#website","url":"https:\/\/www.zuantechnologies.com\/blog\/","name":"Zuan Technologies","description":"Blog","publisher":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.zuantechnologies.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.zuantechnologies.com\/blog\/#organization","name":"Zuan Technologies","url":"https:\/\/www.zuantechnologies.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2018\/01\/Zuan-Logo-FB.png","contentUrl":"https:\/\/www.zuantechnologies.com\/blog\/wp-content\/uploads\/2018\/01\/Zuan-Logo-FB.png","width":180,"height":180,"caption":"Zuan Technologies"},"image":{"@id":"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/person\/fc1e4fab91dc1c141c2dc159b5da4bee","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.zuantechnologies.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/5bfa6b3669e29838e04d72ce0451ea0b873f08264c6a99602161af43703aa8af?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/5bfa6b3669e29838e04d72ce0451ea0b873f08264c6a99602161af43703aa8af?s=96&d=mm&r=g","caption":"admin"},"url":"https:\/\/www.zuantechnologies.com\/blog\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/posts\/1282","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/comments?post=1282"}],"version-history":[{"count":2,"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/posts\/1282\/revisions"}],"predecessor-version":[{"id":5008,"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/posts\/1282\/revisions\/5008"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/media\/1283"}],"wp:attachment":[{"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/media?parent=1282"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/categories?post=1282"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.zuantechnologies.com\/blog\/wp-json\/wp\/v2\/tags?post=1282"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}