How to Ping the Crawler


A search engine's crawler -- also known as a spider or robot-- "crawls" the Internet, reading Web pages and looking for new and changed content. To be listed in a search engine's index, your pages must be crawled. Google and other large search engines have optimized their crawlers to find content very quickly on their own. Those search engines do not accept direct notifications, known as pings. However, you can alert a crawler to the existence of a new or updated page.

  • Create a site map to alert the major search engines to a new site. A site map is an Extensible Markup Language (XML) document that lists the pages on your site that you want search engines to index. You can create a site map manually and upload it to your Web server using FTP software. Or, you can generate one automatically using a plug-in like Google XML Sitemaps for Wordpress or a Web site like xml-site

  • Submit a site map to Google by typing this URL into your web browser:

    Replace "" with the path to the site map on your website.

  • Register your site in Google Webmaster Tools. In addition to assisting you with creating and submitting site maps, Google Webmaster Tools will show you how often Google's crawler -- known as the Googlebot -- has crawled your site, as well as any problems preventing your pages from being indexed. Bing Webmaster Tools and Yahoo Site Explorer are similar services.

  • Ping update services to list your new page on those sites, where they can be found by search engine crawlers. Popular content management and blogging software, like Wordpress and Drupal, pings multiple update services automatically. You can also use a web service like Ping-o-Matic or Pingler to send pings to multiple update services. Most services feature a simple form for the URL and title of the page, as well as a selection of search engines.

Tips & Warnings

  • These techniques are most effective when a Web page is new or updated. Repeated submissions or pings will not improve an unchanged page's rank in a search, and may drop it in rank or remove it from results entirely.

Related Searches


  • Photo Credit Hemera Technologies/ Images
Promoted By Zergnet



You May Also Like

  • How to Strip and Splice Wire

    Whether you're installing a light fixture or replacing an electrical outlet, at some point you need to strip and splice wires. Most...

Related Searches

Check It Out

Geek Vs Geek: Robot battles, hoverboard drag race, and more

Is DIY in your DNA? Become part of our maker community.
Submit Your Work!