How to Make a Domain Searchable by Using Google & Yahoo


After completing development of your website, the next step is to index ("search-enable") your site. When dealing with Google and Yahoo!, you can take several steps (such as creating a robots.txt file and using meta tags) that will help ensure your site becomes search friendly. To direct traffic to your site, it's important to understand how to set up your site for searches and submit it to the search engines of your choice.

Things You'll Need

  • File Transfer Protocol (FTP Software)
  • Knowledge of basic HTML or PHP (based on your site's page types)

Create a Robots.txt File

  • Open your preferred text editing software. You can use Microsoft Word, a simple text editing program, or any word processing software you choose.

  • Place the following code in the opened word processor application:

    User-agent: *
    Disallow: /
    User-agent: Googlebot
    Allow: /
    User-agent: Slurp
    Allow: /

    The code "User-agent" tells search engines whether their search crawler is allowed to index the page in question, while "Disallow: /" and "Allow: /" tell the crawler whether it can proceed with indexing. Placing a slash "/" after the "Disallow" or "Allow" code tells the crawler it can access all pages on your website. For example, if you want to Disallow a folder on your site called "Private," you would type "Disallow: /private/" in your robots.txt file

    Googlebot is Google's search crawler, while Yahoo uses a crawler known as Slurp. Crawlers travel from one website to the next and are responsible for saving web pages on search engine servers for future search purposes.

  • To save the file to your desktop as "robots.txt," click on "file" followed by "save as." Next type in the name "robots" and choose the "text" file type. Do not use the standard "Word document" save type or the file will be saved as robots.doc. If you do accidentally save the file as robots.doc, however, you can right click on the file icon and choose "rename," then type in robots.txt to change the file type.

  • Upload your robots.txt file to your site's main directory (for example, "public_html") using an FTP program such as Filezilla. You may also use whatever file upload method your website host provides.

  • Close your FTP program and wait for Google and Yahoo! to browse your website using the web crawlers mentioned in Step 2. The process of indexing can take a few hours or several months.

Submit Your Website for Searching

  • To submit your URL to Google, visit the Google AddURL website (see Resources).

  • Enter your site's URL in the specified field. Next, enter your site's "keyword focus" in the comments field. Your keywords should be words you wish to target for your site, such as "technology, arts and crafts, news, fashion, celebrities," and so forth. Keywords should provide a generalized overview of your site's content. Click "Add URL" to complete the process.

  • Submit your site to the Yahoo! Directory (see the Resources section). You may choose between the paid method, for a guaranteed 7-day approval process, or the free submission process, which may or may not get your site indexed.

  • Visit Google and Yahoo! and type "site:," replacing "" with the actual URL for your website. Click "Search" to see if your site has been indexed.

Meta Tag Search Help

  • Place meta tags as close to the page's <head> tag as possible. Meta tags are placed inside the sites PHP or HTML files (depending on how your site creates content). To start, create a title tag for each page. This tag shows the title for your page when people use Google, Yahoo! and most other search engines. Type "<title> insert your page's title here</title>." If you've completed sections one and two above, your site is now search ready.

  • Create the description meta tag. This tag should explain in one or two short sentences what the current page content you are writing or have written is about. Type <meta name="description" content="enter your content description here". Place this information directly below the </title> tag so it will show up below the title on search engines.

  • Create keywords for your content and place the keywords meta tag directly below the description tags. Type "<meta name="keywords" content="enter each keyword you wish to use, separated by a comma">.

  • Visit your website and navigate to the page for which you just created meta tags, for example: Check the very top of your web browser to see if the title of the article appears. For example, you would see the phrase "how to fix a phone" if you used those words inside the <title></title> tags.

Tips & Warnings

  • Your website content management system, such as Wordpress or Joomla, may include a specialized section for entering meta tags. If you are not creating pages individually, check your software provider's tech page or FAQs to find out how to add meta tags for each individual page.
  • Quick tip: To index your site more quickly, find sites that are already indexed and get a link to your site. Many webmasters will write a guest post for other sites or form link partnerships. The more links you have to your site, the faster you will be indexed.

Related Searches


Promoted By Zergnet



You May Also Like

Related Searches

Check It Out

Geek Vs Geek: Robot battles, hoverboard drag race, and more

Is DIY in your DNA? Become part of our maker community.
Submit Your Work!