How to Affect How Robots Search or Index Your Site

Preparing your website for indexing will take some time and effort.

All search engine robots interact with and index your website in a number of common ways that you can predict and prepare for. You can make your website look far more desirable to these search engines and improve your search ranking by including critical keywords, fresh content, a clear navigational hierarchy, and more. You will not need any special tools in order to accomplish these optimizations.

Instructions

1. Compile a list of keywords that pertain directly to the topical nature of your site. Ideally, these keywords should be the words you can imagine someone typing into an Internet search box in order to find a site. You can research these keywords using tools such as the Google Wonder Wheel and the YouTube Keyword Tool.

2. Use these keywords frequently throughout your site, but also make sure the content flows and the words fit in naturally.

3. Add your main keyword into the title of your Web page.

4. Constantly create new content. Update the website with new, pertinent articles containing your keywords at least once a week in order to keep your site relevant and ranking well with the search engines. If possible, adding one new piece of content daily is best.

5. Make sure your important keywords are in the first and last sentence of your page as well.

6. Create user-friendly site navigation. Every main page or section should be connected to the next by at least one hyperlink, and the hyperlinks should contain one of your important keywords, if possible. The navigation bar should be visible on every page.

7. Check with your website provider to see if your server supports the "If-Modified-Since HTTP" header. This will notify search engines when fresh content has been added to your website, prompting them to re-crawl your site.

Tips & Warnings

• Do not use frames for your navigation sidebar or for any other part of your site. Frames are separate, scrolling boxes within a page, and they disrupt proper search engine indexing.

• Do not place important links and website content in script code. Script code is established by using "SCRIPT" tags within an HTML document. When using HTML, do not put any site content that you want readers to share and search engines to index within these tags.

References

• Cal Poly: Make a Search Engine-Friendly Web Site