How to Spider a Web Site

Spiders are a form of search engine, and "spidering" a Web site allows you to pull information from other Web sites into your Web site. You tell the spider what keywords to look for, and it searches for them on the Internet and pulls the related information into your site, eliminating the need to cut and paste. The type of spider software you purchase depends on what you want the spider to find for you. There are email, fax and data spiders. Here's how to get started in spidering a site.

Instructions

Difficulty: Moderately Easy

Things You’ll Need:

• Internet access
• Computer
• Spidering software

1. Research different spidering software. Visit electronics stores such as Best Buy. Look at Web sites that sell spidering software such as Newprosoft (see Resources below). An email spider will look for relevant customers for your marketing program. Offline spiders let you collect Web sites that you can look at and research later when you are not online.
2. Purchase the software with the features that you are looking for.
3. Install your spidering software. Most computers will auto-install the software once you place the disc into your computer. If not, follow the install directions that come with your software.
4. Follow the directions and set up a test spider attempt to pull information from other Web sites according to your specifications.
5. Test your spider and see what information is pulled in. If you are pulling in too much information, then disallow sites and information that you do not want to pull.

Tips & Warnings

• Be selective with what you spider at first. You could be overwhelmed with the amount of information that you get.