What Is a Search Engine Spider?

Video Transcript

"Hi, my name is Dave Andrews. Today I'm going to talk to you a little bit about Search Engine Spiders. Now Spider's a program that's used by a search engine that basically scan a web page for links. And then it will visit each of those links. Now what this enables the spider to do, is to weave it's way through the web. Through these links that are created by files. And basically find all of the websites that are connected to each other. And discover new websites that it didn't already have in it's listings. Let's go to our computer here. And I'll give you a little example on how that works. This is my website here. DaveAndrews.org. As you can see, I have links here. Let's do, you know, programs that I sell. As well as some features and information about it. Now what a website or web search engine's Spider will see, is actually the source to go to this web page. So just right click on a blank space somewhere. Click on View Source. Now this is actually what the Spider sees when it opens up my web page. This is the code behind the page. I'm going to scroll down here a little bit. And I should have an anchor tab here. Actually here, it is right here. It says, A HREF equals DaveAndrews.org/Products. Now a web Spider will look for these A tags which are anchors. That's what actually shows up as a link. With a blue underline that you can click on. And let's say for instance this one right here is going to download .PHP. Now the website Spider didn't know anything about download .PHP. But because I have a link to it right here, in this A hyper text reference tab or tag. It knows to go and visit that download page to see if it's something new. That it would like to add to it's listings. It's through all these links that website Spiders, scan through the Internet. Find new pages and find the text on those pages. That's important and people might want to search for. My name is Dave Andrews. And I've just told you a little bit about Web Search Spiders."