Tag(s) / Article: How Does a Search Engine Work
Posted on 4/27/2008 4:34:51 PM
How Does a Search Engine Work?
Basically the search engines read pages that have been submitted to it via a submission form. From there it reads the entire page, boils it down to essential keywords and adds all the links it finds to the search list. Then it takes the next page off the list and starts all over again. You can see that just one submission to a search engine can generate thousands of pages in its database. All pages on the Internet are subject to being scanned by a search engine. If you don't want your pages to be scanned you have to take steps to prevent it. You can do this with Meta Tags or a small file named robot.txt located on your site.