Tuesday, May 29, 2012

How Search Engine Works

Search Engine Working Process

1. Web Crawling
2. Caching
3. Indexing
4. Searching


1. Web Crawling- Search Engine works by storing information about web pages. Crawlers are used to create a copy of visited web pages for search engines to provide fast searches. We can use robots.txt to prevent  crawlers to crawl web pages. Different search engines use different processes of crawling.

Search Engine        Crawlers
Google                  Google-bot
Yahoo                    yahoo! Slurp
Msn                       MSN bots
Altavista                Alta Vista bot
Ask                      Teoma bots

2. Caching-  Web cache is like a mediator between web server(main server) and client server and watches requests come by and saving the copies of responses such as images, html pages. Then If there is any request for same URL,  they can response that instead of asking main server again.

Types of Cache:-

1. Browser Cache 2. Proxy Cache 3. Gateway Cache

3. Indexing- Web Indexing is a process in which search engine store data to retrieve fast and accurate information. For example an index of 1000 documents can take milliseconds, but a sequential  scan take hours.

Three Distinct Parts Of Web Indexing-

1. Web crawler finds and fetches web documents.
2.Indexer sorts every word on every document and stores resulting index of words in huge database.
3.The query processor, which compares your search query to the index and recommends the documents that considers most relevant.

4. Searching- Web search is a query that user put into search engine to take most relevant information.

Thursday, May 24, 2012

About Web Search Engine

Web Search Engine(Search Engine) is a set of programs that  searches for information on the World Wide Web. Search engine works by storing information about web pages. They store information with the help of web crawlers(also called spider). Search results are presented as a list of results referred to as SERPs(search engine result page).


Search Engines Market Share May 2012-

Search Engine Market Share in 2011 Market Share in 2012
Google 82.80% 87.6%
Yahoo! 6.42% 4.89%
Bing 3.91% 4.2%
Baidu4.89%4.7%