Search Engines: What is a Search Engine? | How do Search Engines Work? | Web Crawling / Spiders, Indexing and Searching | Online Libraries and Librarians
Get in Touch
Get a free web services quote or contact me to discuss your project.
Portfolio
See the sites written, edited and/or optimised by The Contented Website.
Search Engines
Return to the index of frequently asked questions about search engine optimisation.
What is a Search Engine?
Search engines are tools or devices, (comprising of spiders /crawlers, indexes, algorithms and search results) designed to search and retrieve information from the World Wide Web (WWW). A search engine searches the web looking for documents according to the specific keywords entered by the search engine user. The search engine then returns a list of the documents it has found on its search results pages (SERPS).
How a search engine works:
A search engine works by crawling, indexing and searching the web.
Crawling
This process knowing as crawling or spidering is really the online equivalent of a team of librarians going into an enormous warehouse (the WWW) filled with books, documents, files, pictures and videos and working out what material they have.
The contents of this material is then analysed by the web crawlers/librarians to determine how it should be indexed. They look at what data is available about the material (the web page Meta data) — data which is consciously (or sometimes unconsciously) supplied by the publishers. In relation to a book, this data could, for example, be its title, title page, index, chapter headings, blurb (jacket text), bibliography and, of course, the content of the pages itself.
With billions of different items to crawl or look at, you can imagine why this team of librarians/crawlers may, at times, skim over the data to glean the essentials. This is one reason why the title of every web page (or book) is the most important information that can be published to help search engine crawlers/librarians construct their index.
Indexing
When the search engine crawlers/librarians have worked out what material they have found on the web/in the warehouse, they then set about creating an index database. Search engines such as Google store all or part of the source page (the code of each web page) in what is called a cache, as well as information about the web pages. Other search engines, such as AltaVista store every word of every page they find.
Searching
When search engine users enter a search (or query) into a search engine, they typically use a keyword or keyword phrase. In response to this query, the search engine examines its index and provides a list of its best-matched or most relevant web pages. This list is presented on what is known as a SERP, a search engine results page and typically shows the web page or document's title, description or part of the text and location.
Each search engine employs its own set of algorithms (methods) to work out which pages are most relevant to a search engine user's initial query. From its early beginnings in 1990, the search engine market is now highly competitive as search engines compete for:
- a share of the extremely lucrative search engine market
- user loyalty
- the best search technology
- most enticing search (and search-related) services.
Glossary
If this all sounds like a foreign language to you, please don't stew about it. You can refer to the (easy-to-understand) glossary of terms for more information.
Store Them Away
If you want to keep any pages tucked away in your online pantry of goodies, simply bookmark the relevant page in your browser by clicking:
- 'Favourites' (if you're using Internet Explorer/IE)
- 'Bookmarks' (if you're using Firefox).