As user, it is very very easy for us to use search engine; simply type the keyword, press enter and let the search engine do their magic. In fractions of second or maybe few seconds (depending on how lucky we are in getting hands on fast internet connection) the search result are displayed on the screen. We simply forgot that there is such complicated process behind this convenient search engine that we use almost everyday.
In fact, yes, there’s a very complicated, elaborate and sophisticated chain of processes behind search engine and these processes are undergoing constant tweaks and improvements everyday. But, simply put, there are three main processes that would describe how search engine works:
- Step 1: Crawling and Indexing
Before a user starts searching for some particular keywords, search engine has already did his work. The first step is where search engine send bots (which is some autonomous program on a network that can interact with computer systems or users) to navigate and crawls [almost] each and every pages internet. These bot’s main task is to gather information about the structure and content of that particular page, following each and every link from page to page. These are not easy task, because by first quarter of 2016 there are 60 TRILLIONS individual page in the internet (and this number is constantly growing). These vast information are then stored in a huge index. The size of this index by the first quarter of 2016 is about 100 MILLION GIGABYTES an of course it is still constantly growing along with the increasing number of individual page in the internet.
- Step 2: Ranking
When a user type particular keyword, this very particular keyword itself is processed. Search engine will check for its spelling, suggest the correct spelling of necessary, finding its synonyms, and try to understand it contextually rather than literally if possible. These processes are aimed to provide user with better search result. Afterwards, search engine will search that processed keyword within its huge index in order to find individual pages on the internet that contain information regarding that particular keyword. This are not done yet because search engine had to rank those page based on its relevancy according to user’s keyword. As if this process is not sophisticated enough, search engine use almost 200 FACTORS to rank their search result. I have no other words to describe this than ‘WOW!’. Note that the algorithm that Google use to do this is called PageRank. Search for ‘pagerank’ and you’ll get more info about it.
- Step 3: Junk Removal
Well, we should bear in mind that not all pages in the internet contains useful information, sometimes the contain junk or even harmful stuff you’d want to stay away from them. So to protect their user from such pages, the last step before search engine display their search result on user’s screen is to filter those search result from unwanted result. If search engine find suspicious page within its search result, it will be removed and site owner will be notified.
Amazing right? Remember that this information is valid when I wrote it but it maybe will dramatically change. If you’re interested on current information regarding how search engine (particularly Google) works please visit this link. I hope you’ll find this useful 🙂