One of the coolest things about the Internet is that it seems to have answers to everything. If in your parents’ time you had to go to the library to satisfy your curiosity, today it only takes a few clicks to find thousands of pages on any topic. But have you ever stopped to think about how this magic happens?
Much of this process occurs invisibly, in three stages. The first step is to search for information on the Internet, even before searching. Search engine companies, such as Google, use computer programs called spiders that can automatically navigate the Internet.
When you access a website, these programs read all the available content, whether it is text, images or videos. This process must be done constantly, as there are new things on the Internet at all times.
After storing the content, comes the next step: indexing. At this stage, everything collected (whether texts, images or videos) is analyzed and stored in a huge list. It’s as if every content has meta tags.
The final step occurs when we type a question into the search engine. It reads all the words to understand what we want and then selects the relevant content in that big list. Finally, based on information like your language and location, it displays the results that make the most sense for you.
My past is more difficult
But it wasn’t always that simple. At the beginning of the Internet, in the 1990s, Google did not yet exist. Therefore, to find something on the Internet, many people had to turn to websites, such as Yahoo! And the site Cadê? Brazilian, which was a large list of smaller sites organized by topics, e.g. Everything was done by hand, which was a lot of work for everyone.
This began to change in 1993, when Wandex was launched, the first tool that automatically searched the Internet for new websites. In ten days, I was able to read about 25,000 pages – the entire Internet at that time – but it was nothing more than a list of pages.
Months later, JumpStation appeared, which in addition to searching and organizing pages, also had a search engine among the results. It was the first automated search engine similar to what Google is today.
What about artificial intelligence?
Currently, with the advancement of artificial intelligence, many people are switching from search engines to search engines
AI-powered chatbots, which can also display results in seconds. But their mechanism is different.
While search engines search the Internet and find content related to their search, AI is trained to read and interpret books, surveys or newspaper pages and, based on them, create a personalized result for the user. Moreover, it connects concepts. Want an example? Brazil became independent (historical fact) on September 7, 1822 (exact date).
The problem is that AI does not update itself. If a chatbot was created in 2018, for example, it wouldn’t know what happened after that year, such as the COVID-19 pandemic or who won the 2022 World Cup.
Therefore, many of them also search online to find more up-to-date content. So if someone asks “What’s the weather forecast for tomorrow?” or “What is the top news of the day?”, they will conduct an Internet search, often using the search engines themselves, to give you an answer.