Overview
The search engine consists of three main components:[12]
- 1) An agent is a search robot. It bypasses the network, downloads and analyzes documents. If a new link is found during site analysis, it falls into the list of web addresses of the robot. Search robots are of the following types: spiders - download sites like the user's browsers; Crawler - discover new, still unknown links based on the analysis of already known documents; indexers - analyze the detected web pages and add data to the index. Many deflated documents are divided into disjoint parts and are cleared from the markup.
- 2) Index is a database compiled by search engine indexing robots. Documents are searched in the index.
- 3) Search engine. The search request from the user is sent to the least loaded server after analyzing the load of the search system.[13][14] To provide such an opportunity, Yandex servers are clustered. Then, the user request is processed by a program called "Metapoisk".[14] Metapoisk analyzes the request in real time: it determines the geographic location of the user, conducts linguistic analysis, etc. The program also determines whether the request belongs to the category of the most popular or recently defined.The issuance of such requests for some time is stored in the memory (cache) of the metasearch, and in case of a match, previously saved results are displayed.[14] If the request is rare and there are no matches in the cache, the system redirects it to the Basic Search program. It analyzes the system index, which is also divided into different duplicate servers (this speeds up the procedure). Then the received information again falls into meta-search, the data is ranked and shown to the user in a final form.[13]
Indexing
In general, Yandex indexes the following file types:[15] html, pdf, rtf, doc, xls, ppt, docx, odt, odp, ods, odg, xlsx, pptx.
The search engine is also able to index text inside Shockwave Flash objects (if the text is not placed on the image itself), if these elements are transferred as a separate page, which has the MIME type - enter before the word, that should be in the document
Search results
Yandex, automatically, along with the original “exact form” of the query, searches for its various variations and formulations.[18]
The Yandex search takes into account the morphology of the Russian language, therefore, regardless of the form of the word in the search query, the search will be performed for all word forms. If morphological analysis is undesirable, you can put an exclamation mark (!) Before the word — the search in this case will show only the specific form of the word. In addition, the search query practically does not take into account the so-called stop-words, that is, prepositions, punctuation, pronouns, etc., due to their wide distribution[12]
As a rule, abbreviations are automatically disclosing, spelling is correcting. It also searches for synonyms (mobile - cellular). The extension of the original user request depends on the context. Expansion does not occur when a set of highly specialized terms, names of proper names of companies (for example, OJSC “Hippo” - OJSC “Hippopotamus”), adding the word “price”, in exact quotes (these are queries highlighted with typewriter quotes).[18]
Search results for each user are formed individually based on their location, language of a query, interests and preferences based on the results of previous and current search sessions.
Search results from the Yandex search engine tend to favor Russian media sources, including state media, and Yandex-delivered ads tend to promote misinformation and propaganda produced by more than half a dozen Russian-language news sites.[24][25]
One study found that Yandex-delivered adverts ran alongside false stories about US bioweapons labs in Ukraine, claims that Ukrainian President Volodymyr Zelenskiy is a drug user, and reports repeating Kremlin claims that the war against Ukraine is going entirely to plan. Other fake news promoted by Yandex ads referred to the Russian invasion by using Kremlin talking points, calling the war an “operation to denazify and demilitarise Ukraine”.[24] Another analysis found that Yandex directs Russian speakers worldwide to manipulated information and often to outright disinformation.[25]
Spam and virus protection
In 2013, Yandex was considered by some to be the safest search engine at the time and the third most secure among all web resources.[26] By 2016, Yandex had slipped down to third with Google being first.[27]
Checking web pages and warning users appeared on Yandex in 2009: since then, on the search results page, next to a dangerous site there is a note “This site may threaten the security of your computer”.[28] Two technologies at once are used to detect threats. The first was purchased from the American antivirus Sophos and based on a signature approach: that means, when accessing a web page, the antivirus system also accesses a database of already known viruses and malware. This approach is fast, but practically powerless against new viruses that have not yet entered the database. Therefore, Yandex along with the signature also uses its own antivirus complex, based on an analysis of the behavioral factor. The Yandex program, when accessing the site, checks whether the latter requested additional files from the browser, redirected it to an extraneous resource, etc. Thus, if information is received that the site begins to perform certain actions (cascading style sheets, JavaScript modules are launched and complete programs) without user permission, it is placed in the “black list” and in the database of virus signatures.
Search ranking
For a long time, the key ranking factor for Yandex was the number of third-party links to a particular site. Each page on the Internet was assigned a unique citation index, similar to the index for authors of scientific articles: the more links, the better. A similar mechanism was implemented in the Yandex and in the Google's PageRank. In order to prevent cheating, Yandex uses multivariate analysis, in which only 70 of the 800 factors are affected by the number of third-party links. Today, the content of the site and the presence or absence of keywords there, the ease of reading the text, the name of the domain, its history and the presence of multimedia content play a much greater role.
On 5 December 2013 Yandex announced a complete refusal of accounting the link factor in the future.[31]
Search hints
As the user types the query in the search bar, the search engine offers hints in the form of a drop-down list. Hints appear even before the search results appears and allow you to refine the query, correct the layout or typo, or go directly to the site you are looking for. For each user, hints are generated based on the history of their search queries using the My Finds service. In 2012, the so-called “Smart Search Hints” appeared, which instantly give out information about the main constants (equator length, speed of light, and so on), traffic jams, and have a built-in calculator. In addition, a translator was integrated in the “Hints” (the query “love in French” instantly gives out amour, affection), the schedule and results of football matches, exchange rates, weather forecasts and more. You can find out the exact time by asking "what time is it." In 2011, Hints in the search for Yandex became completely local to 83 regions of Russia.[32]
In addition to the actual search, Hints are built into Yandex search engines. Dictionaries ”,“ Yandex. Market ”,“ Yandex. Maps "and other Yandex services.
The hint function is a consequence of the development of the technology of intent search and first appeared on Yandex.Bar in August 2007, and in October 2008 it was introduced on the main page of the search engine.[33] Available both in the desktop and mobile versions of the site, Yandex shows its users more than a billion search hints per day[34]