the anatomy of a large-scale hypertextual web search engine

hace un momento estaba escribiendo sobre google como calculadora y, al hilo de mi reflexión sobre su nueva estrategia de servicios, he recordado una referencia muy interesante para los apasionados de las nuevas tenologías y la sociedad de la información. hagamos un poco de historia, y leed del puño y letra de los artífices de google (sergey brin y lawrence page) el artículo en el que desde sus trabajos de doctorado para la universidad de stanford, diseñan un nuevo concepto de buscador: the anatomy of a large-scale hypertextual web search engine. de su abstract:

in this paper, we present google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext. google is designed to crawl and index the web efficiently and produce much more satisfying search results than existing systems. the prototype with a full text and hyperlink database of at least 24 million pages is available at http://google.stanford.edu/

to engineer a search engine is a challenging task. search engines index tens to hundreds of millions of web pages involving a comparable number of distinct terms. they answer tens of millions of queries every day. despite the importance of large-scale search engines on the web, very little academic research has been done on them. furthermore, due to rapid advance in technology and web proliferation, creating a web search engine today is very different from three years ago. this paper provides an in-depth description of our large-scale web search engine — the first such detailed public description we know of to date. apart from the problems of scaling traditional search techniques to data of this magnitude, there are new technical challenges involved with using the additional information present in hypertext to produce better search results. this paper addresses this question of how to build a practical large-scale system which can exploit the additional information present in hypertext. also we look at the problem of how to effectively deal with uncontrolled hypertext collections where anyone can publish anything they want.

si quieres leer de primera mano el desarrollo teórico incluído el sistema ideado para capturar la información e indexarla en un proceso automatizado o por ejemplo el tan usado ya concepto de page rank, no puedes no ojear las 20 paginitas que lo componen. disfruta porque no es fácil ni frecuente ir de museos en internet

Tagged as: buscadores, google, investigación, lecturas