Efficient and Flexible Information Retrieval Using a Relational Database

Posted in Conferences, Companies, Databases, Science, Development on February 01, 2007

Efficient and Flexible Information Retrieval Using a Relational Database
Google Tech Talks
January 11, 2007


Today's large-scale IR systems are not implemented using general-purpose database systems, as the latter tend to be significantly less efficient than custom-built IR engines. This talk demonstrates how recent developments in hardware-conscious database architecture may however satisfy IR needs. The advantage is flexibility of experimentation, as implementing a retrieval system on top of a DBMS boils down to relational query formulation, rather than system programming. We demonstrate in the context of the TeraByte TREC efficiency task that our experimental MonetDB/X100 database system provides highly competitive results both regarding precision and speed. We analyze the two innovations in MonetDB/X100 that most contributed to this successful application of DB technology in IR, namely vectorized in-cache processing and the use of two new light-weight compression schemes that work between the RAM and CPU cache memory levels. This talk is based on the demo presented at CIDR2007.

Watch Video

Tags: Techtalks, Google, Conferences, Databases, Science, Lectures, CIDR2007, Computer Science, Broadcasting, Development, Companies