TREC Implementation

We implement an ad-hoc search algorithm based on some of the top performing algorithms in use at TREC. The details of our implementation are discussed in Section 4. To be fair and to make sure that our implementation of the TREC ad-hoc algorithm is not broken, we test our implementation on several TREC ad-hoc tasks and verify that it is indeed state-of-the-art (see Section 4). Our evaluation procedure needs to accommodate the fact that the TREC ad-hoc algorithms retrieve pages, not sites, whereas most of the queries used in this study seek particular sites. Our implementation, like any other keyword-based system, has a tendency to retrieve multiple pages from a site. To have a site-oriented retrieval, we group the pages by the host they reside on and select the top twenty sites for evaluation. A site/host is given the same rank as the rank of the best page residing on that site. This is in-line with what many commercial search engines do.