Welcome to Virtual IR Lab

Virtual Information Retrieval Lab (VIRLab) is a web-based virtual laboratory for Information Retrieval. It is a unique and novel tool that can help teaching IR models, improving the productivity for doing IR model research, as well as promoting controlled experimental study of IR models.

  • Implementation of a retrieval function

  • Management and evaluation of retrieval functions

  • Pair-wise result comparison of search engines

  • Leaderboard

  • Configured search engines for implemented retrieval functions

  • Per-query effectiveness comparison of two retrieval functions

Easy implementation of retrieval functions

Users only need to write a few lines of code through a Web form to combine statistics retrieved from the indexes without worrying about how to access the indexes. The code will be automatically checked for syntax errors and translated to an executable to rank documents.

Flexible configuration of search engines

Users can configure a search engine by selecting a retrieval function and a test collection. Multiple search engines can be easily created in the same time. Moreover, users can compare two search engines side by side to figure out their ranking differences.

Simplified evaluation process

After creating a retrieval function, the users can evaluate its effectiveness over a few provided test collections by simply clicking a button. If a retrieval function contains multiple parameter values, the users may select to evaluate all of them. Moreover, a leader-board is created for each collection so that the most effective 10 retrieval functions are displayed.

Straightfoward result analysis interface

Users can easily analyze each retrieval function through its corresponding operational search engines over various collections. Specifically, if a search engine is configured using an existing test collection with relevance judgments, the official queries and judgments will be displayed so that the users can easily analyze the search results to figure out when the search engine fails and why. Moreover, the users can also compare the search results of two search engines side by side to figure out their ranking differences.

Demo System

Here is a demo VIRLab system that you can try out.

A tutorial is available after you log in to the system.

For Research

Here is the link to VIRLab-R, a satellite site of VIRLab for studying IR models.

VIRLab-R aims to improve the productivity of doing IR model research and promote controlled experimental study of IR models.

We are always in the process of implementing more existing IR models, and here is a list of models that have been implemented in the system. Please feel free to use the system for your own research on IR models and contribute your implementation of IR models.

For Teaching

References