Tutorial Session
🔍 Retrieve, 🧠 Rerank, 💬 Answer, ⚙️ Experiment
Hands-On IR Research with PyTerrier
Outline
01. Platform
- Introduction
- Data Model & Transformers
- Operators
- Evaluation & Experiments
- Platform extras
02. Retrieval Menagerie
- Lexical & Learned Sparse Retrieval
- Dense Retrieval & Multi-Vec Retrieval
- PRF & Re-Ranking
03. Generation Menagerie
- LLM Backends & Vanilla RAG
- Generation Evaluation
- Retrieval-as-a-tool
04. Conclusion
- Patterns & Anti-Patterns
- Wrap-up
Hands-on Materials
Notebook Demos
Notebook demos are hosted on Google Colab.
Operators
PyTerrier operators for composing transformers into retrieval and ranking pipelines.
Experiments
Declarative experiments, evaluation measures, significance testing, and reproducible comparisons.
Sparse Menagerie
Lexical retrieval and sparse retrieval examples across supported PyTerrier retrieval backends.
Dense Menagerie
Dense retrieval pipelines and vector-based retrieval demonstrations.
Learned Sparse
Learned sparse retrieval models and plugin-based examples.
ColBERT
Late-interaction and multi-vector dense retrieval demos, including ColBERT-style pipelines.
Rerankers
Neural reranking examples, including sequence-to-sequence and pipeline-based rerankers.
RAG
Retrieval-augmented generation, generation evaluation, iterative RAG, and retrieval-as-a-tool.
PRF
Sparse and dense pseudo-relevance feedback demonstrations.
Tutorial History
Previous Tutorials
This SIGIR edition builds on earlier PyTerrier tutorials while shifting the emphasis toward PyTerrier 1.0, modern retrieval workflows, RAG pipelines, and hands-on experimentation.
2021
ECIR 2021
IR From Bag-of-words to BERT and Beyond through Practical Experiments.
2021
CIKM 2021
A virtual edition covering practical IR experiments with PyTerrier and neural ranking methods.
2022
BCS IRSG Search Solutions
An in-person practical tutorial at the British Computer Society headquarters.
Code
GitHub Links
The PyTerrier ecosystem includes the core platform and plugins for dense retrieval, late interaction, RAG, reranking, and high-performance retrieval engines.
PyTerrierCore platform
PyTerrier ColBERTLate-interaction retrieval
PyTerrier RAGRAG pipelines and readers
PyTerrier T5MonoT5 and DuoT5 reranking
PyTerrier PISAFast in-memory retrieval
Terrier TeamPlugin ecosystem
PyTerrier CachingCaching intermediate results
Dense RetrievalDense retrieval extension docs
AnseriniAnserini backend integration
Reference
Documentation
Start from the official documentation for installation, data models, transformers, operators, experiments, extension packages, and troubleshooting.
PyTerrier Documentation
The documentation covers installation, importing datasets, Terrier indexing, running experiments, learning to rank, artifacts, pipeline operators, debugging, and extension packages.
People
Presenter Profiles
The tutorial is delivered by researchers with long-standing experience in information retrieval, PyTerrier, efficient retrieval systems, neural ranking, and RAG.
Craig Macdonald
University of Glasgow
Professor of Information Retrieval. His research focuses on efficient and effective search and recommendation, and he has extensive tutorial and teaching experience using PyTerrier.
Sean MacAvaney
University of Glasgow
Senior Lecturer whose research focuses on efficient neural models for search, learned sparse retrieval, reranking, and practical IR systems.
Nicola Tonellotto
University of Pisa
Associate Professor at the University of Pisa. His research focuses on efficient large-scale IR pipelines and data processing platforms.
Xiao Wang
University of International Business and Economics
Assistant Professor whose research focuses on efficient and effective neural information retrieval, including dense retrieval and neural pseudo-relevance feedback.
Citation
Retrieve, Rerank, Answer, Experiment: Hands-On IR Research with PyTerrier. Craig Macdonald, Sean MacAvaney, Nicola Tonellotto and Xiao Wang. In Proceedings of ACM SIGIR 2026. DOI: 10.1145/3805712.3808640 .
If you use this tutorial or its materials, please cite:
@inproceedings{macdonald2026pyterrierTutorial,
author = {Macdonald, Craig and MacAvaney, Sean and Tonellotto, Nicola and Wang, Xiao},
title = {Retrieve, Rerank, Answer, Experiment: Hands-On IR Research with PyTerrier},
booktitle = {Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '26)},
year = {2026},
location = {Melbourne, VIC, Australia},
doi = {10.1145/3805712.3808640},
isbn = {979-8-4007-2599-9/2026/07},
url = {https://doi.org/10.1145/3805712.3808640},
}