Overview
Web pages change over time: information may be updated, replaced, or removed. As a result, the current version of a Web page may no longer contain information that appeared in earlier versions, and different archived versions of the same page may contain different information.
Task 2 focuses on question answering over archived Web pages. Given a temporal question and access to a corpus of archived Web pages, systems must retrieve temporally relevant archive records, answer the question using the retrieved evidence, and identify the archive records that support the answer.
The task therefore combines three related challenges: temporal retrieval, temporal question answering, and evidence grounding. A successful system must identify not only relevant Web pages, but also the archived versions that correspond to the temporal requirements of the question.
Why does temporal Web QA matter?
Large language models (LLMs) and retrieval-augmented question-answering systems increasingly rely on Web information to answer questions. However, information on the Web is not static: pages are updated, facts change, and earlier versions of information may disappear from the current Web.
This creates an important challenge for questions about the past. A system may retrieve information that is highly relevant to the question but comes from the wrong point in time, for example from a page that was published or updated after the period being asked about. For temporal and historical questions, topical relevance alone is therefore not sufficient—the system must retrieve temporally appropriate versions of the relevant Web pages.
Web archives preserve captures of Web pages across time, making it possible to recover earlier states of online information. By combining archive retrieval with temporal reasoning, systems can answer questions about information represented at particular points in time, compare information across archived versions, verify whether a documented change occurred, and trace how information evolved across multiple observations.
Studying this capability is important for building QA and RAG systems that can reason over time-specific evidence, rather than treating the current Web as a complete representation of information from the past.
Task definition
Given a temporal question and access to the provided Web archive corpus, the goal is to retrieve the archive records needed to answer the question, generate the answer, and identify the records that support it.
The answer must address the temporal aspect of the question and be grounded in evidence from one or more archived Web pages in the provided corpus.
Systems must therefore perform three related steps:
- Temporal retrieval: identify archive records that are both topically relevant and appropriate for the time period specified by the question.
- Temporal question answering: use the retrieved evidence to answer the question about a particular point in time or about how information changed over time.
- Evidence grounding: provide the archive record IDs that support the answer.
The task focuses on question answering over historical Web information. Systems should answer using the provided archive corpus rather than relying only on the current Web.
Systems are not required to reproduce archived pages themselves; they must return the answer together with the supporting archive record IDs.
Input and output
Input
A temporal question and access to the provided Web archive corpus.
The corpus contains multiple archived versions of Web pages from different points in time. Systems must identify the archive records that are relevant to both the topic of the question and its temporal requirements.
Output
An answer to the question together with the archive record IDs that support the answer.
The answer should be based on evidence from the retrieved archived pages, and the cited records should correspond to the relevant time period or periods in the question.
Illustrative example
The following example investigates how the maximum award of the Future Web Research Grant changed over time. It illustrates the Compare versions question type, where the system must retrieve the most relevant archived versions of a Web page from two different points in time and use them to determine how the information changed.
Archived records
futureweb-2015: A 2015 capture of the Future Web Research Grant page lists application deadlines and eligibility requirements.futureweb-2016: A 2016 capture of the Future Web Research Grant page states a maximum award of $75,000.futureweb-2020: A 2020 capture of the Future Web Research Grant page states a maximum award of $100,000.
Question: How did the maximum award listed for the Future Web Research Grant change between the 2016 and 2020 captures?
Answer: It increased from $75,000 to $100,000.
Supporting record IDs: futureweb-2016, futureweb-2020
The system must identify the 2016 and 2020 records as the relevant evidence for the question. The 2015 record is related to the same grant page but does not contain the information needed to answer the question.
Question types
Point in time
Answer a question about information represented in the archive at a specific point in time or during a specified period.
Example: What was the maximum award listed for the Future Web Research Grant in 2018?
First or last observed
Identify the earliest or latest point in time at which a particular fact or piece of information is observed in the archive.
Example: When was the $100,000 maximum award first observed in the archive?
Compare versions
Compare how a particular piece of information differs between two specified points in time.
Example: How did the maximum award listed for the Future Web Research Grant change between 2016 and 2020?
Verify change
Determine whether a stated claim about a change is supported by evidence from the archive.
Example: Is the claim that the maximum award increased from $75,000 in 2016 to $100,000 in 2020 supported by the archive?
Trace changes
Reconstruct how a particular piece of information changed across multiple points in time.
Example: How did the maximum award listed for the Future Web Research Grant change between 2016, 2018, and 2020?
Web archive corpus
The task will use a curated corpus derived from the End of Term Web Archive collections. The corpus will contain archived Web pages from multiple time periods together with the temporal and provenance information needed to identify individual captures.
Record information
- Archive record identifier
- Source URL
- Capture timestamp
- Collection provenance
- Extracted page text
Participants will be able to build their own lexical, dense, or hybrid retrieval systems over the released corpus.
Evaluation
Systems will be evaluated on both answer correctness and the quality of the supporting archive evidence.
The exact evaluation measures will depend on the question type and will be described in the final task specification.
Baselines
Planned baselines include BM25 retrieval, standard retrieve-then-answer pipelines, and evidence-aware answer generation. Baseline outputs will preserve official record IDs.
Submission and participation
Each prediction will contain the question ID, answer, and supporting record IDs. Teams may submit up to three runs, subject to confirmation in the final rules.
The released corpus defines the official evidence universe. Rules for live-Web access, external models, and auxiliary resources will be stated before submissions open.
Important dates
| Corpus and train/dev release | October 2026 |
|---|---|
| Registration | To be announced |
| Test phase | To be announced |
| Results | To be announced |
Downloads and links
Release links will be published on the Resources page as materials become available. Planned resources include the archive corpus and metadata, question files, training and development evidence annotations, a BM25 index, baseline results, validation tools, and the TIRA link.
Task 2 FAQ
Must teams use the planned BM25 index?
No. Teams may build dense, sparse, or hybrid indexes over the frozen corpus while preserving official record IDs.
Why are record IDs required?
They make evidence evaluation reproducible and separate grounded answers from unsupported answers.
Can systems search the live Web?
The final system-boundary rules will state whether any external access is permitted. Official evidence must come from the frozen corpus.