Submission platform
TIRA will be used for official run submission to support comparable and reproducible evaluation. The TemporalWebLab TIRA links will be added when the tasks open.
Not yet open: Do not prepare final uploads from the provisional examples below. Machine-readable schemas and validators will accompany the official release.
Task 1 format
Each test question will require exactly one prediction from the ordered label set.
| Field | Meaning |
|---|---|
question_id | Identifier supplied with the test question. |
prediction | One valid recency-class label. |
Predictions with unknown IDs, missing values, or labels outside the official taxonomy will fail validation.
Task 2 format
Each prediction will pair an answer with one or more supporting record identifiers from the frozen corpus.
| Field | Meaning |
|---|---|
question_id | Identifier supplied with the test question. |
answer | System answer in the required normalized form. |
evidence_record_ids | Stable corpus records supporting the answer. |
Run policy
- The current plan permits up to three runs per task.
- Teams will identify one primary run for official reporting.
- Every run must have a unique name and a documented configuration.
- Late or malformed runs may be excluded from official results.
Validation checklist
- Use the official file encoding and schema.
- Include every required test identifier exactly once.
- Use only allowed Task 1 labels or valid Task 2 record IDs.
- Run the released validator locally.
- Confirm the intended primary run before the deadline.