TREC 2019 | TREC 2020 | dev | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
AP |
nDCG@10 | R@1K | AP |
nDCG@10 | R@1K | RR@10 | R@1K | ||||||
(1a) | BM25 (k1=0.9, b=0.4) | 0.3013 | 0.5058 | 0.7501 | 0.2856 | 0.4796 | 0.7863 | 0.1840 | 0.8526 | ||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
(1b) | BM25+RM3 (k1=0.9, b=0.4) | 0.3390 | 0.5180 | 0.7998 | 0.3019 | 0.4821 | 0.8217 | 0.1564 | 0.8606 | ||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
(2a) | BM25 w/ doc2query-T5 (k1=0.9, b=0.4) | 0.4034 | 0.6417 | 0.8310 | 0.4074 | 0.6187 | 0.8452 | 0.2723 | 0.9470 | ||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
(2b) | BM25+RM3 w/ doc2query-T5 (k1=0.9, b=0.4) | 0.4485 | 0.6548 | 0.8861 | 0.4295 | 0.6172 | 0.8699 | 0.2140 | 0.9463 | ||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
(3a) | uniCOIL (noexp): pre-encoded queries | 0.4033 | 0.6433 | 0.7752 | 0.4021 | 0.6523 | 0.7861 | 0.3153 | 0.9239 | ||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
(3b) | uniCOIL (w/ doc2query-T5): pre-encoded queries | 0.4612 | 0.7024 | 0.8292 | 0.4430 | 0.6745 | 0.8430 | 0.3516 | 0.9582 | ||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
BM25 (k1=0.82, b=0.68) | 0.2903 | 0.4973 | 0.7450 | 0.2876 | 0.4876 | 0.8031 | 0.1875 | 0.8573 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
BM25+RM3 (k1=0.82, b=0.68) | 0.3377 | 0.5231 | 0.7792 | 0.3056 | 0.4808 | 0.8286 | 0.1668 | 0.8687 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
BM25 w/ doc2query-T5 (k1=2.18, b=0.86) | 0.4046 | 0.6336 | 0.8134 | 0.4171 | 0.6265 | 0.8393 | 0.2816 | 0.9506 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
BM25+RM3 w/ doc2query-T5 (k1=2.18, b=0.86) | 0.4360 | 0.6528 | 0.8424 | 0.4347 | 0.6232 | 0.8609 | 0.2374 | 0.9528 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
uniCOIL (noexp): on-the-fly query inference | 0.4033 | 0.6434 | 0.7752 | 0.4022 | 0.6524 | 0.7861 | 0.3153 | 0.9239 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
uniCOIL (w/ doc2query-T5): on-the-fly query inference | 0.4617 | 0.7027 | 0.8291 | 0.4429 | 0.6745 | 0.8433 | 0.3509 | 0.9581 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
TCT_ColBERT-V2-HN+: pre-encoded queries | - | - | - | - | - | - | 0.3584 | 0.9695 | |||||
Not available.
Not available.
Command to generate run on dev queries:
Evaluation commands:
|
|||||||||||||
TCT_ColBERT-V2-HN+: on-the-fly query inference | 0.4469 | 0.7204 | 0.8261 | 0.4754 | 0.6882 | 0.8429 | 0.3584 | 0.9695 | |||||
Command to generate run on TREC 2019 queries:
Evaluation commands:
Command to generate run on TREC 2020 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
|