TREC 2021 | dev | dev2 | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
AP |
nDCG@10 | RR@100 | R@100 | R@1K | RR@100 | R@1K | RR@100 | R@1K | ||||||
(1a) | BM25 original passage (k1=0.9, b=0.4) | 0.1357 | 0.4458 | 0.5060 | 0.3261 | 0.6149 | 0.0719 | 0.5733 | 0.0802 | 0.5839 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(1b) | BM25 augmented passage (k1=0.9, b=0.4) | 0.0977 | 0.3977 | 0.5303 | 0.2709 | 0.5835 | 0.0872 | 0.6925 | 0.0917 | 0.6933 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(1c) | BM25+RM3 original passage (k1=0.9, b=0.4) | 0.1632 | 0.4480 | 0.4925 | 0.3498 | 0.6619 | 0.0619 | 0.5933 | 0.0654 | 0.6049 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(1d) | BM25+RM3 augmented passage (k1=0.9, b=0.4) | 0.1050 | 0.3906 | 0.4906 | 0.2795 | 0.6268 | 0.0674 | 0.6859 | 0.0701 | 0.6838 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(2a) | BM25 w/ doc2query-T5 original passage (k1=0.9, b=0.4) | 0.1874 | 0.4816 | 0.6848 | 0.4076 | 0.7078 | 0.1072 | 0.7083 | 0.1123 | 0.7151 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(2b) | BM25 w/ doc2query-T5 augmented passage (k1=0.9, b=0.4) | 0.1649 | 0.4702 | 0.6391 | 0.3883 | 0.6962 | 0.1172 | 0.7647 | 0.1170 | 0.7659 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(2c) | BM25 w/ doc2query-T5 original passage (k1=0.9, b=0.4) | 0.2271 | 0.5099 | 0.6651 | 0.4444 | 0.7512 | 0.0948 | 0.7174 | 0.0985 | 0.7240 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(2d) | BM25 w/ doc2query-T5 augmented passage (k1=0.9, b=0.4) | 0.1932 | 0.4834 | 0.5882 | 0.4295 | 0.7668 | 0.0880 | 0.7625 | 0.0887 | 0.7649 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(3a) | uniCOIL (noexp): pre-encoded queries | 0.2193 | 0.5756 | 0.6991 | 0.4246 | 0.6897 | 0.1342 | 0.7010 | 0.1385 | 0.7114 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
(3b) | uniCOIL (w/ doc2query-T5): pre-encoded queries | 0.2538 | 0.6159 | 0.7311 | 0.4731 | 0.7551 | 0.1499 | 0.7616 | 0.1577 | 0.7671 | ||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
uniCOIL (noexp): on-the-fly query inference | 0.2194 | 0.5759 | 0.6991 | 0.4247 | 0.6893 | 0.1343 | 0.7010 | 0.1385 | 0.7114 | |||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|
||||||||||||||
uniCOIL (w/ doc2query-T5): on-the-fly query inference | 0.2539 | 0.6160 | 0.7311 | 0.4723 | 0.7560 | 0.1501 | 0.7613 | 0.1576 | 0.7676 | |||||
Command to generate run on TREC 2021 queries:
Evaluation commands:
Command to generate run on dev queries:
Evaluation commands:
Command to generate run on dev2 queries:
Evaluation commands:
|