Detail publikačního výsledku

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

MIN, S.; FAJČÍK, M.; DOČEKAL, M.; ONDŘEJ, K.; SMRŽ, P.

Originální název

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

Anglický název

NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

Druh

Stať ve sborníku v databázi WoS či Scopus

Originální abstrakt

We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and return natural language answers. The aim of the competition was to build systems that can predict correct answers while also satisfying strict on-disk memory budgets. These memory budgets were designed to encourage contestants to explore the trade-off between storing retrieval corpora or the parameters of learned models. In this report, we describe the motivation and organization of the competition, review the best submissions, and analyze system predictions to inform a discussion of evaluation for open-domain QA.

Anglický abstrakt

We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and return natural language answers. The aim of the competition was to build systems that can predict correct answers while also satisfying strict on-disk memory budgets. These memory budgets were designed to encourage contestants to explore the trade-off between storing retrieval corpora or the parameters of learned models. In this report, we describe the motivation and organization of the competition, review the best submissions, and analyze system predictions to inform a discussion of evaluation for open-domain QA.

Klíčová slova

question answering, QA, ODQA, efficientQA, memory, disk memory, budget, efficient parameter, retrieval corpora

Klíčová slova v angličtině

question answering, QA, ODQA, efficientQA, memory, disk memory, budget, efficient parameter, retrieval corpora

Autoři

MIN, S.; FAJČÍK, M.; DOČEKAL, M.; ONDŘEJ, K.; SMRŽ, P.

Rok RIV

2022

Vydáno

01.08.2021

Nakladatel

Proceedings of Machine Learning Research

Místo

online

Kniha

Proceedings of the NeurIPS 2020 Competition and Demonstration Track

Edice

Proceedings of Machine Learning Research

ISSN

2640-3498

Periodikum

Proceedings of Machine Learning Research

Svazek

133

Číslo

133

Stát

Spojené státy americké

Strany od

86

Strany do

111

Strany počet

25

URL

http://proceedings.mlr.press/v133/min21a/min21a.pdf

BibTex

@inproceedings{BUT175821,
  author="MIN, S. and FAJČÍK, M. and DOČEKAL, M. and ONDŘEJ, K. and SMRŽ, P.",
  title="NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned",
  booktitle="Proceedings of the NeurIPS 2020 Competition and Demonstration Track",
  year="2021",
  series="Proceedings of Machine Learning Research",
  journal="Proceedings of Machine Learning Research",
  volume="133",
  number="133",
  pages="86--111",
  publisher="Proceedings of Machine Learning Research",
  address="online",
  url="http://proceedings.mlr.press/v133/min21a/min21a.pdf"
}