I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension

Chuanqi Tan; Furu Wei; Qingyu Zhou; Nan Yang; Weifeng Lv; Ming Zhou

doi:10.1007/978-3-319-99495-6_8

I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension

Chuanqi Tan^*
, Furu Wei
, Qingyu Zhou
, Nan Yang
, Weifeng Lv
, Ming Zhou

^*Corresponding author for this work

School of Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

20 Scopus citations

Abstract

Existing works on machine reading comprehension mostly focus on extracting text spans from passages with the assumption that the passage must contain the answer to the question. This assumption usually cannot be satisfied in real-life applications. In this paper, we study the reading comprehension task in which whether the given passage contains the answer is not specified in advance. The system needs to correctly refuse to give an answer when a passage does not contain the answer. We develop several baselines including the answer extraction based method and the passage triggering based method to address this task. Furthermore, we propose an answer validation model that first extracts the answer and then validates whether it is correct. To evaluate these methods, we build a dataset SQuAD-T based on the SQuAD dataset, which consists of questions in the SQuAD dataset and includes relevant passages that may not contain the answer. We report results on this dataset and provides comparisons and analysis of the different models.

Original language	English
Title of host publication	Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Proceedings
Editors	Dongyan Zhao, Sujian Li, Min Zhang, Vincent Ng, Hongying Zan
Publisher	Springer Verlag
Pages	85-97
Number of pages	13
ISBN (Print)	9783319994949
DOIs	https://doi.org/10.1007/978-3-319-99495-6_8
State	Published - 2018
Event	7th CCF International Conference on Natural Language Processing and Chinese Computing, NLPCC 2018 - Hohhot, China Duration: 26 Aug 2018 → 30 Aug 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11108 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	7th CCF International Conference on Natural Language Processing and Chinese Computing, NLPCC 2018
Country/Territory	China
City	Hohhot
Period	26/08/18 → 30/08/18

Keywords

Answer validation
Machine reading comprehension

Access to Document

10.1007/978-3-319-99495-6_8

Cite this

Tan, C., Wei, F., Zhou, Q., Yang, N., Lv, W., & Zhou, M. (2018). I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension. In D. Zhao, S. Li, M. Zhang, V. Ng, & H. Zan (Eds.), Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Proceedings (pp. 85-97). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11108 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-99495-6_8

Tan, Chuanqi ; Wei, Furu ; Zhou, Qingyu et al. / I Know There Is No Answer : Modeling Answer Validation for Machine Reading Comprehension. Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Proceedings. editor / Dongyan Zhao ; Sujian Li ; Min Zhang ; Vincent Ng ; Hongying Zan. Springer Verlag, 2018. pp. 85-97 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{bdb63c12b3144a7283a28b09d632fe19,

title = "I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension",

abstract = "Existing works on machine reading comprehension mostly focus on extracting text spans from passages with the assumption that the passage must contain the answer to the question. This assumption usually cannot be satisfied in real-life applications. In this paper, we study the reading comprehension task in which whether the given passage contains the answer is not specified in advance. The system needs to correctly refuse to give an answer when a passage does not contain the answer. We develop several baselines including the answer extraction based method and the passage triggering based method to address this task. Furthermore, we propose an answer validation model that first extracts the answer and then validates whether it is correct. To evaluate these methods, we build a dataset SQuAD-T based on the SQuAD dataset, which consists of questions in the SQuAD dataset and includes relevant passages that may not contain the answer. We report results on this dataset and provides comparisons and analysis of the different models.",

keywords = "Answer validation, Machine reading comprehension",

author = "Chuanqi Tan and Furu Wei and Qingyu Zhou and Nan Yang and Weifeng Lv and Ming Zhou",

note = "Publisher Copyright: {\textcopyright} 2018, Springer Nature Switzerland AG.; 7th CCF International Conference on Natural Language Processing and Chinese Computing, NLPCC 2018 ; Conference date: 26-08-2018 Through 30-08-2018",

year = "2018",

doi = "10.1007/978-3-319-99495-6\_8",

language = "英语",

isbn = "9783319994949",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "85--97",

editor = "Dongyan Zhao and Sujian Li and Min Zhang and Vincent Ng and Hongying Zan",

booktitle = "Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Proceedings",

address = "德国",

}

Tan, C, Wei, F, Zhou, Q, Yang, N, Lv, W & Zhou, M 2018, I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension. in D Zhao, S Li, M Zhang, V Ng & H Zan (eds), Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11108 LNAI, Springer Verlag, pp. 85-97, 7th CCF International Conference on Natural Language Processing and Chinese Computing, NLPCC 2018, Hohhot, China, 26/08/18. https://doi.org/10.1007/978-3-319-99495-6_8

I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension. / Tan, Chuanqi; Wei, Furu; Zhou, Qingyu et al.
Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Proceedings. ed. / Dongyan Zhao; Sujian Li; Min Zhang; Vincent Ng; Hongying Zan. Springer Verlag, 2018. p. 85-97 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11108 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - I Know There Is No Answer

T2 - 7th CCF International Conference on Natural Language Processing and Chinese Computing, NLPCC 2018

AU - Tan, Chuanqi

AU - Wei, Furu

AU - Zhou, Qingyu

AU - Yang, Nan

AU - Lv, Weifeng

AU - Zhou, Ming

PY - 2018

Y1 - 2018

N2 - Existing works on machine reading comprehension mostly focus on extracting text spans from passages with the assumption that the passage must contain the answer to the question. This assumption usually cannot be satisfied in real-life applications. In this paper, we study the reading comprehension task in which whether the given passage contains the answer is not specified in advance. The system needs to correctly refuse to give an answer when a passage does not contain the answer. We develop several baselines including the answer extraction based method and the passage triggering based method to address this task. Furthermore, we propose an answer validation model that first extracts the answer and then validates whether it is correct. To evaluate these methods, we build a dataset SQuAD-T based on the SQuAD dataset, which consists of questions in the SQuAD dataset and includes relevant passages that may not contain the answer. We report results on this dataset and provides comparisons and analysis of the different models.

AB - Existing works on machine reading comprehension mostly focus on extracting text spans from passages with the assumption that the passage must contain the answer to the question. This assumption usually cannot be satisfied in real-life applications. In this paper, we study the reading comprehension task in which whether the given passage contains the answer is not specified in advance. The system needs to correctly refuse to give an answer when a passage does not contain the answer. We develop several baselines including the answer extraction based method and the passage triggering based method to address this task. Furthermore, we propose an answer validation model that first extracts the answer and then validates whether it is correct. To evaluate these methods, we build a dataset SQuAD-T based on the SQuAD dataset, which consists of questions in the SQuAD dataset and includes relevant passages that may not contain the answer. We report results on this dataset and provides comparisons and analysis of the different models.

KW - Answer validation

KW - Machine reading comprehension

UR - https://www.scopus.com/pages/publications/85052940360

U2 - 10.1007/978-3-319-99495-6_8

DO - 10.1007/978-3-319-99495-6_8

M3 - 会议稿件

AN - SCOPUS:85052940360

SN - 9783319994949

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 85

EP - 97

BT - Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Proceedings

A2 - Zhao, Dongyan

A2 - Li, Sujian

A2 - Zhang, Min

A2 - Ng, Vincent

A2 - Zan, Hongying

PB - Springer Verlag

Y2 - 26 August 2018 through 30 August 2018

ER -

Tan C, Wei F, Zhou Q, Yang N, Lv W, Zhou M. I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension. In Zhao D, Li S, Zhang M, Ng V, Zan H, editors, Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Proceedings. Springer Verlag. 2018. p. 85-97. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-99495-6_8

I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this